[==========] Running 7 tests from 2 test suites.
[----------] Global test environment set-up.
[----------] 5 tests from LeaderRebalancerTest
[ RUN ] LeaderRebalancerTest.FunctionalTestForDivided
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20250109 08:56:27.702215 20363 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.19.226.254:39769
I20250109 08:56:27.703608 20363 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250109 08:56:27.704612 20363 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250109 08:56:27.718905 20369 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250109 08:56:27.719821 20372 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250109 08:56:27.719895 20370 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250109 08:56:27.721210 20363 server_base.cc:1034] running on GCE node
I20250109 08:56:28.825237 20363 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250109 08:56:28.825511 20363 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250109 08:56:28.825671 20363 hybrid_clock.cc:648] HybridClock initialized: now 1736412988825655 us; error 0 us; skew 500 ppm
I20250109 08:56:28.826465 20363 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250109 08:56:28.832113 20363 webserver.cc:458] Webserver started at http://127.19.226.254:37211/ using document root <none> and password file <none>
I20250109 08:56:28.833019 20363 fs_manager.cc:362] Metadata directory not provided
I20250109 08:56:28.833217 20363 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250109 08:56:28.833603 20363 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250109 08:56:28.843829 20363 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1736412987538469-20363-0/minicluster-data/master-0-root/instance:
uuid: "756e7d8e818247aa9406c4a737856ce4"
format_stamp: "Formatted at 2025-01-09 08:56:28 on dist-test-slave-g8ds"
I20250109 08:56:28.852063 20363 fs_manager.cc:696] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20250109 08:56:28.857540 20378 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250109 08:56:28.858903 20363 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.003s sys 0.002s
I20250109 08:56:28.859232 20363 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1736412987538469-20363-0/minicluster-data/master-0-root
uuid: "756e7d8e818247aa9406c4a737856ce4"
format_stamp: "Formatted at 2025-01-09 08:56:28 on dist-test-slave-g8ds"
I20250109 08:56:28.859565 20363 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1736412987538469-20363-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1736412987538469-20363-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1736412987538469-20363-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250109 08:56:28.927615 20363 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250109 08:56:28.929149 20363 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250109 08:56:28.929602 20363 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250109 08:56:28.996500 20363 rpc_server.cc:307] RPC server started. Bound to: 127.19.226.254:39769
I20250109 08:56:28.996645 20429 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.226.254:39769 every 8 connection(s)
I20250109 08:56:29.003218 20430 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:56:29.026322 20430 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 756e7d8e818247aa9406c4a737856ce4: Bootstrap starting.
I20250109 08:56:29.031931 20430 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 756e7d8e818247aa9406c4a737856ce4: Neither blocks nor log segments found. Creating new log.
I20250109 08:56:29.033699 20430 log.cc:826] T 00000000000000000000000000000000 P 756e7d8e818247aa9406c4a737856ce4: Log is configured to *not* fsync() on all Append() calls
I20250109 08:56:29.038075 20430 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 756e7d8e818247aa9406c4a737856ce4: No bootstrap required, opened a new log
I20250109 08:56:29.060114 20430 raft_consensus.cc:357] T 00000000000000000000000000000000 P 756e7d8e818247aa9406c4a737856ce4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "756e7d8e818247aa9406c4a737856ce4" member_type: VOTER }
I20250109 08:56:29.060688 20430 raft_consensus.cc:383] T 00000000000000000000000000000000 P 756e7d8e818247aa9406c4a737856ce4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:56:29.060887 20430 raft_consensus.cc:738] T 00000000000000000000000000000000 P 756e7d8e818247aa9406c4a737856ce4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 756e7d8e818247aa9406c4a737856ce4, State: Initialized, Role: FOLLOWER
I20250109 08:56:29.061676 20430 consensus_queue.cc:260] T 00000000000000000000000000000000 P 756e7d8e818247aa9406c4a737856ce4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "756e7d8e818247aa9406c4a737856ce4" member_type: VOTER }
I20250109 08:56:29.062155 20430 raft_consensus.cc:397] T 00000000000000000000000000000000 P 756e7d8e818247aa9406c4a737856ce4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250109 08:56:29.062386 20430 raft_consensus.cc:491] T 00000000000000000000000000000000 P 756e7d8e818247aa9406c4a737856ce4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250109 08:56:29.062676 20430 raft_consensus.cc:3054] T 00000000000000000000000000000000 P 756e7d8e818247aa9406c4a737856ce4 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:56:29.066510 20430 raft_consensus.cc:513] T 00000000000000000000000000000000 P 756e7d8e818247aa9406c4a737856ce4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "756e7d8e818247aa9406c4a737856ce4" member_type: VOTER }
I20250109 08:56:29.067160 20430 leader_election.cc:304] T 00000000000000000000000000000000 P 756e7d8e818247aa9406c4a737856ce4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 756e7d8e818247aa9406c4a737856ce4; no voters:
I20250109 08:56:29.068840 20430 leader_election.cc:290] T 00000000000000000000000000000000 P 756e7d8e818247aa9406c4a737856ce4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250109 08:56:29.069203 20433 raft_consensus.cc:2798] T 00000000000000000000000000000000 P 756e7d8e818247aa9406c4a737856ce4 [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:56:29.071588 20433 raft_consensus.cc:695] T 00000000000000000000000000000000 P 756e7d8e818247aa9406c4a737856ce4 [term 1 LEADER]: Becoming Leader. State: Replica: 756e7d8e818247aa9406c4a737856ce4, State: Running, Role: LEADER
I20250109 08:56:29.072438 20433 consensus_queue.cc:237] T 00000000000000000000000000000000 P 756e7d8e818247aa9406c4a737856ce4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "756e7d8e818247aa9406c4a737856ce4" member_type: VOTER }
I20250109 08:56:29.073560 20430 sys_catalog.cc:564] T 00000000000000000000000000000000 P 756e7d8e818247aa9406c4a737856ce4 [sys.catalog]: configured and running, proceeding with master startup.
I20250109 08:56:29.083146 20434 sys_catalog.cc:455] T 00000000000000000000000000000000 P 756e7d8e818247aa9406c4a737856ce4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "756e7d8e818247aa9406c4a737856ce4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "756e7d8e818247aa9406c4a737856ce4" member_type: VOTER } }
I20250109 08:56:29.083907 20434 sys_catalog.cc:458] T 00000000000000000000000000000000 P 756e7d8e818247aa9406c4a737856ce4 [sys.catalog]: This master's current role is: LEADER
I20250109 08:56:29.084890 20435 sys_catalog.cc:455] T 00000000000000000000000000000000 P 756e7d8e818247aa9406c4a737856ce4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 756e7d8e818247aa9406c4a737856ce4. Latest consensus state: current_term: 1 leader_uuid: "756e7d8e818247aa9406c4a737856ce4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "756e7d8e818247aa9406c4a737856ce4" member_type: VOTER } }
I20250109 08:56:29.085399 20435 sys_catalog.cc:458] T 00000000000000000000000000000000 P 756e7d8e818247aa9406c4a737856ce4 [sys.catalog]: This master's current role is: LEADER
I20250109 08:56:29.090032 20443 catalog_manager.cc:1476] Loading table and tablet metadata into memory...
I20250109 08:56:29.099587 20443 catalog_manager.cc:1485] Initializing Kudu cluster ID...
I20250109 08:56:29.101408 20363 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20250109 08:56:29.115278 20443 catalog_manager.cc:1348] Generated new cluster ID: 8669f387c9ab4c9e9c70e30d9d43fe80
I20250109 08:56:29.115599 20443 catalog_manager.cc:1496] Initializing Kudu internal certificate authority...
I20250109 08:56:29.145138 20443 catalog_manager.cc:1371] Generated new certificate authority record
I20250109 08:56:29.146687 20443 catalog_manager.cc:1505] Loading token signing keys...
I20250109 08:56:29.162292 20443 catalog_manager.cc:5899] T 00000000000000000000000000000000 P 756e7d8e818247aa9406c4a737856ce4: Generated new TSK 0
I20250109 08:56:29.163215 20443 catalog_manager.cc:1515] Initializing in-progress tserver states...
I20250109 08:56:29.168754 20363 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250109 08:56:29.177810 20452 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250109 08:56:29.178900 20453 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250109 08:56:29.182466 20455 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250109 08:56:29.183473 20363 server_base.cc:1034] running on GCE node
I20250109 08:56:29.184392 20363 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250109 08:56:29.184662 20363 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250109 08:56:29.184859 20363 hybrid_clock.cc:648] HybridClock initialized: now 1736412989184844 us; error 0 us; skew 500 ppm
I20250109 08:56:29.185631 20363 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250109 08:56:29.189718 20363 webserver.cc:458] Webserver started at http://127.19.226.193:36597/ using document root <none> and password file <none>
I20250109 08:56:29.190411 20363 fs_manager.cc:362] Metadata directory not provided
I20250109 08:56:29.190659 20363 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250109 08:56:29.190986 20363 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250109 08:56:29.192481 20363 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1736412987538469-20363-0/minicluster-data/ts-0-root/instance:
uuid: "ff76b9242fda4d4586c2bb3da829b1ba"
format_stamp: "Formatted at 2025-01-09 08:56:29 on dist-test-slave-g8ds"
I20250109 08:56:29.199394 20363 fs_manager.cc:696] Time spent creating directory manager: real 0.006s user 0.001s sys 0.004s
I20250109 08:56:29.204325 20460 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250109 08:56:29.205202 20363 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20250109 08:56:29.205554 20363 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1736412987538469-20363-0/minicluster-data/ts-0-root
uuid: "ff76b9242fda4d4586c2bb3da829b1ba"
format_stamp: "Formatted at 2025-01-09 08:56:29 on dist-test-slave-g8ds"
I20250109 08:56:29.205929 20363 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1736412987538469-20363-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1736412987538469-20363-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1736412987538469-20363-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250109 08:56:29.222777 20363 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250109 08:56:29.224527 20363 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250109 08:56:29.227213 20363 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250109 08:56:29.232642 20363 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250109 08:56:29.232964 20363 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250109 08:56:29.233271 20363 ts_tablet_manager.cc:610] Registered 0 tablets
I20250109 08:56:29.233492 20363 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250109 08:56:29.319922 20363 rpc_server.cc:307] RPC server started. Bound to: 127.19.226.193:42763
I20250109 08:56:29.319967 20522 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.226.193:42763 every 8 connection(s)
I20250109 08:56:29.327749 20363 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250109 08:56:29.338384 20527 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250109 08:56:29.340106 20528 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250109 08:56:29.349538 20523 heartbeater.cc:346] Connected to a master server at 127.19.226.254:39769
I20250109 08:56:29.350039 20523 heartbeater.cc:463] Registering TS with master...
I20250109 08:56:29.351191 20523 heartbeater.cc:510] Master 127.19.226.254:39769 requested a full tablet report, sending...
W20250109 08:56:29.351781 20531 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250109 08:56:29.352432 20363 server_base.cc:1034] running on GCE node
I20250109 08:56:29.353230 20363 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250109 08:56:29.353430 20363 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250109 08:56:29.353561 20363 hybrid_clock.cc:648] HybridClock initialized: now 1736412989353550 us; error 0 us; skew 500 ppm
I20250109 08:56:29.354076 20363 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250109 08:56:29.354590 20395 ts_manager.cc:194] Registered new tserver with Master: ff76b9242fda4d4586c2bb3da829b1ba (127.19.226.193:42763)
I20250109 08:56:29.357610 20395 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:34500
I20250109 08:56:29.358584 20363 webserver.cc:458] Webserver started at http://127.19.226.194:46471/ using document root <none> and password file <none>
I20250109 08:56:29.359113 20363 fs_manager.cc:362] Metadata directory not provided
I20250109 08:56:29.359370 20363 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250109 08:56:29.359679 20363 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250109 08:56:29.360818 20363 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1736412987538469-20363-0/minicluster-data/ts-1-root/instance:
uuid: "b17b595d5da94e5dbf96bbef1ef3e668"
format_stamp: "Formatted at 2025-01-09 08:56:29 on dist-test-slave-g8ds"
I20250109 08:56:29.366041 20363 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20250109 08:56:29.369926 20535 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250109 08:56:29.370738 20363 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20250109 08:56:29.370995 20363 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1736412987538469-20363-0/minicluster-data/ts-1-root
uuid: "b17b595d5da94e5dbf96bbef1ef3e668"
format_stamp: "Formatted at 2025-01-09 08:56:29 on dist-test-slave-g8ds"
I20250109 08:56:29.371268 20363 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1736412987538469-20363-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1736412987538469-20363-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1736412987538469-20363-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250109 08:56:29.384864 20363 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250109 08:56:29.386132 20363 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250109 08:56:29.388267 20363 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250109 08:56:29.391153 20363 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250109 08:56:29.391366 20363 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250109 08:56:29.391590 20363 ts_tablet_manager.cc:610] Registered 0 tablets
I20250109 08:56:29.391738 20363 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250109 08:56:29.479032 20363 rpc_server.cc:307] RPC server started. Bound to: 127.19.226.194:42855
I20250109 08:56:29.479207 20597 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.226.194:42855 every 8 connection(s)
I20250109 08:56:29.485775 20363 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250109 08:56:29.493927 20601 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250109 08:56:29.498258 20598 heartbeater.cc:346] Connected to a master server at 127.19.226.254:39769
I20250109 08:56:29.498728 20598 heartbeater.cc:463] Registering TS with master...
I20250109 08:56:29.499791 20598 heartbeater.cc:510] Master 127.19.226.254:39769 requested a full tablet report, sending...
I20250109 08:56:29.502194 20395 ts_manager.cc:194] Registered new tserver with Master: b17b595d5da94e5dbf96bbef1ef3e668 (127.19.226.194:42855)
I20250109 08:56:29.503597 20395 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:34506
I20250109 08:56:29.504235 20363 server_base.cc:1034] running on GCE node
W20250109 08:56:29.505100 20604 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250109 08:56:29.505170 20602 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250109 08:56:29.506472 20363 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250109 08:56:29.506676 20363 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250109 08:56:29.506825 20363 hybrid_clock.cc:648] HybridClock initialized: now 1736412989506809 us; error 0 us; skew 500 ppm
I20250109 08:56:29.507293 20363 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250109 08:56:29.510006 20363 webserver.cc:458] Webserver started at http://127.19.226.195:41361/ using document root <none> and password file <none>
I20250109 08:56:29.510462 20363 fs_manager.cc:362] Metadata directory not provided
I20250109 08:56:29.510654 20363 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250109 08:56:29.510887 20363 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250109 08:56:29.511945 20363 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1736412987538469-20363-0/minicluster-data/ts-2-root/instance:
uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58"
format_stamp: "Formatted at 2025-01-09 08:56:29 on dist-test-slave-g8ds"
I20250109 08:56:29.516253 20363 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.001s sys 0.002s
I20250109 08:56:29.519542 20609 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250109 08:56:29.520300 20363 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20250109 08:56:29.520628 20363 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1736412987538469-20363-0/minicluster-data/ts-2-root
uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58"
format_stamp: "Formatted at 2025-01-09 08:56:29 on dist-test-slave-g8ds"
I20250109 08:56:29.520895 20363 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1736412987538469-20363-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1736412987538469-20363-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1736412987538469-20363-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250109 08:56:29.534018 20363 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250109 08:56:29.535200 20363 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250109 08:56:29.537297 20363 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250109 08:56:29.539489 20363 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250109 08:56:29.539685 20363 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250109 08:56:29.539897 20363 ts_tablet_manager.cc:610] Registered 0 tablets
I20250109 08:56:29.540045 20363 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250109 08:56:29.625483 20363 rpc_server.cc:307] RPC server started. Bound to: 127.19.226.195:45649
I20250109 08:56:29.625738 20671 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.226.195:45649 every 8 connection(s)
I20250109 08:56:29.643096 20672 heartbeater.cc:346] Connected to a master server at 127.19.226.254:39769
I20250109 08:56:29.643410 20672 heartbeater.cc:463] Registering TS with master...
I20250109 08:56:29.644269 20672 heartbeater.cc:510] Master 127.19.226.254:39769 requested a full tablet report, sending...
I20250109 08:56:29.646113 20395 ts_manager.cc:194] Registered new tserver with Master: c5bffb8e4c5e4e7a8b4e919c98c26e58 (127.19.226.195:45649)
I20250109 08:56:29.646605 20363 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.016737277s
I20250109 08:56:29.647392 20395 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:34510
I20250109 08:56:29.650238 20363 test_util.cc:274] Using random seed: 661550370
I20250109 08:56:29.704459 20395 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34514:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250109 08:56:29.711479 20395 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250109 08:56:29.823733 20562 tablet_service.cc:1467] Processing CreateTablet for tablet 4173445b18a24674ad20fe4b7e17c697 (DEFAULT_TABLE table=test-workload [id=41015294e19b4559b34d77fe20e20e92]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20250109 08:56:29.825474 20562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4173445b18a24674ad20fe4b7e17c697. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:56:29.825314 20563 tablet_service.cc:1467] Processing CreateTablet for tablet 6b1d1214e23d47ff8ec02197a45d8484 (DEFAULT_TABLE table=test-workload [id=41015294e19b4559b34d77fe20e20e92]), partition=RANGE (key) PARTITION VALUES < 238609294
I20250109 08:56:29.828707 20560 tablet_service.cc:1467] Processing CreateTablet for tablet 6f383be53b01468f85e731bd2d4367c0 (DEFAULT_TABLE table=test-workload [id=41015294e19b4559b34d77fe20e20e92]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20250109 08:56:29.826885 20561 tablet_service.cc:1467] Processing CreateTablet for tablet bc8cfc045acc40efaa744f955d08ea97 (DEFAULT_TABLE table=test-workload [id=41015294e19b4559b34d77fe20e20e92]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20250109 08:56:29.827126 20559 tablet_service.cc:1467] Processing CreateTablet for tablet bdeeebda65be4889bf59888b7a7abfec (DEFAULT_TABLE table=test-workload [id=41015294e19b4559b34d77fe20e20e92]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20250109 08:56:29.827857 20636 tablet_service.cc:1467] Processing CreateTablet for tablet 4173445b18a24674ad20fe4b7e17c697 (DEFAULT_TABLE table=test-workload [id=41015294e19b4559b34d77fe20e20e92]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20250109 08:56:29.826078 20637 tablet_service.cc:1467] Processing CreateTablet for tablet 6b1d1214e23d47ff8ec02197a45d8484 (DEFAULT_TABLE table=test-workload [id=41015294e19b4559b34d77fe20e20e92]), partition=RANGE (key) PARTITION VALUES < 238609294
I20250109 08:56:29.828794 20558 tablet_service.cc:1467] Processing CreateTablet for tablet cfbc2774b91c40d59dc80655451f5bdd (DEFAULT_TABLE table=test-workload [id=41015294e19b4559b34d77fe20e20e92]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20250109 08:56:29.836512 20561 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc8cfc045acc40efaa744f955d08ea97. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:56:29.842833 20631 tablet_service.cc:1467] Processing CreateTablet for tablet 74c881b69c3946429070c729ec66b0e4 (DEFAULT_TABLE table=test-workload [id=41015294e19b4559b34d77fe20e20e92]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20250109 08:56:29.846786 20559 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bdeeebda65be4889bf59888b7a7abfec. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:56:29.849349 20634 tablet_service.cc:1467] Processing CreateTablet for tablet 6f383be53b01468f85e731bd2d4367c0 (DEFAULT_TABLE table=test-workload [id=41015294e19b4559b34d77fe20e20e92]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20250109 08:56:29.850782 20555 tablet_service.cc:1467] Processing CreateTablet for tablet db23d411a3a14ed69a1db0b2c8912807 (DEFAULT_TABLE table=test-workload [id=41015294e19b4559b34d77fe20e20e92]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20250109 08:56:29.848866 20636 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4173445b18a24674ad20fe4b7e17c697. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:56:29.841672 20633 tablet_service.cc:1467] Processing CreateTablet for tablet bdeeebda65be4889bf59888b7a7abfec (DEFAULT_TABLE table=test-workload [id=41015294e19b4559b34d77fe20e20e92]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20250109 08:56:29.852411 20556 tablet_service.cc:1467] Processing CreateTablet for tablet 5378208cf9ad43129b6276fbb65af974 (DEFAULT_TABLE table=test-workload [id=41015294e19b4559b34d77fe20e20e92]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20250109 08:56:29.853224 20557 tablet_service.cc:1467] Processing CreateTablet for tablet 74c881b69c3946429070c729ec66b0e4 (DEFAULT_TABLE table=test-workload [id=41015294e19b4559b34d77fe20e20e92]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20250109 08:56:29.854660 20563 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b1d1214e23d47ff8ec02197a45d8484. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:56:29.849397 20635 tablet_service.cc:1467] Processing CreateTablet for tablet bc8cfc045acc40efaa744f955d08ea97 (DEFAULT_TABLE table=test-workload [id=41015294e19b4559b34d77fe20e20e92]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20250109 08:56:29.848541 20632 tablet_service.cc:1467] Processing CreateTablet for tablet cfbc2774b91c40d59dc80655451f5bdd (DEFAULT_TABLE table=test-workload [id=41015294e19b4559b34d77fe20e20e92]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20250109 08:56:29.856909 20637 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b1d1214e23d47ff8ec02197a45d8484. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:56:29.857450 20630 tablet_service.cc:1467] Processing CreateTablet for tablet 5378208cf9ad43129b6276fbb65af974 (DEFAULT_TABLE table=test-workload [id=41015294e19b4559b34d77fe20e20e92]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20250109 08:56:29.862809 20629 tablet_service.cc:1467] Processing CreateTablet for tablet db23d411a3a14ed69a1db0b2c8912807 (DEFAULT_TABLE table=test-workload [id=41015294e19b4559b34d77fe20e20e92]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20250109 08:56:29.868507 20634 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f383be53b01468f85e731bd2d4367c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:56:29.869056 20560 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f383be53b01468f85e731bd2d4367c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:56:29.872000 20556 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5378208cf9ad43129b6276fbb65af974. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:56:29.872579 20558 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cfbc2774b91c40d59dc80655451f5bdd. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:56:29.874064 20629 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db23d411a3a14ed69a1db0b2c8912807. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:56:29.874573 20557 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 74c881b69c3946429070c729ec66b0e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:56:29.876111 20635 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc8cfc045acc40efaa744f955d08ea97. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:56:29.878890 20555 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db23d411a3a14ed69a1db0b2c8912807. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:56:29.880342 20631 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 74c881b69c3946429070c729ec66b0e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:56:29.897173 20633 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bdeeebda65be4889bf59888b7a7abfec. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:56:29.899207 20632 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cfbc2774b91c40d59dc80655451f5bdd. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:56:29.900604 20630 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5378208cf9ad43129b6276fbb65af974. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:56:29.931592 20486 tablet_service.cc:1467] Processing CreateTablet for tablet bc8cfc045acc40efaa744f955d08ea97 (DEFAULT_TABLE table=test-workload [id=41015294e19b4559b34d77fe20e20e92]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20250109 08:56:29.946290 20488 tablet_service.cc:1467] Processing CreateTablet for tablet 6b1d1214e23d47ff8ec02197a45d8484 (DEFAULT_TABLE table=test-workload [id=41015294e19b4559b34d77fe20e20e92]), partition=RANGE (key) PARTITION VALUES < 238609294
I20250109 08:56:29.947819 20488 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b1d1214e23d47ff8ec02197a45d8484. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:56:29.950770 20486 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc8cfc045acc40efaa744f955d08ea97. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:56:29.953166 20487 tablet_service.cc:1467] Processing CreateTablet for tablet 4173445b18a24674ad20fe4b7e17c697 (DEFAULT_TABLE table=test-workload [id=41015294e19b4559b34d77fe20e20e92]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20250109 08:56:29.954670 20485 tablet_service.cc:1467] Processing CreateTablet for tablet 6f383be53b01468f85e731bd2d4367c0 (DEFAULT_TABLE table=test-workload [id=41015294e19b4559b34d77fe20e20e92]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20250109 08:56:29.961618 20487 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4173445b18a24674ad20fe4b7e17c697. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:56:29.969151 20482 tablet_service.cc:1467] Processing CreateTablet for tablet 74c881b69c3946429070c729ec66b0e4 (DEFAULT_TABLE table=test-workload [id=41015294e19b4559b34d77fe20e20e92]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20250109 08:56:30.002135 20485 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f383be53b01468f85e731bd2d4367c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:56:30.010986 20480 tablet_service.cc:1467] Processing CreateTablet for tablet db23d411a3a14ed69a1db0b2c8912807 (DEFAULT_TABLE table=test-workload [id=41015294e19b4559b34d77fe20e20e92]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20250109 08:56:30.012580 20484 tablet_service.cc:1467] Processing CreateTablet for tablet bdeeebda65be4889bf59888b7a7abfec (DEFAULT_TABLE table=test-workload [id=41015294e19b4559b34d77fe20e20e92]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20250109 08:56:30.018772 20483 tablet_service.cc:1467] Processing CreateTablet for tablet cfbc2774b91c40d59dc80655451f5bdd (DEFAULT_TABLE table=test-workload [id=41015294e19b4559b34d77fe20e20e92]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20250109 08:56:30.013815 20481 tablet_service.cc:1467] Processing CreateTablet for tablet 5378208cf9ad43129b6276fbb65af974 (DEFAULT_TABLE table=test-workload [id=41015294e19b4559b34d77fe20e20e92]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20250109 08:56:30.024497 20480 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db23d411a3a14ed69a1db0b2c8912807. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:56:30.037526 20481 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5378208cf9ad43129b6276fbb65af974. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:56:30.040349 20482 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 74c881b69c3946429070c729ec66b0e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:56:30.043044 20484 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bdeeebda65be4889bf59888b7a7abfec. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:56:30.060513 20691 tablet_bootstrap.cc:492] T 6b1d1214e23d47ff8ec02197a45d8484 P b17b595d5da94e5dbf96bbef1ef3e668: Bootstrap starting.
I20250109 08:56:30.062002 20483 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cfbc2774b91c40d59dc80655451f5bdd. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:56:30.083398 20690 tablet_bootstrap.cc:492] T 4173445b18a24674ad20fe4b7e17c697 P c5bffb8e4c5e4e7a8b4e919c98c26e58: Bootstrap starting.
W20250109 08:56:30.095103 20442 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20250109 08:56:30.148952 20691 tablet_bootstrap.cc:654] T 6b1d1214e23d47ff8ec02197a45d8484 P b17b595d5da94e5dbf96bbef1ef3e668: Neither blocks nor log segments found. Creating new log.
I20250109 08:56:30.152256 20692 tablet_bootstrap.cc:492] T 6b1d1214e23d47ff8ec02197a45d8484 P ff76b9242fda4d4586c2bb3da829b1ba: Bootstrap starting.
I20250109 08:56:30.155642 20690 tablet_bootstrap.cc:654] T 4173445b18a24674ad20fe4b7e17c697 P c5bffb8e4c5e4e7a8b4e919c98c26e58: Neither blocks nor log segments found. Creating new log.
I20250109 08:56:30.162847 20692 tablet_bootstrap.cc:654] T 6b1d1214e23d47ff8ec02197a45d8484 P ff76b9242fda4d4586c2bb3da829b1ba: Neither blocks nor log segments found. Creating new log.
I20250109 08:56:30.175362 20690 tablet_bootstrap.cc:492] T 4173445b18a24674ad20fe4b7e17c697 P c5bffb8e4c5e4e7a8b4e919c98c26e58: No bootstrap required, opened a new log
I20250109 08:56:30.175832 20690 ts_tablet_manager.cc:1397] T 4173445b18a24674ad20fe4b7e17c697 P c5bffb8e4c5e4e7a8b4e919c98c26e58: Time spent bootstrapping tablet: real 0.136s user 0.011s sys 0.006s
I20250109 08:56:30.177636 20692 tablet_bootstrap.cc:492] T 6b1d1214e23d47ff8ec02197a45d8484 P ff76b9242fda4d4586c2bb3da829b1ba: No bootstrap required, opened a new log
I20250109 08:56:30.178057 20692 ts_tablet_manager.cc:1397] T 6b1d1214e23d47ff8ec02197a45d8484 P ff76b9242fda4d4586c2bb3da829b1ba: Time spent bootstrapping tablet: real 0.026s user 0.015s sys 0.000s
I20250109 08:56:30.179984 20690 raft_consensus.cc:357] T 4173445b18a24674ad20fe4b7e17c697 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:30.180696 20690 raft_consensus.cc:383] T 4173445b18a24674ad20fe4b7e17c697 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:56:30.180536 20692 raft_consensus.cc:357] T 6b1d1214e23d47ff8ec02197a45d8484 P ff76b9242fda4d4586c2bb3da829b1ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:30.181036 20690 raft_consensus.cc:738] T 4173445b18a24674ad20fe4b7e17c697 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5bffb8e4c5e4e7a8b4e919c98c26e58, State: Initialized, Role: FOLLOWER
I20250109 08:56:30.181317 20692 raft_consensus.cc:383] T 6b1d1214e23d47ff8ec02197a45d8484 P ff76b9242fda4d4586c2bb3da829b1ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:56:30.181628 20692 raft_consensus.cc:738] T 6b1d1214e23d47ff8ec02197a45d8484 P ff76b9242fda4d4586c2bb3da829b1ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff76b9242fda4d4586c2bb3da829b1ba, State: Initialized, Role: FOLLOWER
I20250109 08:56:30.181859 20690 consensus_queue.cc:260] T 4173445b18a24674ad20fe4b7e17c697 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:30.182934 20691 tablet_bootstrap.cc:492] T 6b1d1214e23d47ff8ec02197a45d8484 P b17b595d5da94e5dbf96bbef1ef3e668: No bootstrap required, opened a new log
I20250109 08:56:30.182695 20692 consensus_queue.cc:260] T 6b1d1214e23d47ff8ec02197a45d8484 P ff76b9242fda4d4586c2bb3da829b1ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:30.183583 20691 ts_tablet_manager.cc:1397] T 6b1d1214e23d47ff8ec02197a45d8484 P b17b595d5da94e5dbf96bbef1ef3e668: Time spent bootstrapping tablet: real 0.128s user 0.006s sys 0.009s
I20250109 08:56:30.186674 20691 raft_consensus.cc:357] T 6b1d1214e23d47ff8ec02197a45d8484 P b17b595d5da94e5dbf96bbef1ef3e668 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:30.187765 20691 raft_consensus.cc:383] T 6b1d1214e23d47ff8ec02197a45d8484 P b17b595d5da94e5dbf96bbef1ef3e668 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:56:30.188117 20691 raft_consensus.cc:738] T 6b1d1214e23d47ff8ec02197a45d8484 P b17b595d5da94e5dbf96bbef1ef3e668 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b17b595d5da94e5dbf96bbef1ef3e668, State: Initialized, Role: FOLLOWER
I20250109 08:56:30.188798 20692 ts_tablet_manager.cc:1428] T 6b1d1214e23d47ff8ec02197a45d8484 P ff76b9242fda4d4586c2bb3da829b1ba: Time spent starting tablet: real 0.010s user 0.007s sys 0.004s
I20250109 08:56:30.188937 20691 consensus_queue.cc:260] T 6b1d1214e23d47ff8ec02197a45d8484 P b17b595d5da94e5dbf96bbef1ef3e668 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:30.189646 20523 heartbeater.cc:502] Master 127.19.226.254:39769 was elected leader, sending a full tablet report...
I20250109 08:56:30.189886 20692 tablet_bootstrap.cc:492] T bc8cfc045acc40efaa744f955d08ea97 P ff76b9242fda4d4586c2bb3da829b1ba: Bootstrap starting.
I20250109 08:56:30.196655 20692 tablet_bootstrap.cc:654] T bc8cfc045acc40efaa744f955d08ea97 P ff76b9242fda4d4586c2bb3da829b1ba: Neither blocks nor log segments found. Creating new log.
I20250109 08:56:30.200409 20672 heartbeater.cc:502] Master 127.19.226.254:39769 was elected leader, sending a full tablet report...
I20250109 08:56:30.203318 20690 ts_tablet_manager.cc:1428] T 4173445b18a24674ad20fe4b7e17c697 P c5bffb8e4c5e4e7a8b4e919c98c26e58: Time spent starting tablet: real 0.027s user 0.012s sys 0.003s
I20250109 08:56:30.206849 20691 ts_tablet_manager.cc:1428] T 6b1d1214e23d47ff8ec02197a45d8484 P b17b595d5da94e5dbf96bbef1ef3e668: Time spent starting tablet: real 0.023s user 0.007s sys 0.010s
I20250109 08:56:30.207337 20598 heartbeater.cc:502] Master 127.19.226.254:39769 was elected leader, sending a full tablet report...
I20250109 08:56:30.208063 20690 tablet_bootstrap.cc:492] T bdeeebda65be4889bf59888b7a7abfec P c5bffb8e4c5e4e7a8b4e919c98c26e58: Bootstrap starting.
I20250109 08:56:30.212919 20691 tablet_bootstrap.cc:492] T 6f383be53b01468f85e731bd2d4367c0 P b17b595d5da94e5dbf96bbef1ef3e668: Bootstrap starting.
I20250109 08:56:30.215478 20692 tablet_bootstrap.cc:492] T bc8cfc045acc40efaa744f955d08ea97 P ff76b9242fda4d4586c2bb3da829b1ba: No bootstrap required, opened a new log
I20250109 08:56:30.215945 20692 ts_tablet_manager.cc:1397] T bc8cfc045acc40efaa744f955d08ea97 P ff76b9242fda4d4586c2bb3da829b1ba: Time spent bootstrapping tablet: real 0.026s user 0.007s sys 0.003s
I20250109 08:56:30.218735 20692 raft_consensus.cc:357] T bc8cfc045acc40efaa744f955d08ea97 P ff76b9242fda4d4586c2bb3da829b1ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:30.219477 20692 raft_consensus.cc:383] T bc8cfc045acc40efaa744f955d08ea97 P ff76b9242fda4d4586c2bb3da829b1ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:56:30.219763 20692 raft_consensus.cc:738] T bc8cfc045acc40efaa744f955d08ea97 P ff76b9242fda4d4586c2bb3da829b1ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff76b9242fda4d4586c2bb3da829b1ba, State: Initialized, Role: FOLLOWER
I20250109 08:56:30.220472 20692 consensus_queue.cc:260] T bc8cfc045acc40efaa744f955d08ea97 P ff76b9242fda4d4586c2bb3da829b1ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:30.222283 20690 tablet_bootstrap.cc:654] T bdeeebda65be4889bf59888b7a7abfec P c5bffb8e4c5e4e7a8b4e919c98c26e58: Neither blocks nor log segments found. Creating new log.
I20250109 08:56:30.226660 20691 tablet_bootstrap.cc:654] T 6f383be53b01468f85e731bd2d4367c0 P b17b595d5da94e5dbf96bbef1ef3e668: Neither blocks nor log segments found. Creating new log.
I20250109 08:56:30.222960 20692 ts_tablet_manager.cc:1428] T bc8cfc045acc40efaa744f955d08ea97 P ff76b9242fda4d4586c2bb3da829b1ba: Time spent starting tablet: real 0.007s user 0.004s sys 0.004s
I20250109 08:56:30.229153 20692 tablet_bootstrap.cc:492] T 5378208cf9ad43129b6276fbb65af974 P ff76b9242fda4d4586c2bb3da829b1ba: Bootstrap starting.
I20250109 08:56:30.235136 20692 tablet_bootstrap.cc:654] T 5378208cf9ad43129b6276fbb65af974 P ff76b9242fda4d4586c2bb3da829b1ba: Neither blocks nor log segments found. Creating new log.
W20250109 08:56:30.238989 20599 tablet.cc:2367] T 6b1d1214e23d47ff8ec02197a45d8484 P b17b595d5da94e5dbf96bbef1ef3e668: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20250109 08:56:30.247357 20690 tablet_bootstrap.cc:492] T bdeeebda65be4889bf59888b7a7abfec P c5bffb8e4c5e4e7a8b4e919c98c26e58: No bootstrap required, opened a new log
I20250109 08:56:30.247859 20690 ts_tablet_manager.cc:1397] T bdeeebda65be4889bf59888b7a7abfec P c5bffb8e4c5e4e7a8b4e919c98c26e58: Time spent bootstrapping tablet: real 0.044s user 0.014s sys 0.004s
I20250109 08:56:30.250028 20691 tablet_bootstrap.cc:492] T 6f383be53b01468f85e731bd2d4367c0 P b17b595d5da94e5dbf96bbef1ef3e668: No bootstrap required, opened a new log
I20250109 08:56:30.250458 20691 ts_tablet_manager.cc:1397] T 6f383be53b01468f85e731bd2d4367c0 P b17b595d5da94e5dbf96bbef1ef3e668: Time spent bootstrapping tablet: real 0.038s user 0.003s sys 0.015s
I20250109 08:56:30.250563 20690 raft_consensus.cc:357] T bdeeebda65be4889bf59888b7a7abfec P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:30.251339 20690 raft_consensus.cc:383] T bdeeebda65be4889bf59888b7a7abfec P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:56:30.251636 20690 raft_consensus.cc:738] T bdeeebda65be4889bf59888b7a7abfec P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5bffb8e4c5e4e7a8b4e919c98c26e58, State: Initialized, Role: FOLLOWER
I20250109 08:56:30.252295 20690 consensus_queue.cc:260] T bdeeebda65be4889bf59888b7a7abfec P c5bffb8e4c5e4e7a8b4e919c98c26e58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:30.253295 20691 raft_consensus.cc:357] T 6f383be53b01468f85e731bd2d4367c0 P b17b595d5da94e5dbf96bbef1ef3e668 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:30.254041 20691 raft_consensus.cc:383] T 6f383be53b01468f85e731bd2d4367c0 P b17b595d5da94e5dbf96bbef1ef3e668 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:56:30.254316 20691 raft_consensus.cc:738] T 6f383be53b01468f85e731bd2d4367c0 P b17b595d5da94e5dbf96bbef1ef3e668 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b17b595d5da94e5dbf96bbef1ef3e668, State: Initialized, Role: FOLLOWER
I20250109 08:56:30.254491 20690 ts_tablet_manager.cc:1428] T bdeeebda65be4889bf59888b7a7abfec P c5bffb8e4c5e4e7a8b4e919c98c26e58: Time spent starting tablet: real 0.006s user 0.000s sys 0.007s
I20250109 08:56:30.255374 20690 tablet_bootstrap.cc:492] T bc8cfc045acc40efaa744f955d08ea97 P c5bffb8e4c5e4e7a8b4e919c98c26e58: Bootstrap starting.
I20250109 08:56:30.254952 20691 consensus_queue.cc:260] T 6f383be53b01468f85e731bd2d4367c0 P b17b595d5da94e5dbf96bbef1ef3e668 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:30.257772 20691 ts_tablet_manager.cc:1428] T 6f383be53b01468f85e731bd2d4367c0 P b17b595d5da94e5dbf96bbef1ef3e668: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20250109 08:56:30.258639 20691 tablet_bootstrap.cc:492] T cfbc2774b91c40d59dc80655451f5bdd P b17b595d5da94e5dbf96bbef1ef3e668: Bootstrap starting.
I20250109 08:56:30.262470 20690 tablet_bootstrap.cc:654] T bc8cfc045acc40efaa744f955d08ea97 P c5bffb8e4c5e4e7a8b4e919c98c26e58: Neither blocks nor log segments found. Creating new log.
I20250109 08:56:30.264619 20691 tablet_bootstrap.cc:654] T cfbc2774b91c40d59dc80655451f5bdd P b17b595d5da94e5dbf96bbef1ef3e668: Neither blocks nor log segments found. Creating new log.
I20250109 08:56:30.299189 20690 tablet_bootstrap.cc:492] T bc8cfc045acc40efaa744f955d08ea97 P c5bffb8e4c5e4e7a8b4e919c98c26e58: No bootstrap required, opened a new log
I20250109 08:56:30.299572 20692 tablet_bootstrap.cc:492] T 5378208cf9ad43129b6276fbb65af974 P ff76b9242fda4d4586c2bb3da829b1ba: No bootstrap required, opened a new log
I20250109 08:56:30.299705 20690 ts_tablet_manager.cc:1397] T bc8cfc045acc40efaa744f955d08ea97 P c5bffb8e4c5e4e7a8b4e919c98c26e58: Time spent bootstrapping tablet: real 0.044s user 0.031s sys 0.011s
I20250109 08:56:30.300074 20692 ts_tablet_manager.cc:1397] T 5378208cf9ad43129b6276fbb65af974 P ff76b9242fda4d4586c2bb3da829b1ba: Time spent bootstrapping tablet: real 0.071s user 0.019s sys 0.020s
I20250109 08:56:30.300616 20691 tablet_bootstrap.cc:492] T cfbc2774b91c40d59dc80655451f5bdd P b17b595d5da94e5dbf96bbef1ef3e668: No bootstrap required, opened a new log
I20250109 08:56:30.301091 20691 ts_tablet_manager.cc:1397] T cfbc2774b91c40d59dc80655451f5bdd P b17b595d5da94e5dbf96bbef1ef3e668: Time spent bootstrapping tablet: real 0.043s user 0.005s sys 0.007s
I20250109 08:56:30.302472 20690 raft_consensus.cc:357] T bc8cfc045acc40efaa744f955d08ea97 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:30.302780 20692 raft_consensus.cc:357] T 5378208cf9ad43129b6276fbb65af974 P ff76b9242fda4d4586c2bb3da829b1ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:30.303238 20690 raft_consensus.cc:383] T bc8cfc045acc40efaa744f955d08ea97 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:56:30.303550 20692 raft_consensus.cc:383] T 5378208cf9ad43129b6276fbb65af974 P ff76b9242fda4d4586c2bb3da829b1ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:56:30.303570 20690 raft_consensus.cc:738] T bc8cfc045acc40efaa744f955d08ea97 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5bffb8e4c5e4e7a8b4e919c98c26e58, State: Initialized, Role: FOLLOWER
I20250109 08:56:30.304065 20692 raft_consensus.cc:738] T 5378208cf9ad43129b6276fbb65af974 P ff76b9242fda4d4586c2bb3da829b1ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff76b9242fda4d4586c2bb3da829b1ba, State: Initialized, Role: FOLLOWER
I20250109 08:56:30.303947 20691 raft_consensus.cc:357] T cfbc2774b91c40d59dc80655451f5bdd P b17b595d5da94e5dbf96bbef1ef3e668 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:30.304528 20690 consensus_queue.cc:260] T bc8cfc045acc40efaa744f955d08ea97 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:30.305223 20691 raft_consensus.cc:383] T cfbc2774b91c40d59dc80655451f5bdd P b17b595d5da94e5dbf96bbef1ef3e668 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:56:30.305094 20692 consensus_queue.cc:260] T 5378208cf9ad43129b6276fbb65af974 P ff76b9242fda4d4586c2bb3da829b1ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:30.307176 20690 ts_tablet_manager.cc:1428] T bc8cfc045acc40efaa744f955d08ea97 P c5bffb8e4c5e4e7a8b4e919c98c26e58: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20250109 08:56:30.307650 20692 ts_tablet_manager.cc:1428] T 5378208cf9ad43129b6276fbb65af974 P ff76b9242fda4d4586c2bb3da829b1ba: Time spent starting tablet: real 0.007s user 0.002s sys 0.004s
I20250109 08:56:30.308125 20690 tablet_bootstrap.cc:492] T db23d411a3a14ed69a1db0b2c8912807 P c5bffb8e4c5e4e7a8b4e919c98c26e58: Bootstrap starting.
I20250109 08:56:30.305661 20691 raft_consensus.cc:738] T cfbc2774b91c40d59dc80655451f5bdd P b17b595d5da94e5dbf96bbef1ef3e668 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b17b595d5da94e5dbf96bbef1ef3e668, State: Initialized, Role: FOLLOWER
I20250109 08:56:30.308718 20692 tablet_bootstrap.cc:492] T db23d411a3a14ed69a1db0b2c8912807 P ff76b9242fda4d4586c2bb3da829b1ba: Bootstrap starting.
I20250109 08:56:30.309161 20691 consensus_queue.cc:260] T cfbc2774b91c40d59dc80655451f5bdd P b17b595d5da94e5dbf96bbef1ef3e668 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:30.313560 20691 ts_tablet_manager.cc:1428] T cfbc2774b91c40d59dc80655451f5bdd P b17b595d5da94e5dbf96bbef1ef3e668: Time spent starting tablet: real 0.012s user 0.002s sys 0.004s
I20250109 08:56:30.314702 20691 tablet_bootstrap.cc:492] T bc8cfc045acc40efaa744f955d08ea97 P b17b595d5da94e5dbf96bbef1ef3e668: Bootstrap starting.
I20250109 08:56:30.315398 20690 tablet_bootstrap.cc:654] T db23d411a3a14ed69a1db0b2c8912807 P c5bffb8e4c5e4e7a8b4e919c98c26e58: Neither blocks nor log segments found. Creating new log.
I20250109 08:56:30.315873 20692 tablet_bootstrap.cc:654] T db23d411a3a14ed69a1db0b2c8912807 P ff76b9242fda4d4586c2bb3da829b1ba: Neither blocks nor log segments found. Creating new log.
I20250109 08:56:30.317476 20696 raft_consensus.cc:491] T bdeeebda65be4889bf59888b7a7abfec P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:56:30.317817 20697 raft_consensus.cc:491] T bc8cfc045acc40efaa744f955d08ea97 P ff76b9242fda4d4586c2bb3da829b1ba [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:56:30.318053 20696 raft_consensus.cc:513] T bdeeebda65be4889bf59888b7a7abfec P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:30.318231 20697 raft_consensus.cc:513] T bc8cfc045acc40efaa744f955d08ea97 P ff76b9242fda4d4586c2bb3da829b1ba [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:30.322991 20697 leader_election.cc:290] T bc8cfc045acc40efaa744f955d08ea97 P ff76b9242fda4d4586c2bb3da829b1ba [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b17b595d5da94e5dbf96bbef1ef3e668 (127.19.226.194:42855), c5bffb8e4c5e4e7a8b4e919c98c26e58 (127.19.226.195:45649)
I20250109 08:56:30.324800 20696 leader_election.cc:290] T bdeeebda65be4889bf59888b7a7abfec P c5bffb8e4c5e4e7a8b4e919c98c26e58 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ff76b9242fda4d4586c2bb3da829b1ba (127.19.226.193:42763), b17b595d5da94e5dbf96bbef1ef3e668 (127.19.226.194:42855)
I20250109 08:56:30.331246 20697 raft_consensus.cc:491] T 6b1d1214e23d47ff8ec02197a45d8484 P ff76b9242fda4d4586c2bb3da829b1ba [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:56:30.332608 20697 raft_consensus.cc:513] T 6b1d1214e23d47ff8ec02197a45d8484 P ff76b9242fda4d4586c2bb3da829b1ba [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:30.338825 20697 leader_election.cc:290] T 6b1d1214e23d47ff8ec02197a45d8484 P ff76b9242fda4d4586c2bb3da829b1ba [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b17b595d5da94e5dbf96bbef1ef3e668 (127.19.226.194:42855), c5bffb8e4c5e4e7a8b4e919c98c26e58 (127.19.226.195:45649)
I20250109 08:56:30.347527 20691 tablet_bootstrap.cc:654] T bc8cfc045acc40efaa744f955d08ea97 P b17b595d5da94e5dbf96bbef1ef3e668: Neither blocks nor log segments found. Creating new log.
I20250109 08:56:30.350132 20692 tablet_bootstrap.cc:492] T db23d411a3a14ed69a1db0b2c8912807 P ff76b9242fda4d4586c2bb3da829b1ba: No bootstrap required, opened a new log
I20250109 08:56:30.350669 20692 ts_tablet_manager.cc:1397] T db23d411a3a14ed69a1db0b2c8912807 P ff76b9242fda4d4586c2bb3da829b1ba: Time spent bootstrapping tablet: real 0.042s user 0.012s sys 0.005s
I20250109 08:56:30.357340 20692 raft_consensus.cc:357] T db23d411a3a14ed69a1db0b2c8912807 P ff76b9242fda4d4586c2bb3da829b1ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:30.358135 20692 raft_consensus.cc:383] T db23d411a3a14ed69a1db0b2c8912807 P ff76b9242fda4d4586c2bb3da829b1ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:56:30.358451 20692 raft_consensus.cc:738] T db23d411a3a14ed69a1db0b2c8912807 P ff76b9242fda4d4586c2bb3da829b1ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff76b9242fda4d4586c2bb3da829b1ba, State: Initialized, Role: FOLLOWER
I20250109 08:56:30.359149 20692 consensus_queue.cc:260] T db23d411a3a14ed69a1db0b2c8912807 P ff76b9242fda4d4586c2bb3da829b1ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:30.361737 20498 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bdeeebda65be4889bf59888b7a7abfec" candidate_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" is_pre_election: true
I20250109 08:56:30.365693 20692 ts_tablet_manager.cc:1428] T db23d411a3a14ed69a1db0b2c8912807 P ff76b9242fda4d4586c2bb3da829b1ba: Time spent starting tablet: real 0.015s user 0.002s sys 0.006s
I20250109 08:56:30.366696 20692 tablet_bootstrap.cc:492] T 4173445b18a24674ad20fe4b7e17c697 P ff76b9242fda4d4586c2bb3da829b1ba: Bootstrap starting.
I20250109 08:56:30.368323 20691 tablet_bootstrap.cc:492] T bc8cfc045acc40efaa744f955d08ea97 P b17b595d5da94e5dbf96bbef1ef3e668: No bootstrap required, opened a new log
I20250109 08:56:30.369478 20691 ts_tablet_manager.cc:1397] T bc8cfc045acc40efaa744f955d08ea97 P b17b595d5da94e5dbf96bbef1ef3e668: Time spent bootstrapping tablet: real 0.055s user 0.007s sys 0.005s
I20250109 08:56:30.372236 20691 raft_consensus.cc:357] T bc8cfc045acc40efaa744f955d08ea97 P b17b595d5da94e5dbf96bbef1ef3e668 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:30.373092 20692 tablet_bootstrap.cc:654] T 4173445b18a24674ad20fe4b7e17c697 P ff76b9242fda4d4586c2bb3da829b1ba: Neither blocks nor log segments found. Creating new log.
I20250109 08:56:30.373368 20691 raft_consensus.cc:383] T bc8cfc045acc40efaa744f955d08ea97 P b17b595d5da94e5dbf96bbef1ef3e668 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:56:30.373744 20691 raft_consensus.cc:738] T bc8cfc045acc40efaa744f955d08ea97 P b17b595d5da94e5dbf96bbef1ef3e668 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b17b595d5da94e5dbf96bbef1ef3e668, State: Initialized, Role: FOLLOWER
W20250109 08:56:30.375811 20613 leader_election.cc:343] T bdeeebda65be4889bf59888b7a7abfec P c5bffb8e4c5e4e7a8b4e919c98c26e58 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ff76b9242fda4d4586c2bb3da829b1ba (127.19.226.193:42763): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:56:30.379668 20647 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc8cfc045acc40efaa744f955d08ea97" candidate_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" is_pre_election: true
I20250109 08:56:30.380470 20647 raft_consensus.cc:2463] T bc8cfc045acc40efaa744f955d08ea97 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff76b9242fda4d4586c2bb3da829b1ba in term 0.
I20250109 08:56:30.382220 20461 leader_election.cc:304] T bc8cfc045acc40efaa744f955d08ea97 P ff76b9242fda4d4586c2bb3da829b1ba [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c5bffb8e4c5e4e7a8b4e919c98c26e58, ff76b9242fda4d4586c2bb3da829b1ba; no voters:
I20250109 08:56:30.383023 20697 raft_consensus.cc:2798] T bc8cfc045acc40efaa744f955d08ea97 P ff76b9242fda4d4586c2bb3da829b1ba [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:56:30.383371 20697 raft_consensus.cc:491] T bc8cfc045acc40efaa744f955d08ea97 P ff76b9242fda4d4586c2bb3da829b1ba [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:56:30.383286 20647 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b1d1214e23d47ff8ec02197a45d8484" candidate_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" is_pre_election: true
I20250109 08:56:30.383483 20573 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bdeeebda65be4889bf59888b7a7abfec" candidate_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" is_pre_election: true
I20250109 08:56:30.383834 20697 raft_consensus.cc:3054] T bc8cfc045acc40efaa744f955d08ea97 P ff76b9242fda4d4586c2bb3da829b1ba [term 0 FOLLOWER]: Advancing to term 1
W20250109 08:56:30.384647 20461 leader_election.cc:343] T 6b1d1214e23d47ff8ec02197a45d8484 P ff76b9242fda4d4586c2bb3da829b1ba [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c5bffb8e4c5e4e7a8b4e919c98c26e58 (127.19.226.195:45649): Illegal state: must be running to vote when last-logged opid is not known
W20250109 08:56:30.385257 20611 leader_election.cc:343] T bdeeebda65be4889bf59888b7a7abfec P c5bffb8e4c5e4e7a8b4e919c98c26e58 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b17b595d5da94e5dbf96bbef1ef3e668 (127.19.226.194:42855): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:56:30.385681 20611 leader_election.cc:304] T bdeeebda65be4889bf59888b7a7abfec P c5bffb8e4c5e4e7a8b4e919c98c26e58 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c5bffb8e4c5e4e7a8b4e919c98c26e58; no voters: b17b595d5da94e5dbf96bbef1ef3e668, ff76b9242fda4d4586c2bb3da829b1ba
I20250109 08:56:30.386029 20573 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc8cfc045acc40efaa744f955d08ea97" candidate_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" is_pre_election: true
I20250109 08:56:30.386788 20696 raft_consensus.cc:2743] T bdeeebda65be4889bf59888b7a7abfec P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:56:30.387656 20572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b1d1214e23d47ff8ec02197a45d8484" candidate_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" is_pre_election: true
I20250109 08:56:30.380705 20691 consensus_queue.cc:260] T bc8cfc045acc40efaa744f955d08ea97 P b17b595d5da94e5dbf96bbef1ef3e668 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:30.388243 20572 raft_consensus.cc:2463] T 6b1d1214e23d47ff8ec02197a45d8484 P b17b595d5da94e5dbf96bbef1ef3e668 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff76b9242fda4d4586c2bb3da829b1ba in term 0.
I20250109 08:56:30.388345 20573 raft_consensus.cc:2463] T bc8cfc045acc40efaa744f955d08ea97 P b17b595d5da94e5dbf96bbef1ef3e668 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff76b9242fda4d4586c2bb3da829b1ba in term 0.
I20250109 08:56:30.390574 20690 tablet_bootstrap.cc:492] T db23d411a3a14ed69a1db0b2c8912807 P c5bffb8e4c5e4e7a8b4e919c98c26e58: No bootstrap required, opened a new log
I20250109 08:56:30.391011 20690 ts_tablet_manager.cc:1397] T db23d411a3a14ed69a1db0b2c8912807 P c5bffb8e4c5e4e7a8b4e919c98c26e58: Time spent bootstrapping tablet: real 0.083s user 0.006s sys 0.007s
I20250109 08:56:30.391947 20462 leader_election.cc:304] T 6b1d1214e23d47ff8ec02197a45d8484 P ff76b9242fda4d4586c2bb3da829b1ba [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b17b595d5da94e5dbf96bbef1ef3e668, ff76b9242fda4d4586c2bb3da829b1ba; no voters: c5bffb8e4c5e4e7a8b4e919c98c26e58
I20250109 08:56:30.394258 20690 raft_consensus.cc:357] T db23d411a3a14ed69a1db0b2c8912807 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:30.395043 20690 raft_consensus.cc:383] T db23d411a3a14ed69a1db0b2c8912807 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:56:30.395340 20690 raft_consensus.cc:738] T db23d411a3a14ed69a1db0b2c8912807 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5bffb8e4c5e4e7a8b4e919c98c26e58, State: Initialized, Role: FOLLOWER
I20250109 08:56:30.396346 20691 ts_tablet_manager.cc:1428] T bc8cfc045acc40efaa744f955d08ea97 P b17b595d5da94e5dbf96bbef1ef3e668: Time spent starting tablet: real 0.027s user 0.006s sys 0.000s
I20250109 08:56:30.396045 20690 consensus_queue.cc:260] T db23d411a3a14ed69a1db0b2c8912807 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:30.400717 20690 ts_tablet_manager.cc:1428] T db23d411a3a14ed69a1db0b2c8912807 P c5bffb8e4c5e4e7a8b4e919c98c26e58: Time spent starting tablet: real 0.009s user 0.007s sys 0.002s
I20250109 08:56:30.401593 20690 tablet_bootstrap.cc:492] T 6b1d1214e23d47ff8ec02197a45d8484 P c5bffb8e4c5e4e7a8b4e919c98c26e58: Bootstrap starting.
I20250109 08:56:30.403643 20691 tablet_bootstrap.cc:492] T db23d411a3a14ed69a1db0b2c8912807 P b17b595d5da94e5dbf96bbef1ef3e668: Bootstrap starting.
I20250109 08:56:30.406903 20697 raft_consensus.cc:513] T bc8cfc045acc40efaa744f955d08ea97 P ff76b9242fda4d4586c2bb3da829b1ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:30.410181 20692 tablet_bootstrap.cc:492] T 4173445b18a24674ad20fe4b7e17c697 P ff76b9242fda4d4586c2bb3da829b1ba: No bootstrap required, opened a new log
I20250109 08:56:30.410296 20697 leader_election.cc:290] T bc8cfc045acc40efaa744f955d08ea97 P ff76b9242fda4d4586c2bb3da829b1ba [CANDIDATE]: Term 1 election: Requested vote from peers b17b595d5da94e5dbf96bbef1ef3e668 (127.19.226.194:42855), c5bffb8e4c5e4e7a8b4e919c98c26e58 (127.19.226.195:45649)
I20250109 08:56:30.410758 20692 ts_tablet_manager.cc:1397] T 4173445b18a24674ad20fe4b7e17c697 P ff76b9242fda4d4586c2bb3da829b1ba: Time spent bootstrapping tablet: real 0.044s user 0.005s sys 0.012s
I20250109 08:56:30.410892 20647 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc8cfc045acc40efaa744f955d08ea97" candidate_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58"
I20250109 08:56:30.411609 20647 raft_consensus.cc:3054] T bc8cfc045acc40efaa744f955d08ea97 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:56:30.410856 20697 raft_consensus.cc:2798] T 6b1d1214e23d47ff8ec02197a45d8484 P ff76b9242fda4d4586c2bb3da829b1ba [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:56:30.414649 20697 raft_consensus.cc:491] T 6b1d1214e23d47ff8ec02197a45d8484 P ff76b9242fda4d4586c2bb3da829b1ba [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:56:30.414829 20573 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc8cfc045acc40efaa744f955d08ea97" candidate_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b17b595d5da94e5dbf96bbef1ef3e668"
I20250109 08:56:30.415014 20697 raft_consensus.cc:3054] T 6b1d1214e23d47ff8ec02197a45d8484 P ff76b9242fda4d4586c2bb3da829b1ba [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:56:30.415635 20573 raft_consensus.cc:3054] T bc8cfc045acc40efaa744f955d08ea97 P b17b595d5da94e5dbf96bbef1ef3e668 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:56:30.417335 20696 raft_consensus.cc:491] T db23d411a3a14ed69a1db0b2c8912807 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:56:30.421049 20696 raft_consensus.cc:513] T db23d411a3a14ed69a1db0b2c8912807 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:30.422358 20690 tablet_bootstrap.cc:654] T 6b1d1214e23d47ff8ec02197a45d8484 P c5bffb8e4c5e4e7a8b4e919c98c26e58: Neither blocks nor log segments found. Creating new log.
I20250109 08:56:30.423040 20696 leader_election.cc:290] T db23d411a3a14ed69a1db0b2c8912807 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ff76b9242fda4d4586c2bb3da829b1ba (127.19.226.193:42763), b17b595d5da94e5dbf96bbef1ef3e668 (127.19.226.194:42855)
I20250109 08:56:30.417121 20692 raft_consensus.cc:357] T 4173445b18a24674ad20fe4b7e17c697 P ff76b9242fda4d4586c2bb3da829b1ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:30.423987 20692 raft_consensus.cc:383] T 4173445b18a24674ad20fe4b7e17c697 P ff76b9242fda4d4586c2bb3da829b1ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:56:30.424055 20498 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db23d411a3a14ed69a1db0b2c8912807" candidate_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" is_pre_election: true
I20250109 08:56:30.424348 20692 raft_consensus.cc:738] T 4173445b18a24674ad20fe4b7e17c697 P ff76b9242fda4d4586c2bb3da829b1ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff76b9242fda4d4586c2bb3da829b1ba, State: Initialized, Role: FOLLOWER
I20250109 08:56:30.424204 20572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db23d411a3a14ed69a1db0b2c8912807" candidate_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" is_pre_election: true
I20250109 08:56:30.424432 20647 raft_consensus.cc:2463] T bc8cfc045acc40efaa744f955d08ea97 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff76b9242fda4d4586c2bb3da829b1ba in term 1.
W20250109 08:56:30.425869 20611 leader_election.cc:343] T db23d411a3a14ed69a1db0b2c8912807 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b17b595d5da94e5dbf96bbef1ef3e668 (127.19.226.194:42855): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:56:30.425748 20692 consensus_queue.cc:260] T 4173445b18a24674ad20fe4b7e17c697 P ff76b9242fda4d4586c2bb3da829b1ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:30.426560 20691 tablet_bootstrap.cc:654] T db23d411a3a14ed69a1db0b2c8912807 P b17b595d5da94e5dbf96bbef1ef3e668: Neither blocks nor log segments found. Creating new log.
I20250109 08:56:30.426625 20461 leader_election.cc:304] T bc8cfc045acc40efaa744f955d08ea97 P ff76b9242fda4d4586c2bb3da829b1ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c5bffb8e4c5e4e7a8b4e919c98c26e58, ff76b9242fda4d4586c2bb3da829b1ba; no voters:
I20250109 08:56:30.428354 20573 raft_consensus.cc:2463] T bc8cfc045acc40efaa744f955d08ea97 P b17b595d5da94e5dbf96bbef1ef3e668 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff76b9242fda4d4586c2bb3da829b1ba in term 1.
I20250109 08:56:30.432108 20716 raft_consensus.cc:491] T db23d411a3a14ed69a1db0b2c8912807 P ff76b9242fda4d4586c2bb3da829b1ba [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:56:30.433485 20716 raft_consensus.cc:513] T db23d411a3a14ed69a1db0b2c8912807 P ff76b9242fda4d4586c2bb3da829b1ba [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:30.434667 20498 raft_consensus.cc:2463] T db23d411a3a14ed69a1db0b2c8912807 P ff76b9242fda4d4586c2bb3da829b1ba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c5bffb8e4c5e4e7a8b4e919c98c26e58 in term 0.
I20250109 08:56:30.436488 20698 raft_consensus.cc:491] T 6f383be53b01468f85e731bd2d4367c0 P b17b595d5da94e5dbf96bbef1ef3e668 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:56:30.436985 20691 tablet_bootstrap.cc:492] T db23d411a3a14ed69a1db0b2c8912807 P b17b595d5da94e5dbf96bbef1ef3e668: No bootstrap required, opened a new log
I20250109 08:56:30.437479 20691 ts_tablet_manager.cc:1397] T db23d411a3a14ed69a1db0b2c8912807 P b17b595d5da94e5dbf96bbef1ef3e668: Time spent bootstrapping tablet: real 0.034s user 0.014s sys 0.000s
I20250109 08:56:30.437289 20613 leader_election.cc:304] T db23d411a3a14ed69a1db0b2c8912807 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c5bffb8e4c5e4e7a8b4e919c98c26e58, ff76b9242fda4d4586c2bb3da829b1ba; no voters: b17b595d5da94e5dbf96bbef1ef3e668
I20250109 08:56:30.438140 20696 raft_consensus.cc:2798] T db23d411a3a14ed69a1db0b2c8912807 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:56:30.438477 20696 raft_consensus.cc:491] T db23d411a3a14ed69a1db0b2c8912807 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:56:30.438817 20696 raft_consensus.cc:3054] T db23d411a3a14ed69a1db0b2c8912807 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:56:30.436964 20698 raft_consensus.cc:513] T 6f383be53b01468f85e731bd2d4367c0 P b17b595d5da94e5dbf96bbef1ef3e668 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:30.447630 20716 leader_election.cc:290] T db23d411a3a14ed69a1db0b2c8912807 P ff76b9242fda4d4586c2bb3da829b1ba [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b17b595d5da94e5dbf96bbef1ef3e668 (127.19.226.194:42855), c5bffb8e4c5e4e7a8b4e919c98c26e58 (127.19.226.195:45649)
I20250109 08:56:30.449169 20573 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db23d411a3a14ed69a1db0b2c8912807" candidate_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" is_pre_election: true
I20250109 08:56:30.449548 20716 raft_consensus.cc:2798] T bc8cfc045acc40efaa744f955d08ea97 P ff76b9242fda4d4586c2bb3da829b1ba [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:56:30.450002 20716 raft_consensus.cc:695] T bc8cfc045acc40efaa744f955d08ea97 P ff76b9242fda4d4586c2bb3da829b1ba [term 1 LEADER]: Becoming Leader. State: Replica: ff76b9242fda4d4586c2bb3da829b1ba, State: Running, Role: LEADER
I20250109 08:56:30.450851 20716 consensus_queue.cc:237] T bc8cfc045acc40efaa744f955d08ea97 P ff76b9242fda4d4586c2bb3da829b1ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:30.456488 20698 leader_election.cc:290] T 6f383be53b01468f85e731bd2d4367c0 P b17b595d5da94e5dbf96bbef1ef3e668 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ff76b9242fda4d4586c2bb3da829b1ba (127.19.226.193:42763), c5bffb8e4c5e4e7a8b4e919c98c26e58 (127.19.226.195:45649)
I20250109 08:56:30.456746 20690 tablet_bootstrap.cc:492] T 6b1d1214e23d47ff8ec02197a45d8484 P c5bffb8e4c5e4e7a8b4e919c98c26e58: No bootstrap required, opened a new log
I20250109 08:56:30.463014 20697 raft_consensus.cc:513] T 6b1d1214e23d47ff8ec02197a45d8484 P ff76b9242fda4d4586c2bb3da829b1ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:30.461369 20690 ts_tablet_manager.cc:1397] T 6b1d1214e23d47ff8ec02197a45d8484 P c5bffb8e4c5e4e7a8b4e919c98c26e58: Time spent bootstrapping tablet: real 0.060s user 0.005s sys 0.014s
I20250109 08:56:30.468555 20697 leader_election.cc:290] T 6b1d1214e23d47ff8ec02197a45d8484 P ff76b9242fda4d4586c2bb3da829b1ba [CANDIDATE]: Term 1 election: Requested vote from peers b17b595d5da94e5dbf96bbef1ef3e668 (127.19.226.194:42855), c5bffb8e4c5e4e7a8b4e919c98c26e58 (127.19.226.195:45649)
I20250109 08:56:30.472180 20692 ts_tablet_manager.cc:1428] T 4173445b18a24674ad20fe4b7e17c697 P ff76b9242fda4d4586c2bb3da829b1ba: Time spent starting tablet: real 0.061s user 0.012s sys 0.023s
I20250109 08:56:30.471447 20696 raft_consensus.cc:513] T db23d411a3a14ed69a1db0b2c8912807 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:30.473304 20692 tablet_bootstrap.cc:492] T bdeeebda65be4889bf59888b7a7abfec P ff76b9242fda4d4586c2bb3da829b1ba: Bootstrap starting.
W20250109 08:56:30.473748 20461 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.226.195:45649, user_credentials={real_user=slave}} blocked reactor thread for 47197.7us
I20250109 08:56:30.477825 20647 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db23d411a3a14ed69a1db0b2c8912807" candidate_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" is_pre_election: true
I20250109 08:56:30.478940 20647 raft_consensus.cc:2388] T db23d411a3a14ed69a1db0b2c8912807 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff76b9242fda4d4586c2bb3da829b1ba in current term 1: Already voted for candidate c5bffb8e4c5e4e7a8b4e919c98c26e58 in this term.
I20250109 08:56:30.479360 20696 leader_election.cc:290] T db23d411a3a14ed69a1db0b2c8912807 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [CANDIDATE]: Term 1 election: Requested vote from peers ff76b9242fda4d4586c2bb3da829b1ba (127.19.226.193:42763), b17b595d5da94e5dbf96bbef1ef3e668 (127.19.226.194:42855)
I20250109 08:56:30.479305 20498 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db23d411a3a14ed69a1db0b2c8912807" candidate_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff76b9242fda4d4586c2bb3da829b1ba"
I20250109 08:56:30.480688 20717 raft_consensus.cc:491] T 5378208cf9ad43129b6276fbb65af974 P ff76b9242fda4d4586c2bb3da829b1ba [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:56:30.480751 20646 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b1d1214e23d47ff8ec02197a45d8484" candidate_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58"
I20250109 08:56:30.481907 20498 raft_consensus.cc:3054] T db23d411a3a14ed69a1db0b2c8912807 P ff76b9242fda4d4586c2bb3da829b1ba [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:56:30.481633 20717 raft_consensus.cc:513] T 5378208cf9ad43129b6276fbb65af974 P ff76b9242fda4d4586c2bb3da829b1ba [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:30.486941 20691 raft_consensus.cc:357] T db23d411a3a14ed69a1db0b2c8912807 P b17b595d5da94e5dbf96bbef1ef3e668 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:30.487746 20691 raft_consensus.cc:383] T db23d411a3a14ed69a1db0b2c8912807 P b17b595d5da94e5dbf96bbef1ef3e668 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:56:30.488047 20691 raft_consensus.cc:738] T db23d411a3a14ed69a1db0b2c8912807 P b17b595d5da94e5dbf96bbef1ef3e668 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b17b595d5da94e5dbf96bbef1ef3e668, State: Initialized, Role: FOLLOWER
I20250109 08:56:30.489185 20717 leader_election.cc:290] T 5378208cf9ad43129b6276fbb65af974 P ff76b9242fda4d4586c2bb3da829b1ba [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b17b595d5da94e5dbf96bbef1ef3e668 (127.19.226.194:42855), c5bffb8e4c5e4e7a8b4e919c98c26e58 (127.19.226.195:45649)
I20250109 08:56:30.492666 20691 consensus_queue.cc:260] T db23d411a3a14ed69a1db0b2c8912807 P b17b595d5da94e5dbf96bbef1ef3e668 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
W20250109 08:56:30.495307 20461 leader_election.cc:343] T 6b1d1214e23d47ff8ec02197a45d8484 P ff76b9242fda4d4586c2bb3da829b1ba [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c5bffb8e4c5e4e7a8b4e919c98c26e58 (127.19.226.195:45649): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:56:30.496562 20698 raft_consensus.cc:491] T cfbc2774b91c40d59dc80655451f5bdd P b17b595d5da94e5dbf96bbef1ef3e668 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:56:30.497071 20691 ts_tablet_manager.cc:1428] T db23d411a3a14ed69a1db0b2c8912807 P b17b595d5da94e5dbf96bbef1ef3e668: Time spent starting tablet: real 0.059s user 0.005s sys 0.004s
I20250109 08:56:30.498296 20692 tablet_bootstrap.cc:654] T bdeeebda65be4889bf59888b7a7abfec P ff76b9242fda4d4586c2bb3da829b1ba: Neither blocks nor log segments found. Creating new log.
I20250109 08:56:30.499951 20698 raft_consensus.cc:513] T cfbc2774b91c40d59dc80655451f5bdd P b17b595d5da94e5dbf96bbef1ef3e668 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:30.500947 20646 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5378208cf9ad43129b6276fbb65af974" candidate_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" is_pre_election: true
I20250109 08:56:30.502276 20698 leader_election.cc:290] T cfbc2774b91c40d59dc80655451f5bdd P b17b595d5da94e5dbf96bbef1ef3e668 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ff76b9242fda4d4586c2bb3da829b1ba (127.19.226.193:42763), c5bffb8e4c5e4e7a8b4e919c98c26e58 (127.19.226.195:45649)
I20250109 08:56:30.493165 20690 raft_consensus.cc:357] T 6b1d1214e23d47ff8ec02197a45d8484 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:30.502769 20690 raft_consensus.cc:383] T 6b1d1214e23d47ff8ec02197a45d8484 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:56:30.503068 20690 raft_consensus.cc:738] T 6b1d1214e23d47ff8ec02197a45d8484 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5bffb8e4c5e4e7a8b4e919c98c26e58, State: Initialized, Role: FOLLOWER
I20250109 08:56:30.500905 20691 tablet_bootstrap.cc:492] T 4173445b18a24674ad20fe4b7e17c697 P b17b595d5da94e5dbf96bbef1ef3e668: Bootstrap starting.
I20250109 08:56:30.505964 20498 raft_consensus.cc:2463] T db23d411a3a14ed69a1db0b2c8912807 P ff76b9242fda4d4586c2bb3da829b1ba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c5bffb8e4c5e4e7a8b4e919c98c26e58 in term 1.
I20250109 08:56:30.506822 20573 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db23d411a3a14ed69a1db0b2c8912807" candidate_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b17b595d5da94e5dbf96bbef1ef3e668"
I20250109 08:56:30.507373 20573 raft_consensus.cc:3054] T db23d411a3a14ed69a1db0b2c8912807 P b17b595d5da94e5dbf96bbef1ef3e668 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:56:30.504323 20690 consensus_queue.cc:260] T 6b1d1214e23d47ff8ec02197a45d8484 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:30.514357 20690 ts_tablet_manager.cc:1428] T 6b1d1214e23d47ff8ec02197a45d8484 P c5bffb8e4c5e4e7a8b4e919c98c26e58: Time spent starting tablet: real 0.050s user 0.008s sys 0.000s
I20250109 08:56:30.514906 20613 leader_election.cc:304] T db23d411a3a14ed69a1db0b2c8912807 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c5bffb8e4c5e4e7a8b4e919c98c26e58, ff76b9242fda4d4586c2bb3da829b1ba; no voters:
I20250109 08:56:30.515282 20690 tablet_bootstrap.cc:492] T 5378208cf9ad43129b6276fbb65af974 P c5bffb8e4c5e4e7a8b4e919c98c26e58: Bootstrap starting.
I20250109 08:56:30.515651 20696 raft_consensus.cc:2798] T db23d411a3a14ed69a1db0b2c8912807 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 1 FOLLOWER]: Leader election won for term 1
W20250109 08:56:30.504990 20461 leader_election.cc:343] T 5378208cf9ad43129b6276fbb65af974 P ff76b9242fda4d4586c2bb3da829b1ba [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c5bffb8e4c5e4e7a8b4e919c98c26e58 (127.19.226.195:45649): Illegal state: must be running to vote when last-logged opid is not known
W20250109 08:56:30.526108 20462 leader_election.cc:343] T db23d411a3a14ed69a1db0b2c8912807 P ff76b9242fda4d4586c2bb3da829b1ba [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b17b595d5da94e5dbf96bbef1ef3e668 (127.19.226.194:42855): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:56:30.527522 20462 leader_election.cc:304] T db23d411a3a14ed69a1db0b2c8912807 P ff76b9242fda4d4586c2bb3da829b1ba [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff76b9242fda4d4586c2bb3da829b1ba; no voters: b17b595d5da94e5dbf96bbef1ef3e668, c5bffb8e4c5e4e7a8b4e919c98c26e58
I20250109 08:56:30.528646 20717 raft_consensus.cc:2743] T db23d411a3a14ed69a1db0b2c8912807 P ff76b9242fda4d4586c2bb3da829b1ba [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:56:30.536098 20498 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f383be53b01468f85e731bd2d4367c0" candidate_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" is_pre_election: true
I20250109 08:56:30.539583 20572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b1d1214e23d47ff8ec02197a45d8484" candidate_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b17b595d5da94e5dbf96bbef1ef3e668"
I20250109 08:56:30.540237 20572 raft_consensus.cc:3054] T 6b1d1214e23d47ff8ec02197a45d8484 P b17b595d5da94e5dbf96bbef1ef3e668 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:56:30.558010 20498 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfbc2774b91c40d59dc80655451f5bdd" candidate_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" is_pre_election: true
I20250109 08:56:30.558719 20691 tablet_bootstrap.cc:654] T 4173445b18a24674ad20fe4b7e17c697 P b17b595d5da94e5dbf96bbef1ef3e668: Neither blocks nor log segments found. Creating new log.
I20250109 08:56:30.559101 20696 raft_consensus.cc:695] T db23d411a3a14ed69a1db0b2c8912807 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 1 LEADER]: Becoming Leader. State: Replica: c5bffb8e4c5e4e7a8b4e919c98c26e58, State: Running, Role: LEADER
I20250109 08:56:30.564924 20571 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5378208cf9ad43129b6276fbb65af974" candidate_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" is_pre_election: true
I20250109 08:56:30.588775 20573 raft_consensus.cc:2463] T db23d411a3a14ed69a1db0b2c8912807 P b17b595d5da94e5dbf96bbef1ef3e668 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c5bffb8e4c5e4e7a8b4e919c98c26e58 in term 1.
I20250109 08:56:30.589479 20690 tablet_bootstrap.cc:654] T 5378208cf9ad43129b6276fbb65af974 P c5bffb8e4c5e4e7a8b4e919c98c26e58: Neither blocks nor log segments found. Creating new log.
W20250109 08:56:30.596319 20462 leader_election.cc:343] T 5378208cf9ad43129b6276fbb65af974 P ff76b9242fda4d4586c2bb3da829b1ba [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b17b595d5da94e5dbf96bbef1ef3e668 (127.19.226.194:42855): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:56:30.590840 20572 raft_consensus.cc:2463] T 6b1d1214e23d47ff8ec02197a45d8484 P b17b595d5da94e5dbf96bbef1ef3e668 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff76b9242fda4d4586c2bb3da829b1ba in term 1.
I20250109 08:56:30.596925 20462 leader_election.cc:304] T 5378208cf9ad43129b6276fbb65af974 P ff76b9242fda4d4586c2bb3da829b1ba [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff76b9242fda4d4586c2bb3da829b1ba; no voters: b17b595d5da94e5dbf96bbef1ef3e668, c5bffb8e4c5e4e7a8b4e919c98c26e58
I20250109 08:56:30.598526 20716 raft_consensus.cc:2743] T 5378208cf9ad43129b6276fbb65af974 P ff76b9242fda4d4586c2bb3da829b1ba [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:56:30.590549 20696 consensus_queue.cc:237] T db23d411a3a14ed69a1db0b2c8912807 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:30.601377 20692 tablet_bootstrap.cc:492] T bdeeebda65be4889bf59888b7a7abfec P ff76b9242fda4d4586c2bb3da829b1ba: No bootstrap required, opened a new log
I20250109 08:56:30.601924 20462 leader_election.cc:304] T 6b1d1214e23d47ff8ec02197a45d8484 P ff76b9242fda4d4586c2bb3da829b1ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b17b595d5da94e5dbf96bbef1ef3e668, ff76b9242fda4d4586c2bb3da829b1ba; no voters: c5bffb8e4c5e4e7a8b4e919c98c26e58
I20250109 08:56:30.603772 20692 ts_tablet_manager.cc:1397] T bdeeebda65be4889bf59888b7a7abfec P ff76b9242fda4d4586c2bb3da829b1ba: Time spent bootstrapping tablet: real 0.131s user 0.012s sys 0.019s
I20250109 08:56:30.604197 20716 raft_consensus.cc:2798] T 6b1d1214e23d47ff8ec02197a45d8484 P ff76b9242fda4d4586c2bb3da829b1ba [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:56:30.604811 20716 raft_consensus.cc:695] T 6b1d1214e23d47ff8ec02197a45d8484 P ff76b9242fda4d4586c2bb3da829b1ba [term 1 LEADER]: Becoming Leader. State: Replica: ff76b9242fda4d4586c2bb3da829b1ba, State: Running, Role: LEADER
I20250109 08:56:30.604866 20646 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f383be53b01468f85e731bd2d4367c0" candidate_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" is_pre_election: true
I20250109 08:56:30.605633 20716 consensus_queue.cc:237] T 6b1d1214e23d47ff8ec02197a45d8484 P ff76b9242fda4d4586c2bb3da829b1ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:30.606051 20647 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfbc2774b91c40d59dc80655451f5bdd" candidate_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" is_pre_election: true
I20250109 08:56:30.601783 20391 catalog_manager.cc:5526] T bc8cfc045acc40efaa744f955d08ea97 P ff76b9242fda4d4586c2bb3da829b1ba reported cstate change: term changed from 0 to 1, leader changed from <none> to ff76b9242fda4d4586c2bb3da829b1ba (127.19.226.193). New cstate: current_term: 1 leader_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } health_report { overall_health: UNKNOWN } } }
W20250109 08:56:30.610499 20539 leader_election.cc:343] T 6f383be53b01468f85e731bd2d4367c0 P b17b595d5da94e5dbf96bbef1ef3e668 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ff76b9242fda4d4586c2bb3da829b1ba (127.19.226.193:42763): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:56:30.606885 20692 raft_consensus.cc:357] T bdeeebda65be4889bf59888b7a7abfec P ff76b9242fda4d4586c2bb3da829b1ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:30.613018 20692 raft_consensus.cc:383] T bdeeebda65be4889bf59888b7a7abfec P ff76b9242fda4d4586c2bb3da829b1ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:56:30.616344 20692 raft_consensus.cc:738] T bdeeebda65be4889bf59888b7a7abfec P ff76b9242fda4d4586c2bb3da829b1ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff76b9242fda4d4586c2bb3da829b1ba, State: Initialized, Role: FOLLOWER
W20250109 08:56:30.616462 20536 leader_election.cc:343] T 6f383be53b01468f85e731bd2d4367c0 P b17b595d5da94e5dbf96bbef1ef3e668 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c5bffb8e4c5e4e7a8b4e919c98c26e58 (127.19.226.195:45649): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:56:30.617185 20692 consensus_queue.cc:260] T bdeeebda65be4889bf59888b7a7abfec P ff76b9242fda4d4586c2bb3da829b1ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:30.614806 20691 tablet_bootstrap.cc:492] T 4173445b18a24674ad20fe4b7e17c697 P b17b595d5da94e5dbf96bbef1ef3e668: No bootstrap required, opened a new log
I20250109 08:56:30.618789 20691 ts_tablet_manager.cc:1397] T 4173445b18a24674ad20fe4b7e17c697 P b17b595d5da94e5dbf96bbef1ef3e668: Time spent bootstrapping tablet: real 0.118s user 0.007s sys 0.022s
W20250109 08:56:30.615134 20539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.226.193:42763, user_credentials={real_user=slave}} blocked reactor thread for 49382.3us
W20250109 08:56:30.620781 20539 leader_election.cc:343] T cfbc2774b91c40d59dc80655451f5bdd P b17b595d5da94e5dbf96bbef1ef3e668 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ff76b9242fda4d4586c2bb3da829b1ba (127.19.226.193:42763): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:56:30.618705 20536 leader_election.cc:304] T 6f383be53b01468f85e731bd2d4367c0 P b17b595d5da94e5dbf96bbef1ef3e668 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b17b595d5da94e5dbf96bbef1ef3e668; no voters: c5bffb8e4c5e4e7a8b4e919c98c26e58, ff76b9242fda4d4586c2bb3da829b1ba
I20250109 08:56:30.621716 20698 raft_consensus.cc:2743] T 6f383be53b01468f85e731bd2d4367c0 P b17b595d5da94e5dbf96bbef1ef3e668 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20250109 08:56:30.622056 20536 leader_election.cc:343] T cfbc2774b91c40d59dc80655451f5bdd P b17b595d5da94e5dbf96bbef1ef3e668 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c5bffb8e4c5e4e7a8b4e919c98c26e58 (127.19.226.195:45649): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:56:30.623848 20690 tablet_bootstrap.cc:492] T 5378208cf9ad43129b6276fbb65af974 P c5bffb8e4c5e4e7a8b4e919c98c26e58: No bootstrap required, opened a new log
I20250109 08:56:30.625591 20690 ts_tablet_manager.cc:1397] T 5378208cf9ad43129b6276fbb65af974 P c5bffb8e4c5e4e7a8b4e919c98c26e58: Time spent bootstrapping tablet: real 0.110s user 0.004s sys 0.022s
I20250109 08:56:30.625605 20536 leader_election.cc:304] T cfbc2774b91c40d59dc80655451f5bdd P b17b595d5da94e5dbf96bbef1ef3e668 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b17b595d5da94e5dbf96bbef1ef3e668; no voters: c5bffb8e4c5e4e7a8b4e919c98c26e58, ff76b9242fda4d4586c2bb3da829b1ba
I20250109 08:56:30.622076 20692 ts_tablet_manager.cc:1428] T bdeeebda65be4889bf59888b7a7abfec P ff76b9242fda4d4586c2bb3da829b1ba: Time spent starting tablet: real 0.018s user 0.000s sys 0.006s
I20250109 08:56:30.626896 20698 raft_consensus.cc:2743] T cfbc2774b91c40d59dc80655451f5bdd P b17b595d5da94e5dbf96bbef1ef3e668 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:56:30.627566 20692 tablet_bootstrap.cc:492] T 6f383be53b01468f85e731bd2d4367c0 P ff76b9242fda4d4586c2bb3da829b1ba: Bootstrap starting.
I20250109 08:56:30.624408 20691 raft_consensus.cc:357] T 4173445b18a24674ad20fe4b7e17c697 P b17b595d5da94e5dbf96bbef1ef3e668 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:30.630151 20691 raft_consensus.cc:383] T 4173445b18a24674ad20fe4b7e17c697 P b17b595d5da94e5dbf96bbef1ef3e668 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:56:30.629856 20690 raft_consensus.cc:357] T 5378208cf9ad43129b6276fbb65af974 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:30.634056 20690 raft_consensus.cc:383] T 5378208cf9ad43129b6276fbb65af974 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:56:30.634436 20690 raft_consensus.cc:738] T 5378208cf9ad43129b6276fbb65af974 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5bffb8e4c5e4e7a8b4e919c98c26e58, State: Initialized, Role: FOLLOWER
I20250109 08:56:30.635210 20690 consensus_queue.cc:260] T 5378208cf9ad43129b6276fbb65af974 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:30.636693 20724 raft_consensus.cc:491] T 4173445b18a24674ad20fe4b7e17c697 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:56:30.638181 20724 raft_consensus.cc:513] T 4173445b18a24674ad20fe4b7e17c697 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:30.640547 20724 leader_election.cc:290] T 4173445b18a24674ad20fe4b7e17c697 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ff76b9242fda4d4586c2bb3da829b1ba (127.19.226.193:42763), b17b595d5da94e5dbf96bbef1ef3e668 (127.19.226.194:42855)
I20250109 08:56:30.641386 20498 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4173445b18a24674ad20fe4b7e17c697" candidate_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" is_pre_election: true
I20250109 08:56:30.642100 20498 raft_consensus.cc:2463] T 4173445b18a24674ad20fe4b7e17c697 P ff76b9242fda4d4586c2bb3da829b1ba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c5bffb8e4c5e4e7a8b4e919c98c26e58 in term 0.
I20250109 08:56:30.643045 20690 ts_tablet_manager.cc:1428] T 5378208cf9ad43129b6276fbb65af974 P c5bffb8e4c5e4e7a8b4e919c98c26e58: Time spent starting tablet: real 0.017s user 0.000s sys 0.007s
I20250109 08:56:30.642354 20395 catalog_manager.cc:5526] T db23d411a3a14ed69a1db0b2c8912807 P c5bffb8e4c5e4e7a8b4e919c98c26e58 reported cstate change: term changed from 0 to 1, leader changed from <none> to c5bffb8e4c5e4e7a8b4e919c98c26e58 (127.19.226.195). New cstate: current_term: 1 leader_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } health_report { overall_health: HEALTHY } } }
I20250109 08:56:30.644012 20690 tablet_bootstrap.cc:492] T cfbc2774b91c40d59dc80655451f5bdd P c5bffb8e4c5e4e7a8b4e919c98c26e58: Bootstrap starting.
I20250109 08:56:30.643298 20613 leader_election.cc:304] T 4173445b18a24674ad20fe4b7e17c697 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c5bffb8e4c5e4e7a8b4e919c98c26e58, ff76b9242fda4d4586c2bb3da829b1ba; no voters:
I20250109 08:56:30.630455 20691 raft_consensus.cc:738] T 4173445b18a24674ad20fe4b7e17c697 P b17b595d5da94e5dbf96bbef1ef3e668 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b17b595d5da94e5dbf96bbef1ef3e668, State: Initialized, Role: FOLLOWER
I20250109 08:56:30.646067 20724 raft_consensus.cc:2798] T 4173445b18a24674ad20fe4b7e17c697 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:56:30.646435 20724 raft_consensus.cc:491] T 4173445b18a24674ad20fe4b7e17c697 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:56:30.646430 20572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4173445b18a24674ad20fe4b7e17c697" candidate_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" is_pre_election: true
I20250109 08:56:30.646907 20724 raft_consensus.cc:3054] T 4173445b18a24674ad20fe4b7e17c697 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:56:30.653117 20691 consensus_queue.cc:260] T 4173445b18a24674ad20fe4b7e17c697 P b17b595d5da94e5dbf96bbef1ef3e668 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:30.655071 20572 raft_consensus.cc:2463] T 4173445b18a24674ad20fe4b7e17c697 P b17b595d5da94e5dbf96bbef1ef3e668 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c5bffb8e4c5e4e7a8b4e919c98c26e58 in term 0.
I20250109 08:56:30.656021 20691 ts_tablet_manager.cc:1428] T 4173445b18a24674ad20fe4b7e17c697 P b17b595d5da94e5dbf96bbef1ef3e668: Time spent starting tablet: real 0.037s user 0.007s sys 0.002s
I20250109 08:56:30.657866 20691 tablet_bootstrap.cc:492] T 5378208cf9ad43129b6276fbb65af974 P b17b595d5da94e5dbf96bbef1ef3e668: Bootstrap starting.
I20250109 08:56:30.658651 20724 raft_consensus.cc:513] T 4173445b18a24674ad20fe4b7e17c697 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:30.661962 20498 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4173445b18a24674ad20fe4b7e17c697" candidate_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff76b9242fda4d4586c2bb3da829b1ba"
I20250109 08:56:30.662963 20498 raft_consensus.cc:3054] T 4173445b18a24674ad20fe4b7e17c697 P ff76b9242fda4d4586c2bb3da829b1ba [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:56:30.663355 20692 tablet_bootstrap.cc:654] T 6f383be53b01468f85e731bd2d4367c0 P ff76b9242fda4d4586c2bb3da829b1ba: Neither blocks nor log segments found. Creating new log.
I20250109 08:56:30.665225 20572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4173445b18a24674ad20fe4b7e17c697" candidate_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b17b595d5da94e5dbf96bbef1ef3e668"
I20250109 08:56:30.665916 20572 raft_consensus.cc:3054] T 4173445b18a24674ad20fe4b7e17c697 P b17b595d5da94e5dbf96bbef1ef3e668 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:56:30.667604 20690 tablet_bootstrap.cc:654] T cfbc2774b91c40d59dc80655451f5bdd P c5bffb8e4c5e4e7a8b4e919c98c26e58: Neither blocks nor log segments found. Creating new log.
I20250109 08:56:30.669488 20724 leader_election.cc:290] T 4173445b18a24674ad20fe4b7e17c697 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [CANDIDATE]: Term 1 election: Requested vote from peers ff76b9242fda4d4586c2bb3da829b1ba (127.19.226.193:42763), b17b595d5da94e5dbf96bbef1ef3e668 (127.19.226.194:42855)
I20250109 08:56:30.673040 20572 raft_consensus.cc:2463] T 4173445b18a24674ad20fe4b7e17c697 P b17b595d5da94e5dbf96bbef1ef3e668 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c5bffb8e4c5e4e7a8b4e919c98c26e58 in term 1.
I20250109 08:56:30.673372 20498 raft_consensus.cc:2463] T 4173445b18a24674ad20fe4b7e17c697 P ff76b9242fda4d4586c2bb3da829b1ba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c5bffb8e4c5e4e7a8b4e919c98c26e58 in term 1.
I20250109 08:56:30.676828 20611 leader_election.cc:304] T 4173445b18a24674ad20fe4b7e17c697 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b17b595d5da94e5dbf96bbef1ef3e668, c5bffb8e4c5e4e7a8b4e919c98c26e58; no voters:
I20250109 08:56:30.679414 20724 raft_consensus.cc:2798] T 4173445b18a24674ad20fe4b7e17c697 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:56:30.679952 20724 raft_consensus.cc:695] T 4173445b18a24674ad20fe4b7e17c697 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 1 LEADER]: Becoming Leader. State: Replica: c5bffb8e4c5e4e7a8b4e919c98c26e58, State: Running, Role: LEADER
I20250109 08:56:30.681375 20724 consensus_queue.cc:237] T 4173445b18a24674ad20fe4b7e17c697 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:30.701800 20691 tablet_bootstrap.cc:654] T 5378208cf9ad43129b6276fbb65af974 P b17b595d5da94e5dbf96bbef1ef3e668: Neither blocks nor log segments found. Creating new log.
I20250109 08:56:30.702134 20692 tablet_bootstrap.cc:492] T 6f383be53b01468f85e731bd2d4367c0 P ff76b9242fda4d4586c2bb3da829b1ba: No bootstrap required, opened a new log
I20250109 08:56:30.702638 20692 ts_tablet_manager.cc:1397] T 6f383be53b01468f85e731bd2d4367c0 P ff76b9242fda4d4586c2bb3da829b1ba: Time spent bootstrapping tablet: real 0.075s user 0.022s sys 0.010s
I20250109 08:56:30.707132 20690 tablet_bootstrap.cc:492] T cfbc2774b91c40d59dc80655451f5bdd P c5bffb8e4c5e4e7a8b4e919c98c26e58: No bootstrap required, opened a new log
I20250109 08:56:30.707646 20690 ts_tablet_manager.cc:1397] T cfbc2774b91c40d59dc80655451f5bdd P c5bffb8e4c5e4e7a8b4e919c98c26e58: Time spent bootstrapping tablet: real 0.064s user 0.021s sys 0.017s
I20250109 08:56:30.712656 20391 catalog_manager.cc:5526] T 6b1d1214e23d47ff8ec02197a45d8484 P ff76b9242fda4d4586c2bb3da829b1ba reported cstate change: term changed from 0 to 1, leader changed from <none> to ff76b9242fda4d4586c2bb3da829b1ba (127.19.226.193). New cstate: current_term: 1 leader_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } health_report { overall_health: UNKNOWN } } }
I20250109 08:56:30.715322 20692 raft_consensus.cc:357] T 6f383be53b01468f85e731bd2d4367c0 P ff76b9242fda4d4586c2bb3da829b1ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:30.716104 20692 raft_consensus.cc:383] T 6f383be53b01468f85e731bd2d4367c0 P ff76b9242fda4d4586c2bb3da829b1ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:56:30.716428 20692 raft_consensus.cc:738] T 6f383be53b01468f85e731bd2d4367c0 P ff76b9242fda4d4586c2bb3da829b1ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff76b9242fda4d4586c2bb3da829b1ba, State: Initialized, Role: FOLLOWER
I20250109 08:56:30.717110 20692 consensus_queue.cc:260] T 6f383be53b01468f85e731bd2d4367c0 P ff76b9242fda4d4586c2bb3da829b1ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:30.719508 20692 ts_tablet_manager.cc:1428] T 6f383be53b01468f85e731bd2d4367c0 P ff76b9242fda4d4586c2bb3da829b1ba: Time spent starting tablet: real 0.016s user 0.004s sys 0.000s
I20250109 08:56:30.713598 20690 raft_consensus.cc:357] T cfbc2774b91c40d59dc80655451f5bdd P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:30.720614 20690 raft_consensus.cc:383] T cfbc2774b91c40d59dc80655451f5bdd P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:56:30.720897 20690 raft_consensus.cc:738] T cfbc2774b91c40d59dc80655451f5bdd P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5bffb8e4c5e4e7a8b4e919c98c26e58, State: Initialized, Role: FOLLOWER
I20250109 08:56:30.721671 20690 consensus_queue.cc:260] T cfbc2774b91c40d59dc80655451f5bdd P c5bffb8e4c5e4e7a8b4e919c98c26e58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:30.722452 20691 tablet_bootstrap.cc:492] T 5378208cf9ad43129b6276fbb65af974 P b17b595d5da94e5dbf96bbef1ef3e668: No bootstrap required, opened a new log
I20250109 08:56:30.722880 20691 ts_tablet_manager.cc:1397] T 5378208cf9ad43129b6276fbb65af974 P b17b595d5da94e5dbf96bbef1ef3e668: Time spent bootstrapping tablet: real 0.065s user 0.011s sys 0.002s
I20250109 08:56:30.724247 20690 ts_tablet_manager.cc:1428] T cfbc2774b91c40d59dc80655451f5bdd P c5bffb8e4c5e4e7a8b4e919c98c26e58: Time spent starting tablet: real 0.016s user 0.008s sys 0.000s
I20250109 08:56:30.725210 20690 tablet_bootstrap.cc:492] T 74c881b69c3946429070c729ec66b0e4 P c5bffb8e4c5e4e7a8b4e919c98c26e58: Bootstrap starting.
I20250109 08:56:30.725580 20691 raft_consensus.cc:357] T 5378208cf9ad43129b6276fbb65af974 P b17b595d5da94e5dbf96bbef1ef3e668 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:30.729466 20691 raft_consensus.cc:383] T 5378208cf9ad43129b6276fbb65af974 P b17b595d5da94e5dbf96bbef1ef3e668 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:56:30.729779 20691 raft_consensus.cc:738] T 5378208cf9ad43129b6276fbb65af974 P b17b595d5da94e5dbf96bbef1ef3e668 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b17b595d5da94e5dbf96bbef1ef3e668, State: Initialized, Role: FOLLOWER
I20250109 08:56:30.730482 20691 consensus_queue.cc:260] T 5378208cf9ad43129b6276fbb65af974 P b17b595d5da94e5dbf96bbef1ef3e668 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:30.732544 20692 tablet_bootstrap.cc:492] T cfbc2774b91c40d59dc80655451f5bdd P ff76b9242fda4d4586c2bb3da829b1ba: Bootstrap starting.
I20250109 08:56:30.733325 20691 ts_tablet_manager.cc:1428] T 5378208cf9ad43129b6276fbb65af974 P b17b595d5da94e5dbf96bbef1ef3e668: Time spent starting tablet: real 0.010s user 0.005s sys 0.001s
I20250109 08:56:30.734229 20691 tablet_bootstrap.cc:492] T bdeeebda65be4889bf59888b7a7abfec P b17b595d5da94e5dbf96bbef1ef3e668: Bootstrap starting.
I20250109 08:56:30.738832 20692 tablet_bootstrap.cc:654] T cfbc2774b91c40d59dc80655451f5bdd P ff76b9242fda4d4586c2bb3da829b1ba: Neither blocks nor log segments found. Creating new log.
I20250109 08:56:30.740288 20691 tablet_bootstrap.cc:654] T bdeeebda65be4889bf59888b7a7abfec P b17b595d5da94e5dbf96bbef1ef3e668: Neither blocks nor log segments found. Creating new log.
I20250109 08:56:30.732234 20690 tablet_bootstrap.cc:654] T 74c881b69c3946429070c729ec66b0e4 P c5bffb8e4c5e4e7a8b4e919c98c26e58: Neither blocks nor log segments found. Creating new log.
I20250109 08:56:30.743738 20393 catalog_manager.cc:5526] T 4173445b18a24674ad20fe4b7e17c697 P c5bffb8e4c5e4e7a8b4e919c98c26e58 reported cstate change: term changed from 0 to 1, leader changed from <none> to c5bffb8e4c5e4e7a8b4e919c98c26e58 (127.19.226.195). New cstate: current_term: 1 leader_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } health_report { overall_health: HEALTHY } } }
I20250109 08:56:30.750541 20724 raft_consensus.cc:491] T 5378208cf9ad43129b6276fbb65af974 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:56:30.751659 20724 raft_consensus.cc:513] T 5378208cf9ad43129b6276fbb65af974 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:30.759006 20724 leader_election.cc:290] T 5378208cf9ad43129b6276fbb65af974 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ff76b9242fda4d4586c2bb3da829b1ba (127.19.226.193:42763), b17b595d5da94e5dbf96bbef1ef3e668 (127.19.226.194:42855)
I20250109 08:56:30.760573 20498 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5378208cf9ad43129b6276fbb65af974" candidate_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" is_pre_election: true
I20250109 08:56:30.760802 20572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5378208cf9ad43129b6276fbb65af974" candidate_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" is_pre_election: true
I20250109 08:56:30.761430 20498 raft_consensus.cc:2463] T 5378208cf9ad43129b6276fbb65af974 P ff76b9242fda4d4586c2bb3da829b1ba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c5bffb8e4c5e4e7a8b4e919c98c26e58 in term 0.
I20250109 08:56:30.761554 20572 raft_consensus.cc:2463] T 5378208cf9ad43129b6276fbb65af974 P b17b595d5da94e5dbf96bbef1ef3e668 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c5bffb8e4c5e4e7a8b4e919c98c26e58 in term 0.
I20250109 08:56:30.762881 20692 tablet_bootstrap.cc:492] T cfbc2774b91c40d59dc80655451f5bdd P ff76b9242fda4d4586c2bb3da829b1ba: No bootstrap required, opened a new log
I20250109 08:56:30.763362 20692 ts_tablet_manager.cc:1397] T cfbc2774b91c40d59dc80655451f5bdd P ff76b9242fda4d4586c2bb3da829b1ba: Time spent bootstrapping tablet: real 0.043s user 0.016s sys 0.006s
I20250109 08:56:30.766229 20692 raft_consensus.cc:357] T cfbc2774b91c40d59dc80655451f5bdd P ff76b9242fda4d4586c2bb3da829b1ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:30.766973 20692 raft_consensus.cc:383] T cfbc2774b91c40d59dc80655451f5bdd P ff76b9242fda4d4586c2bb3da829b1ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:56:30.767262 20692 raft_consensus.cc:738] T cfbc2774b91c40d59dc80655451f5bdd P ff76b9242fda4d4586c2bb3da829b1ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff76b9242fda4d4586c2bb3da829b1ba, State: Initialized, Role: FOLLOWER
I20250109 08:56:30.769116 20692 consensus_queue.cc:260] T cfbc2774b91c40d59dc80655451f5bdd P ff76b9242fda4d4586c2bb3da829b1ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:30.771005 20691 tablet_bootstrap.cc:492] T bdeeebda65be4889bf59888b7a7abfec P b17b595d5da94e5dbf96bbef1ef3e668: No bootstrap required, opened a new log
I20250109 08:56:30.771456 20692 ts_tablet_manager.cc:1428] T cfbc2774b91c40d59dc80655451f5bdd P ff76b9242fda4d4586c2bb3da829b1ba: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20250109 08:56:30.771423 20691 ts_tablet_manager.cc:1397] T bdeeebda65be4889bf59888b7a7abfec P b17b595d5da94e5dbf96bbef1ef3e668: Time spent bootstrapping tablet: real 0.037s user 0.010s sys 0.000s
I20250109 08:56:30.772733 20692 tablet_bootstrap.cc:492] T 74c881b69c3946429070c729ec66b0e4 P ff76b9242fda4d4586c2bb3da829b1ba: Bootstrap starting.
I20250109 08:56:30.773077 20611 leader_election.cc:304] T 5378208cf9ad43129b6276fbb65af974 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b17b595d5da94e5dbf96bbef1ef3e668, c5bffb8e4c5e4e7a8b4e919c98c26e58; no voters:
I20250109 08:56:30.774302 20724 raft_consensus.cc:2798] T 5378208cf9ad43129b6276fbb65af974 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:56:30.774654 20724 raft_consensus.cc:491] T 5378208cf9ad43129b6276fbb65af974 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:56:30.775027 20724 raft_consensus.cc:3054] T 5378208cf9ad43129b6276fbb65af974 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:56:30.778023 20690 tablet_bootstrap.cc:492] T 74c881b69c3946429070c729ec66b0e4 P c5bffb8e4c5e4e7a8b4e919c98c26e58: No bootstrap required, opened a new log
I20250109 08:56:30.778465 20690 ts_tablet_manager.cc:1397] T 74c881b69c3946429070c729ec66b0e4 P c5bffb8e4c5e4e7a8b4e919c98c26e58: Time spent bootstrapping tablet: real 0.053s user 0.013s sys 0.015s
I20250109 08:56:30.778105 20691 raft_consensus.cc:357] T bdeeebda65be4889bf59888b7a7abfec P b17b595d5da94e5dbf96bbef1ef3e668 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:30.779381 20691 raft_consensus.cc:383] T bdeeebda65be4889bf59888b7a7abfec P b17b595d5da94e5dbf96bbef1ef3e668 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:56:30.779601 20691 raft_consensus.cc:738] T bdeeebda65be4889bf59888b7a7abfec P b17b595d5da94e5dbf96bbef1ef3e668 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b17b595d5da94e5dbf96bbef1ef3e668, State: Initialized, Role: FOLLOWER
I20250109 08:56:30.780033 20691 consensus_queue.cc:260] T bdeeebda65be4889bf59888b7a7abfec P b17b595d5da94e5dbf96bbef1ef3e668 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:30.793062 20691 ts_tablet_manager.cc:1428] T bdeeebda65be4889bf59888b7a7abfec P b17b595d5da94e5dbf96bbef1ef3e668: Time spent starting tablet: real 0.021s user 0.005s sys 0.000s
I20250109 08:56:30.794721 20692 tablet_bootstrap.cc:654] T 74c881b69c3946429070c729ec66b0e4 P ff76b9242fda4d4586c2bb3da829b1ba: Neither blocks nor log segments found. Creating new log.
I20250109 08:56:30.795711 20724 raft_consensus.cc:513] T 5378208cf9ad43129b6276fbb65af974 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:30.794659 20690 raft_consensus.cc:357] T 74c881b69c3946429070c729ec66b0e4 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:30.797703 20724 leader_election.cc:290] T 5378208cf9ad43129b6276fbb65af974 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [CANDIDATE]: Term 1 election: Requested vote from peers ff76b9242fda4d4586c2bb3da829b1ba (127.19.226.193:42763), b17b595d5da94e5dbf96bbef1ef3e668 (127.19.226.194:42855)
I20250109 08:56:30.798794 20690 raft_consensus.cc:383] T 74c881b69c3946429070c729ec66b0e4 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:56:30.799440 20690 raft_consensus.cc:738] T 74c881b69c3946429070c729ec66b0e4 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5bffb8e4c5e4e7a8b4e919c98c26e58, State: Initialized, Role: FOLLOWER
I20250109 08:56:30.800171 20690 consensus_queue.cc:260] T 74c881b69c3946429070c729ec66b0e4 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:30.801692 20572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5378208cf9ad43129b6276fbb65af974" candidate_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b17b595d5da94e5dbf96bbef1ef3e668"
I20250109 08:56:30.802529 20690 ts_tablet_manager.cc:1428] T 74c881b69c3946429070c729ec66b0e4 P c5bffb8e4c5e4e7a8b4e919c98c26e58: Time spent starting tablet: real 0.024s user 0.006s sys 0.001s
I20250109 08:56:30.800091 20498 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5378208cf9ad43129b6276fbb65af974" candidate_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff76b9242fda4d4586c2bb3da829b1ba"
I20250109 08:56:30.803236 20498 raft_consensus.cc:3054] T 5378208cf9ad43129b6276fbb65af974 P ff76b9242fda4d4586c2bb3da829b1ba [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:56:30.803541 20690 tablet_bootstrap.cc:492] T 6f383be53b01468f85e731bd2d4367c0 P c5bffb8e4c5e4e7a8b4e919c98c26e58: Bootstrap starting.
I20250109 08:56:30.809444 20690 tablet_bootstrap.cc:654] T 6f383be53b01468f85e731bd2d4367c0 P c5bffb8e4c5e4e7a8b4e919c98c26e58: Neither blocks nor log segments found. Creating new log.
I20250109 08:56:30.802410 20572 raft_consensus.cc:3054] T 5378208cf9ad43129b6276fbb65af974 P b17b595d5da94e5dbf96bbef1ef3e668 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:56:30.814075 20692 tablet_bootstrap.cc:492] T 74c881b69c3946429070c729ec66b0e4 P ff76b9242fda4d4586c2bb3da829b1ba: No bootstrap required, opened a new log
I20250109 08:56:30.814507 20691 tablet_bootstrap.cc:492] T 74c881b69c3946429070c729ec66b0e4 P b17b595d5da94e5dbf96bbef1ef3e668: Bootstrap starting.
I20250109 08:56:30.816720 20690 tablet_bootstrap.cc:492] T 6f383be53b01468f85e731bd2d4367c0 P c5bffb8e4c5e4e7a8b4e919c98c26e58: No bootstrap required, opened a new log
I20250109 08:56:30.814508 20692 ts_tablet_manager.cc:1397] T 74c881b69c3946429070c729ec66b0e4 P ff76b9242fda4d4586c2bb3da829b1ba: Time spent bootstrapping tablet: real 0.042s user 0.021s sys 0.011s
I20250109 08:56:30.817166 20690 ts_tablet_manager.cc:1397] T 6f383be53b01468f85e731bd2d4367c0 P c5bffb8e4c5e4e7a8b4e919c98c26e58: Time spent bootstrapping tablet: real 0.014s user 0.009s sys 0.000s
I20250109 08:56:30.819392 20716 raft_consensus.cc:491] T bdeeebda65be4889bf59888b7a7abfec P ff76b9242fda4d4586c2bb3da829b1ba [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:56:30.819902 20716 raft_consensus.cc:513] T bdeeebda65be4889bf59888b7a7abfec P ff76b9242fda4d4586c2bb3da829b1ba [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:30.819995 20690 raft_consensus.cc:357] T 6f383be53b01468f85e731bd2d4367c0 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:30.820760 20690 raft_consensus.cc:383] T 6f383be53b01468f85e731bd2d4367c0 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:56:30.821046 20690 raft_consensus.cc:738] T 6f383be53b01468f85e731bd2d4367c0 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5bffb8e4c5e4e7a8b4e919c98c26e58, State: Initialized, Role: FOLLOWER
I20250109 08:56:30.821856 20716 leader_election.cc:290] T bdeeebda65be4889bf59888b7a7abfec P ff76b9242fda4d4586c2bb3da829b1ba [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b17b595d5da94e5dbf96bbef1ef3e668 (127.19.226.194:42855), c5bffb8e4c5e4e7a8b4e919c98c26e58 (127.19.226.195:45649)
I20250109 08:56:30.821822 20690 consensus_queue.cc:260] T 6f383be53b01468f85e731bd2d4367c0 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:30.823863 20571 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bdeeebda65be4889bf59888b7a7abfec" candidate_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" is_pre_election: true
I20250109 08:56:30.824239 20647 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bdeeebda65be4889bf59888b7a7abfec" candidate_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" is_pre_election: true
I20250109 08:56:30.824491 20690 ts_tablet_manager.cc:1428] T 6f383be53b01468f85e731bd2d4367c0 P c5bffb8e4c5e4e7a8b4e919c98c26e58: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20250109 08:56:30.824566 20571 raft_consensus.cc:2463] T bdeeebda65be4889bf59888b7a7abfec P b17b595d5da94e5dbf96bbef1ef3e668 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff76b9242fda4d4586c2bb3da829b1ba in term 0.
I20250109 08:56:30.824932 20647 raft_consensus.cc:2463] T bdeeebda65be4889bf59888b7a7abfec P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff76b9242fda4d4586c2bb3da829b1ba in term 0.
I20250109 08:56:30.825659 20462 leader_election.cc:304] T bdeeebda65be4889bf59888b7a7abfec P ff76b9242fda4d4586c2bb3da829b1ba [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b17b595d5da94e5dbf96bbef1ef3e668, ff76b9242fda4d4586c2bb3da829b1ba; no voters:
I20250109 08:56:30.826339 20691 tablet_bootstrap.cc:654] T 74c881b69c3946429070c729ec66b0e4 P b17b595d5da94e5dbf96bbef1ef3e668: Neither blocks nor log segments found. Creating new log.
I20250109 08:56:30.826894 20716 raft_consensus.cc:2798] T bdeeebda65be4889bf59888b7a7abfec P ff76b9242fda4d4586c2bb3da829b1ba [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:56:30.827297 20716 raft_consensus.cc:491] T bdeeebda65be4889bf59888b7a7abfec P ff76b9242fda4d4586c2bb3da829b1ba [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:56:30.827688 20716 raft_consensus.cc:3054] T bdeeebda65be4889bf59888b7a7abfec P ff76b9242fda4d4586c2bb3da829b1ba [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:56:30.832499 20572 raft_consensus.cc:2463] T 5378208cf9ad43129b6276fbb65af974 P b17b595d5da94e5dbf96bbef1ef3e668 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c5bffb8e4c5e4e7a8b4e919c98c26e58 in term 1.
I20250109 08:56:30.833811 20611 leader_election.cc:304] T 5378208cf9ad43129b6276fbb65af974 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b17b595d5da94e5dbf96bbef1ef3e668, c5bffb8e4c5e4e7a8b4e919c98c26e58; no voters:
I20250109 08:56:30.834695 20724 raft_consensus.cc:2798] T 5378208cf9ad43129b6276fbb65af974 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:56:30.835140 20724 raft_consensus.cc:695] T 5378208cf9ad43129b6276fbb65af974 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 1 LEADER]: Becoming Leader. State: Replica: c5bffb8e4c5e4e7a8b4e919c98c26e58, State: Running, Role: LEADER
I20250109 08:56:30.835906 20724 consensus_queue.cc:237] T 5378208cf9ad43129b6276fbb65af974 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:30.834178 20716 raft_consensus.cc:513] T bdeeebda65be4889bf59888b7a7abfec P ff76b9242fda4d4586c2bb3da829b1ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:30.821666 20692 raft_consensus.cc:357] T 74c881b69c3946429070c729ec66b0e4 P ff76b9242fda4d4586c2bb3da829b1ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:30.841512 20691 tablet_bootstrap.cc:492] T 74c881b69c3946429070c729ec66b0e4 P b17b595d5da94e5dbf96bbef1ef3e668: No bootstrap required, opened a new log
I20250109 08:56:30.842077 20692 raft_consensus.cc:383] T 74c881b69c3946429070c729ec66b0e4 P ff76b9242fda4d4586c2bb3da829b1ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:56:30.842418 20691 ts_tablet_manager.cc:1397] T 74c881b69c3946429070c729ec66b0e4 P b17b595d5da94e5dbf96bbef1ef3e668: Time spent bootstrapping tablet: real 0.028s user 0.019s sys 0.001s
I20250109 08:56:30.842489 20692 raft_consensus.cc:738] T 74c881b69c3946429070c729ec66b0e4 P ff76b9242fda4d4586c2bb3da829b1ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff76b9242fda4d4586c2bb3da829b1ba, State: Initialized, Role: FOLLOWER
I20250109 08:56:30.843374 20692 consensus_queue.cc:260] T 74c881b69c3946429070c729ec66b0e4 P ff76b9242fda4d4586c2bb3da829b1ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:30.845110 20691 raft_consensus.cc:357] T 74c881b69c3946429070c729ec66b0e4 P b17b595d5da94e5dbf96bbef1ef3e668 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:30.845898 20691 raft_consensus.cc:383] T 74c881b69c3946429070c729ec66b0e4 P b17b595d5da94e5dbf96bbef1ef3e668 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:56:30.846174 20691 raft_consensus.cc:738] T 74c881b69c3946429070c729ec66b0e4 P b17b595d5da94e5dbf96bbef1ef3e668 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b17b595d5da94e5dbf96bbef1ef3e668, State: Initialized, Role: FOLLOWER
I20250109 08:56:30.846865 20691 consensus_queue.cc:260] T 74c881b69c3946429070c729ec66b0e4 P b17b595d5da94e5dbf96bbef1ef3e668 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:30.848516 20647 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bdeeebda65be4889bf59888b7a7abfec" candidate_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58"
I20250109 08:56:30.848950 20691 ts_tablet_manager.cc:1428] T 74c881b69c3946429070c729ec66b0e4 P b17b595d5da94e5dbf96bbef1ef3e668: Time spent starting tablet: real 0.006s user 0.005s sys 0.003s
I20250109 08:56:30.845590 20692 ts_tablet_manager.cc:1428] T 74c881b69c3946429070c729ec66b0e4 P ff76b9242fda4d4586c2bb3da829b1ba: Time spent starting tablet: real 0.028s user 0.003s sys 0.002s
I20250109 08:56:30.849114 20647 raft_consensus.cc:3054] T bdeeebda65be4889bf59888b7a7abfec P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:56:30.848543 20571 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bdeeebda65be4889bf59888b7a7abfec" candidate_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b17b595d5da94e5dbf96bbef1ef3e668"
I20250109 08:56:30.853133 20571 raft_consensus.cc:3054] T bdeeebda65be4889bf59888b7a7abfec P b17b595d5da94e5dbf96bbef1ef3e668 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:56:30.854277 20717 consensus_queue.cc:1035] T bc8cfc045acc40efaa744f955d08ea97 P ff76b9242fda4d4586c2bb3da829b1ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:56:30.859838 20571 raft_consensus.cc:2463] T bdeeebda65be4889bf59888b7a7abfec P b17b595d5da94e5dbf96bbef1ef3e668 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff76b9242fda4d4586c2bb3da829b1ba in term 1.
I20250109 08:56:30.862346 20392 catalog_manager.cc:5526] T 5378208cf9ad43129b6276fbb65af974 P c5bffb8e4c5e4e7a8b4e919c98c26e58 reported cstate change: term changed from 0 to 1, leader changed from <none> to c5bffb8e4c5e4e7a8b4e919c98c26e58 (127.19.226.195). New cstate: current_term: 1 leader_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } health_report { overall_health: HEALTHY } } }
I20250109 08:56:30.864881 20462 leader_election.cc:304] T bdeeebda65be4889bf59888b7a7abfec P ff76b9242fda4d4586c2bb3da829b1ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b17b595d5da94e5dbf96bbef1ef3e668, ff76b9242fda4d4586c2bb3da829b1ba; no voters:
I20250109 08:56:30.865717 20724 raft_consensus.cc:491] T 6b1d1214e23d47ff8ec02197a45d8484 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:56:30.866252 20717 raft_consensus.cc:2798] T bdeeebda65be4889bf59888b7a7abfec P ff76b9242fda4d4586c2bb3da829b1ba [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:56:30.866168 20724 raft_consensus.cc:513] T 6b1d1214e23d47ff8ec02197a45d8484 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:30.866811 20717 raft_consensus.cc:695] T bdeeebda65be4889bf59888b7a7abfec P ff76b9242fda4d4586c2bb3da829b1ba [term 1 LEADER]: Becoming Leader. State: Replica: ff76b9242fda4d4586c2bb3da829b1ba, State: Running, Role: LEADER
I20250109 08:56:30.863456 20647 raft_consensus.cc:2463] T bdeeebda65be4889bf59888b7a7abfec P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff76b9242fda4d4586c2bb3da829b1ba in term 1.
I20250109 08:56:30.869621 20497 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b1d1214e23d47ff8ec02197a45d8484" candidate_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" is_pre_election: true
I20250109 08:56:30.870168 20724 leader_election.cc:290] T 6b1d1214e23d47ff8ec02197a45d8484 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ff76b9242fda4d4586c2bb3da829b1ba (127.19.226.193:42763), b17b595d5da94e5dbf96bbef1ef3e668 (127.19.226.194:42855)
I20250109 08:56:30.872954 20716 leader_election.cc:290] T bdeeebda65be4889bf59888b7a7abfec P ff76b9242fda4d4586c2bb3da829b1ba [CANDIDATE]: Term 1 election: Requested vote from peers b17b595d5da94e5dbf96bbef1ef3e668 (127.19.226.194:42855), c5bffb8e4c5e4e7a8b4e919c98c26e58 (127.19.226.195:45649)
I20250109 08:56:30.867712 20717 consensus_queue.cc:237] T bdeeebda65be4889bf59888b7a7abfec P ff76b9242fda4d4586c2bb3da829b1ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:30.876596 20571 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b1d1214e23d47ff8ec02197a45d8484" candidate_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" is_pre_election: true
I20250109 08:56:30.877406 20571 raft_consensus.cc:2388] T 6b1d1214e23d47ff8ec02197a45d8484 P b17b595d5da94e5dbf96bbef1ef3e668 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c5bffb8e4c5e4e7a8b4e919c98c26e58 in current term 1: Already voted for candidate ff76b9242fda4d4586c2bb3da829b1ba in this term.
I20250109 08:56:30.878320 20611 leader_election.cc:304] T 6b1d1214e23d47ff8ec02197a45d8484 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c5bffb8e4c5e4e7a8b4e919c98c26e58; no voters: b17b595d5da94e5dbf96bbef1ef3e668, ff76b9242fda4d4586c2bb3da829b1ba
I20250109 08:56:30.879371 20724 raft_consensus.cc:3054] T 6b1d1214e23d47ff8ec02197a45d8484 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:56:30.886041 20724 raft_consensus.cc:2743] T 6b1d1214e23d47ff8ec02197a45d8484 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:56:30.816596 20498 raft_consensus.cc:2463] T 5378208cf9ad43129b6276fbb65af974 P ff76b9242fda4d4586c2bb3da829b1ba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c5bffb8e4c5e4e7a8b4e919c98c26e58 in term 1.
I20250109 08:56:30.891219 20716 raft_consensus.cc:491] T cfbc2774b91c40d59dc80655451f5bdd P ff76b9242fda4d4586c2bb3da829b1ba [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:56:30.891714 20716 raft_consensus.cc:513] T cfbc2774b91c40d59dc80655451f5bdd P ff76b9242fda4d4586c2bb3da829b1ba [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:30.895680 20647 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfbc2774b91c40d59dc80655451f5bdd" candidate_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" is_pre_election: true
I20250109 08:56:30.896308 20647 raft_consensus.cc:2463] T cfbc2774b91c40d59dc80655451f5bdd P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff76b9242fda4d4586c2bb3da829b1ba in term 0.
I20250109 08:56:30.896414 20572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfbc2774b91c40d59dc80655451f5bdd" candidate_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" is_pre_election: true
I20250109 08:56:30.897114 20572 raft_consensus.cc:2463] T cfbc2774b91c40d59dc80655451f5bdd P b17b595d5da94e5dbf96bbef1ef3e668 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff76b9242fda4d4586c2bb3da829b1ba in term 0.
I20250109 08:56:30.897418 20461 leader_election.cc:304] T cfbc2774b91c40d59dc80655451f5bdd P ff76b9242fda4d4586c2bb3da829b1ba [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c5bffb8e4c5e4e7a8b4e919c98c26e58, ff76b9242fda4d4586c2bb3da829b1ba; no voters:
I20250109 08:56:30.898176 20717 raft_consensus.cc:2798] T cfbc2774b91c40d59dc80655451f5bdd P ff76b9242fda4d4586c2bb3da829b1ba [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:56:30.898530 20717 raft_consensus.cc:491] T cfbc2774b91c40d59dc80655451f5bdd P ff76b9242fda4d4586c2bb3da829b1ba [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:56:30.898739 20716 leader_election.cc:290] T cfbc2774b91c40d59dc80655451f5bdd P ff76b9242fda4d4586c2bb3da829b1ba [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b17b595d5da94e5dbf96bbef1ef3e668 (127.19.226.194:42855), c5bffb8e4c5e4e7a8b4e919c98c26e58 (127.19.226.195:45649)
I20250109 08:56:30.898874 20717 raft_consensus.cc:3054] T cfbc2774b91c40d59dc80655451f5bdd P ff76b9242fda4d4586c2bb3da829b1ba [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:56:30.901264 20393 catalog_manager.cc:5526] T bdeeebda65be4889bf59888b7a7abfec P ff76b9242fda4d4586c2bb3da829b1ba reported cstate change: term changed from 0 to 1, leader changed from <none> to ff76b9242fda4d4586c2bb3da829b1ba (127.19.226.193). New cstate: current_term: 1 leader_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } health_report { overall_health: UNKNOWN } } }
I20250109 08:56:30.903167 20717 raft_consensus.cc:513] T cfbc2774b91c40d59dc80655451f5bdd P ff76b9242fda4d4586c2bb3da829b1ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:30.904981 20717 leader_election.cc:290] T cfbc2774b91c40d59dc80655451f5bdd P ff76b9242fda4d4586c2bb3da829b1ba [CANDIDATE]: Term 1 election: Requested vote from peers b17b595d5da94e5dbf96bbef1ef3e668 (127.19.226.194:42855), c5bffb8e4c5e4e7a8b4e919c98c26e58 (127.19.226.195:45649)
I20250109 08:56:30.905733 20572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfbc2774b91c40d59dc80655451f5bdd" candidate_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b17b595d5da94e5dbf96bbef1ef3e668"
I20250109 08:56:30.906349 20572 raft_consensus.cc:3054] T cfbc2774b91c40d59dc80655451f5bdd P b17b595d5da94e5dbf96bbef1ef3e668 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:56:30.906414 20647 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfbc2774b91c40d59dc80655451f5bdd" candidate_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58"
I20250109 08:56:30.907101 20647 raft_consensus.cc:3054] T cfbc2774b91c40d59dc80655451f5bdd P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:56:30.913329 20647 raft_consensus.cc:2463] T cfbc2774b91c40d59dc80655451f5bdd P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff76b9242fda4d4586c2bb3da829b1ba in term 1.
I20250109 08:56:30.914484 20461 leader_election.cc:304] T cfbc2774b91c40d59dc80655451f5bdd P ff76b9242fda4d4586c2bb3da829b1ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c5bffb8e4c5e4e7a8b4e919c98c26e58, ff76b9242fda4d4586c2bb3da829b1ba; no voters:
I20250109 08:56:30.915359 20717 raft_consensus.cc:2798] T cfbc2774b91c40d59dc80655451f5bdd P ff76b9242fda4d4586c2bb3da829b1ba [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:56:30.916039 20717 raft_consensus.cc:695] T cfbc2774b91c40d59dc80655451f5bdd P ff76b9242fda4d4586c2bb3da829b1ba [term 1 LEADER]: Becoming Leader. State: Replica: ff76b9242fda4d4586c2bb3da829b1ba, State: Running, Role: LEADER
I20250109 08:56:30.916896 20717 consensus_queue.cc:237] T cfbc2774b91c40d59dc80655451f5bdd P ff76b9242fda4d4586c2bb3da829b1ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:30.924302 20572 raft_consensus.cc:2463] T cfbc2774b91c40d59dc80655451f5bdd P b17b595d5da94e5dbf96bbef1ef3e668 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff76b9242fda4d4586c2bb3da829b1ba in term 1.
I20250109 08:56:30.933583 20697 raft_consensus.cc:491] T 74c881b69c3946429070c729ec66b0e4 P ff76b9242fda4d4586c2bb3da829b1ba [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:56:30.934072 20697 raft_consensus.cc:513] T 74c881b69c3946429070c729ec66b0e4 P ff76b9242fda4d4586c2bb3da829b1ba [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:30.936034 20697 leader_election.cc:290] T 74c881b69c3946429070c729ec66b0e4 P ff76b9242fda4d4586c2bb3da829b1ba [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b17b595d5da94e5dbf96bbef1ef3e668 (127.19.226.194:42855), c5bffb8e4c5e4e7a8b4e919c98c26e58 (127.19.226.195:45649)
I20250109 08:56:30.937230 20572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74c881b69c3946429070c729ec66b0e4" candidate_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" is_pre_election: true
I20250109 08:56:30.937572 20647 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74c881b69c3946429070c729ec66b0e4" candidate_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" is_pre_election: true
I20250109 08:56:30.937896 20572 raft_consensus.cc:2463] T 74c881b69c3946429070c729ec66b0e4 P b17b595d5da94e5dbf96bbef1ef3e668 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff76b9242fda4d4586c2bb3da829b1ba in term 0.
I20250109 08:56:30.938170 20647 raft_consensus.cc:2463] T 74c881b69c3946429070c729ec66b0e4 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff76b9242fda4d4586c2bb3da829b1ba in term 0.
I20250109 08:56:30.939249 20461 leader_election.cc:304] T 74c881b69c3946429070c729ec66b0e4 P ff76b9242fda4d4586c2bb3da829b1ba [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c5bffb8e4c5e4e7a8b4e919c98c26e58, ff76b9242fda4d4586c2bb3da829b1ba; no voters:
I20250109 08:56:30.940284 20697 raft_consensus.cc:2798] T 74c881b69c3946429070c729ec66b0e4 P ff76b9242fda4d4586c2bb3da829b1ba [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:56:30.940668 20697 raft_consensus.cc:491] T 74c881b69c3946429070c729ec66b0e4 P ff76b9242fda4d4586c2bb3da829b1ba [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:56:30.940989 20697 raft_consensus.cc:3054] T 74c881b69c3946429070c729ec66b0e4 P ff76b9242fda4d4586c2bb3da829b1ba [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:56:30.947248 20697 raft_consensus.cc:513] T 74c881b69c3946429070c729ec66b0e4 P ff76b9242fda4d4586c2bb3da829b1ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:30.949997 20572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74c881b69c3946429070c729ec66b0e4" candidate_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b17b595d5da94e5dbf96bbef1ef3e668"
I20250109 08:56:30.950726 20572 raft_consensus.cc:3054] T 74c881b69c3946429070c729ec66b0e4 P b17b595d5da94e5dbf96bbef1ef3e668 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:56:30.951912 20697 leader_election.cc:290] T 74c881b69c3946429070c729ec66b0e4 P ff76b9242fda4d4586c2bb3da829b1ba [CANDIDATE]: Term 1 election: Requested vote from peers b17b595d5da94e5dbf96bbef1ef3e668 (127.19.226.194:42855), c5bffb8e4c5e4e7a8b4e919c98c26e58 (127.19.226.195:45649)
I20250109 08:56:30.952200 20393 catalog_manager.cc:5526] T cfbc2774b91c40d59dc80655451f5bdd P ff76b9242fda4d4586c2bb3da829b1ba reported cstate change: term changed from 0 to 1, leader changed from <none> to ff76b9242fda4d4586c2bb3da829b1ba (127.19.226.193). New cstate: current_term: 1 leader_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } health_report { overall_health: UNKNOWN } } }
I20250109 08:56:30.952992 20646 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74c881b69c3946429070c729ec66b0e4" candidate_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58"
I20250109 08:56:30.953580 20646 raft_consensus.cc:3054] T 74c881b69c3946429070c729ec66b0e4 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:56:30.954437 20718 consensus_queue.cc:1035] T bc8cfc045acc40efaa744f955d08ea97 P ff76b9242fda4d4586c2bb3da829b1ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250109 08:56:30.961453 20646 raft_consensus.cc:2463] T 74c881b69c3946429070c729ec66b0e4 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff76b9242fda4d4586c2bb3da829b1ba in term 1.
I20250109 08:56:30.962590 20461 leader_election.cc:304] T 74c881b69c3946429070c729ec66b0e4 P ff76b9242fda4d4586c2bb3da829b1ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c5bffb8e4c5e4e7a8b4e919c98c26e58, ff76b9242fda4d4586c2bb3da829b1ba; no voters:
I20250109 08:56:30.963402 20718 raft_consensus.cc:2798] T 74c881b69c3946429070c729ec66b0e4 P ff76b9242fda4d4586c2bb3da829b1ba [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:56:30.963845 20718 raft_consensus.cc:695] T 74c881b69c3946429070c729ec66b0e4 P ff76b9242fda4d4586c2bb3da829b1ba [term 1 LEADER]: Becoming Leader. State: Replica: ff76b9242fda4d4586c2bb3da829b1ba, State: Running, Role: LEADER
I20250109 08:56:30.964534 20572 raft_consensus.cc:2463] T 74c881b69c3946429070c729ec66b0e4 P b17b595d5da94e5dbf96bbef1ef3e668 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff76b9242fda4d4586c2bb3da829b1ba in term 1.
I20250109 08:56:30.964648 20718 consensus_queue.cc:237] T 74c881b69c3946429070c729ec66b0e4 P ff76b9242fda4d4586c2bb3da829b1ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:30.984495 20393 catalog_manager.cc:5526] T 74c881b69c3946429070c729ec66b0e4 P ff76b9242fda4d4586c2bb3da829b1ba reported cstate change: term changed from 0 to 1, leader changed from <none> to ff76b9242fda4d4586c2bb3da829b1ba (127.19.226.193). New cstate: current_term: 1 leader_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } health_report { overall_health: UNKNOWN } } }
I20250109 08:56:31.020144 20718 consensus_queue.cc:1035] T 6b1d1214e23d47ff8ec02197a45d8484 P ff76b9242fda4d4586c2bb3da829b1ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250109 08:56:31.030659 20717 consensus_queue.cc:1035] T 6b1d1214e23d47ff8ec02197a45d8484 P ff76b9242fda4d4586c2bb3da829b1ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:56:31.070494 20724 consensus_queue.cc:1035] T db23d411a3a14ed69a1db0b2c8912807 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:56:31.074905 20724 consensus_queue.cc:1035] T 4173445b18a24674ad20fe4b7e17c697 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:56:31.098619 20724 consensus_queue.cc:1035] T 4173445b18a24674ad20fe4b7e17c697 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20250109 08:56:31.100428 20442 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20250109 08:56:31.100909 20724 raft_consensus.cc:491] T 6f383be53b01468f85e731bd2d4367c0 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:56:31.101346 20724 raft_consensus.cc:513] T 6f383be53b01468f85e731bd2d4367c0 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:31.103428 20724 leader_election.cc:290] T 6f383be53b01468f85e731bd2d4367c0 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ff76b9242fda4d4586c2bb3da829b1ba (127.19.226.193:42763), b17b595d5da94e5dbf96bbef1ef3e668 (127.19.226.194:42855)
I20250109 08:56:31.105764 20573 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f383be53b01468f85e731bd2d4367c0" candidate_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" is_pre_election: true
I20250109 08:56:31.106389 20573 raft_consensus.cc:2463] T 6f383be53b01468f85e731bd2d4367c0 P b17b595d5da94e5dbf96bbef1ef3e668 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c5bffb8e4c5e4e7a8b4e919c98c26e58 in term 0.
I20250109 08:56:31.107138 20497 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f383be53b01468f85e731bd2d4367c0" candidate_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" is_pre_election: true
I20250109 08:56:31.107721 20497 raft_consensus.cc:2463] T 6f383be53b01468f85e731bd2d4367c0 P ff76b9242fda4d4586c2bb3da829b1ba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c5bffb8e4c5e4e7a8b4e919c98c26e58 in term 0.
I20250109 08:56:31.108914 20613 leader_election.cc:304] T 6f383be53b01468f85e731bd2d4367c0 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c5bffb8e4c5e4e7a8b4e919c98c26e58, ff76b9242fda4d4586c2bb3da829b1ba; no voters:
I20250109 08:56:31.109969 20756 consensus_queue.cc:1035] T db23d411a3a14ed69a1db0b2c8912807 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250109 08:56:31.111084 20757 raft_consensus.cc:2798] T 6f383be53b01468f85e731bd2d4367c0 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:56:31.111475 20757 raft_consensus.cc:491] T 6f383be53b01468f85e731bd2d4367c0 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:56:31.111807 20757 raft_consensus.cc:3054] T 6f383be53b01468f85e731bd2d4367c0 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:56:31.129477 20757 raft_consensus.cc:513] T 6f383be53b01468f85e731bd2d4367c0 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:31.131433 20757 leader_election.cc:290] T 6f383be53b01468f85e731bd2d4367c0 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [CANDIDATE]: Term 1 election: Requested vote from peers ff76b9242fda4d4586c2bb3da829b1ba (127.19.226.193:42763), b17b595d5da94e5dbf96bbef1ef3e668 (127.19.226.194:42855)
I20250109 08:56:31.132171 20498 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f383be53b01468f85e731bd2d4367c0" candidate_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff76b9242fda4d4586c2bb3da829b1ba"
I20250109 08:56:31.132954 20498 raft_consensus.cc:3054] T 6f383be53b01468f85e731bd2d4367c0 P ff76b9242fda4d4586c2bb3da829b1ba [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:56:31.132851 20572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f383be53b01468f85e731bd2d4367c0" candidate_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b17b595d5da94e5dbf96bbef1ef3e668"
I20250109 08:56:31.133562 20572 raft_consensus.cc:3054] T 6f383be53b01468f85e731bd2d4367c0 P b17b595d5da94e5dbf96bbef1ef3e668 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:56:31.139889 20572 raft_consensus.cc:2463] T 6f383be53b01468f85e731bd2d4367c0 P b17b595d5da94e5dbf96bbef1ef3e668 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c5bffb8e4c5e4e7a8b4e919c98c26e58 in term 1.
I20250109 08:56:31.140983 20611 leader_election.cc:304] T 6f383be53b01468f85e731bd2d4367c0 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b17b595d5da94e5dbf96bbef1ef3e668, c5bffb8e4c5e4e7a8b4e919c98c26e58; no voters:
I20250109 08:56:31.141947 20757 raft_consensus.cc:2798] T 6f383be53b01468f85e731bd2d4367c0 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:56:31.142414 20757 raft_consensus.cc:695] T 6f383be53b01468f85e731bd2d4367c0 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 1 LEADER]: Becoming Leader. State: Replica: c5bffb8e4c5e4e7a8b4e919c98c26e58, State: Running, Role: LEADER
I20250109 08:56:31.143342 20498 raft_consensus.cc:2463] T 6f383be53b01468f85e731bd2d4367c0 P ff76b9242fda4d4586c2bb3da829b1ba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c5bffb8e4c5e4e7a8b4e919c98c26e58 in term 1.
I20250109 08:56:31.143193 20757 consensus_queue.cc:237] T 6f383be53b01468f85e731bd2d4367c0 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:31.154877 20391 catalog_manager.cc:5526] T 6f383be53b01468f85e731bd2d4367c0 P c5bffb8e4c5e4e7a8b4e919c98c26e58 reported cstate change: term changed from 0 to 1, leader changed from <none> to c5bffb8e4c5e4e7a8b4e919c98c26e58 (127.19.226.195). New cstate: current_term: 1 leader_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } health_report { overall_health: HEALTHY } } }
I20250109 08:56:31.240469 20757 consensus_queue.cc:1035] T 5378208cf9ad43129b6276fbb65af974 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:56:31.252137 20696 consensus_queue.cc:1035] T 5378208cf9ad43129b6276fbb65af974 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:56:31.264053 20717 consensus_queue.cc:1035] T bdeeebda65be4889bf59888b7a7abfec P ff76b9242fda4d4586c2bb3da829b1ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:56:31.278496 20718 consensus_queue.cc:1035] T bdeeebda65be4889bf59888b7a7abfec P ff76b9242fda4d4586c2bb3da829b1ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:56:31.299790 20647 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "4173445b18a24674ad20fe4b7e17c697"
dest_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58"
mode: GRACEFUL
new_leader_uuid: "ff76b9242fda4d4586c2bb3da829b1ba"
from {username='slave'} at 127.0.0.1:49942
I20250109 08:56:31.300530 20647 raft_consensus.cc:604] T 4173445b18a24674ad20fe4b7e17c697 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 1 LEADER]: Received request to transfer leadership to TS ff76b9242fda4d4586c2bb3da829b1ba
I20250109 08:56:31.307708 20647 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "6f383be53b01468f85e731bd2d4367c0"
dest_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58"
mode: GRACEFUL
new_leader_uuid: "ff76b9242fda4d4586c2bb3da829b1ba"
from {username='slave'} at 127.0.0.1:49942
I20250109 08:56:31.308349 20647 raft_consensus.cc:604] T 6f383be53b01468f85e731bd2d4367c0 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 1 LEADER]: Received request to transfer leadership to TS ff76b9242fda4d4586c2bb3da829b1ba
I20250109 08:56:31.320904 20497 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "bdeeebda65be4889bf59888b7a7abfec"
dest_uuid: "ff76b9242fda4d4586c2bb3da829b1ba"
mode: GRACEFUL
new_leader_uuid: "b17b595d5da94e5dbf96bbef1ef3e668"
from {username='slave'} at 127.0.0.1:46200
I20250109 08:56:31.321413 20497 raft_consensus.cc:604] T bdeeebda65be4889bf59888b7a7abfec P ff76b9242fda4d4586c2bb3da829b1ba [term 1 LEADER]: Received request to transfer leadership to TS b17b595d5da94e5dbf96bbef1ef3e668
I20250109 08:56:31.326382 20497 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "cfbc2774b91c40d59dc80655451f5bdd"
dest_uuid: "ff76b9242fda4d4586c2bb3da829b1ba"
mode: GRACEFUL
new_leader_uuid: "b17b595d5da94e5dbf96bbef1ef3e668"
from {username='slave'} at 127.0.0.1:46200
I20250109 08:56:31.326845 20497 raft_consensus.cc:604] T cfbc2774b91c40d59dc80655451f5bdd P ff76b9242fda4d4586c2bb3da829b1ba [term 1 LEADER]: Received request to transfer leadership to TS b17b595d5da94e5dbf96bbef1ef3e668
I20250109 08:56:31.331422 20497 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "74c881b69c3946429070c729ec66b0e4"
dest_uuid: "ff76b9242fda4d4586c2bb3da829b1ba"
mode: GRACEFUL
new_leader_uuid: "b17b595d5da94e5dbf96bbef1ef3e668"
from {username='slave'} at 127.0.0.1:46200
I20250109 08:56:31.331861 20497 raft_consensus.cc:604] T 74c881b69c3946429070c729ec66b0e4 P ff76b9242fda4d4586c2bb3da829b1ba [term 1 LEADER]: Received request to transfer leadership to TS b17b595d5da94e5dbf96bbef1ef3e668
I20250109 08:56:31.336506 20647 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "5378208cf9ad43129b6276fbb65af974"
dest_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58"
mode: GRACEFUL
new_leader_uuid: "b17b595d5da94e5dbf96bbef1ef3e668"
from {username='slave'} at 127.0.0.1:49942
I20250109 08:56:31.336956 20647 raft_consensus.cc:604] T 5378208cf9ad43129b6276fbb65af974 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 1 LEADER]: Received request to transfer leadership to TS b17b595d5da94e5dbf96bbef1ef3e668
I20250109 08:56:31.433413 20718 consensus_queue.cc:1035] T 74c881b69c3946429070c729ec66b0e4 P ff76b9242fda4d4586c2bb3da829b1ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:56:31.440044 20697 raft_consensus.cc:988] T 74c881b69c3946429070c729ec66b0e4 P ff76b9242fda4d4586c2bb3da829b1ba: : Instructing follower b17b595d5da94e5dbf96bbef1ef3e668 to start an election
I20250109 08:56:31.440526 20716 raft_consensus.cc:1076] T 74c881b69c3946429070c729ec66b0e4 P ff76b9242fda4d4586c2bb3da829b1ba [term 1 LEADER]: Signalling peer b17b595d5da94e5dbf96bbef1ef3e668 to start an election
I20250109 08:56:31.441785 20717 consensus_queue.cc:1035] T cfbc2774b91c40d59dc80655451f5bdd P ff76b9242fda4d4586c2bb3da829b1ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:56:31.447037 20770 consensus_queue.cc:1035] T 74c881b69c3946429070c729ec66b0e4 P ff76b9242fda4d4586c2bb3da829b1ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:56:31.449683 20718 consensus_queue.cc:1035] T cfbc2774b91c40d59dc80655451f5bdd P ff76b9242fda4d4586c2bb3da829b1ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:56:31.451009 20572 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "74c881b69c3946429070c729ec66b0e4"
dest_uuid: "b17b595d5da94e5dbf96bbef1ef3e668"
from {username='slave'} at 127.0.0.1:49890
I20250109 08:56:31.455925 20572 raft_consensus.cc:491] T 74c881b69c3946429070c729ec66b0e4 P b17b595d5da94e5dbf96bbef1ef3e668 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:56:31.456359 20572 raft_consensus.cc:3054] T 74c881b69c3946429070c729ec66b0e4 P b17b595d5da94e5dbf96bbef1ef3e668 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:56:31.468286 20717 raft_consensus.cc:988] T cfbc2774b91c40d59dc80655451f5bdd P ff76b9242fda4d4586c2bb3da829b1ba: : Instructing follower b17b595d5da94e5dbf96bbef1ef3e668 to start an election
I20250109 08:56:31.468753 20717 raft_consensus.cc:1076] T cfbc2774b91c40d59dc80655451f5bdd P ff76b9242fda4d4586c2bb3da829b1ba [term 1 LEADER]: Signalling peer b17b595d5da94e5dbf96bbef1ef3e668 to start an election
I20250109 08:56:31.471736 20573 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "cfbc2774b91c40d59dc80655451f5bdd"
dest_uuid: "b17b595d5da94e5dbf96bbef1ef3e668"
from {username='slave'} at 127.0.0.1:49890
I20250109 08:56:31.472388 20573 raft_consensus.cc:491] T cfbc2774b91c40d59dc80655451f5bdd P b17b595d5da94e5dbf96bbef1ef3e668 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:56:31.472790 20573 raft_consensus.cc:3054] T cfbc2774b91c40d59dc80655451f5bdd P b17b595d5da94e5dbf96bbef1ef3e668 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:56:31.482079 20573 raft_consensus.cc:513] T cfbc2774b91c40d59dc80655451f5bdd P b17b595d5da94e5dbf96bbef1ef3e668 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:31.484994 20497 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfbc2774b91c40d59dc80655451f5bdd" candidate_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ff76b9242fda4d4586c2bb3da829b1ba"
I20250109 08:56:31.485679 20497 raft_consensus.cc:3049] T cfbc2774b91c40d59dc80655451f5bdd P ff76b9242fda4d4586c2bb3da829b1ba [term 1 LEADER]: Stepping down as leader of term 1
I20250109 08:56:31.486052 20497 raft_consensus.cc:738] T cfbc2774b91c40d59dc80655451f5bdd P ff76b9242fda4d4586c2bb3da829b1ba [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ff76b9242fda4d4586c2bb3da829b1ba, State: Running, Role: LEADER
I20250109 08:56:31.485951 20572 raft_consensus.cc:513] T 74c881b69c3946429070c729ec66b0e4 P b17b595d5da94e5dbf96bbef1ef3e668 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:31.486943 20497 consensus_queue.cc:260] T cfbc2774b91c40d59dc80655451f5bdd P ff76b9242fda4d4586c2bb3da829b1ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:31.489190 20572 leader_election.cc:290] T 74c881b69c3946429070c729ec66b0e4 P b17b595d5da94e5dbf96bbef1ef3e668 [CANDIDATE]: Term 2 election: Requested vote from peers ff76b9242fda4d4586c2bb3da829b1ba (127.19.226.193:42763), c5bffb8e4c5e4e7a8b4e919c98c26e58 (127.19.226.195:45649)
I20250109 08:56:31.489454 20645 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74c881b69c3946429070c729ec66b0e4" candidate_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58"
I20250109 08:56:31.485635 20573 leader_election.cc:290] T cfbc2774b91c40d59dc80655451f5bdd P b17b595d5da94e5dbf96bbef1ef3e668 [CANDIDATE]: Term 2 election: Requested vote from peers ff76b9242fda4d4586c2bb3da829b1ba (127.19.226.193:42763), c5bffb8e4c5e4e7a8b4e919c98c26e58 (127.19.226.195:45649)
I20250109 08:56:31.489456 20498 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74c881b69c3946429070c729ec66b0e4" candidate_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ff76b9242fda4d4586c2bb3da829b1ba"
I20250109 08:56:31.491940 20498 raft_consensus.cc:3049] T 74c881b69c3946429070c729ec66b0e4 P ff76b9242fda4d4586c2bb3da829b1ba [term 1 LEADER]: Stepping down as leader of term 1
I20250109 08:56:31.492260 20498 raft_consensus.cc:738] T 74c881b69c3946429070c729ec66b0e4 P ff76b9242fda4d4586c2bb3da829b1ba [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ff76b9242fda4d4586c2bb3da829b1ba, State: Running, Role: LEADER
I20250109 08:56:31.492959 20498 consensus_queue.cc:260] T 74c881b69c3946429070c729ec66b0e4 P ff76b9242fda4d4586c2bb3da829b1ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:31.493486 20646 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfbc2774b91c40d59dc80655451f5bdd" candidate_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58"
I20250109 08:56:31.494041 20646 raft_consensus.cc:3054] T cfbc2774b91c40d59dc80655451f5bdd P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:56:31.494144 20498 raft_consensus.cc:3054] T 74c881b69c3946429070c729ec66b0e4 P ff76b9242fda4d4586c2bb3da829b1ba [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:56:31.488320 20497 raft_consensus.cc:3054] T cfbc2774b91c40d59dc80655451f5bdd P ff76b9242fda4d4586c2bb3da829b1ba [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:56:31.505990 20646 raft_consensus.cc:2463] T cfbc2774b91c40d59dc80655451f5bdd P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b17b595d5da94e5dbf96bbef1ef3e668 in term 2.
I20250109 08:56:31.507249 20536 leader_election.cc:304] T cfbc2774b91c40d59dc80655451f5bdd P b17b595d5da94e5dbf96bbef1ef3e668 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b17b595d5da94e5dbf96bbef1ef3e668, c5bffb8e4c5e4e7a8b4e919c98c26e58; no voters:
I20250109 08:56:31.508011 20698 raft_consensus.cc:2798] T cfbc2774b91c40d59dc80655451f5bdd P b17b595d5da94e5dbf96bbef1ef3e668 [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:56:31.511642 20498 raft_consensus.cc:2463] T 74c881b69c3946429070c729ec66b0e4 P ff76b9242fda4d4586c2bb3da829b1ba [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b17b595d5da94e5dbf96bbef1ef3e668 in term 2.
I20250109 08:56:31.512984 20539 leader_election.cc:304] T 74c881b69c3946429070c729ec66b0e4 P b17b595d5da94e5dbf96bbef1ef3e668 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b17b595d5da94e5dbf96bbef1ef3e668, ff76b9242fda4d4586c2bb3da829b1ba; no voters: c5bffb8e4c5e4e7a8b4e919c98c26e58
I20250109 08:56:31.513136 20497 raft_consensus.cc:2463] T cfbc2774b91c40d59dc80655451f5bdd P ff76b9242fda4d4586c2bb3da829b1ba [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b17b595d5da94e5dbf96bbef1ef3e668 in term 2.
I20250109 08:56:31.517055 20698 raft_consensus.cc:695] T cfbc2774b91c40d59dc80655451f5bdd P b17b595d5da94e5dbf96bbef1ef3e668 [term 2 LEADER]: Becoming Leader. State: Replica: b17b595d5da94e5dbf96bbef1ef3e668, State: Running, Role: LEADER
I20250109 08:56:31.518126 20774 raft_consensus.cc:2798] T 74c881b69c3946429070c729ec66b0e4 P b17b595d5da94e5dbf96bbef1ef3e668 [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:56:31.520146 20774 raft_consensus.cc:695] T 74c881b69c3946429070c729ec66b0e4 P b17b595d5da94e5dbf96bbef1ef3e668 [term 2 LEADER]: Becoming Leader. State: Replica: b17b595d5da94e5dbf96bbef1ef3e668, State: Running, Role: LEADER
I20250109 08:56:31.522563 20774 consensus_queue.cc:237] T 74c881b69c3946429070c729ec66b0e4 P b17b595d5da94e5dbf96bbef1ef3e668 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:31.517927 20698 consensus_queue.cc:237] T cfbc2774b91c40d59dc80655451f5bdd P b17b595d5da94e5dbf96bbef1ef3e668 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:31.538204 20391 catalog_manager.cc:5526] T cfbc2774b91c40d59dc80655451f5bdd P b17b595d5da94e5dbf96bbef1ef3e668 reported cstate change: term changed from 1 to 2, leader changed from ff76b9242fda4d4586c2bb3da829b1ba (127.19.226.193) to b17b595d5da94e5dbf96bbef1ef3e668 (127.19.226.194). New cstate: current_term: 2 leader_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } health_report { overall_health: UNKNOWN } } }
I20250109 08:56:31.549209 20391 catalog_manager.cc:5526] T 74c881b69c3946429070c729ec66b0e4 P b17b595d5da94e5dbf96bbef1ef3e668 reported cstate change: term changed from 1 to 2, leader changed from ff76b9242fda4d4586c2bb3da829b1ba (127.19.226.193) to b17b595d5da94e5dbf96bbef1ef3e668 (127.19.226.194). New cstate: current_term: 2 leader_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } health_report { overall_health: UNKNOWN } } }
I20250109 08:56:31.653952 20696 consensus_queue.cc:1035] T 6f383be53b01468f85e731bd2d4367c0 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:56:31.661584 20756 raft_consensus.cc:988] T 6f383be53b01468f85e731bd2d4367c0 P c5bffb8e4c5e4e7a8b4e919c98c26e58: : Instructing follower ff76b9242fda4d4586c2bb3da829b1ba to start an election
I20250109 08:56:31.662213 20758 raft_consensus.cc:1076] T 6f383be53b01468f85e731bd2d4367c0 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 1 LEADER]: Signalling peer ff76b9242fda4d4586c2bb3da829b1ba to start an election
I20250109 08:56:31.664013 20498 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6f383be53b01468f85e731bd2d4367c0"
dest_uuid: "ff76b9242fda4d4586c2bb3da829b1ba"
from {username='slave'} at 127.0.0.1:46182
I20250109 08:56:31.664626 20498 raft_consensus.cc:491] T 6f383be53b01468f85e731bd2d4367c0 P ff76b9242fda4d4586c2bb3da829b1ba [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:56:31.665000 20498 raft_consensus.cc:3054] T 6f383be53b01468f85e731bd2d4367c0 P ff76b9242fda4d4586c2bb3da829b1ba [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:56:31.669994 20756 consensus_queue.cc:1035] T 6f383be53b01468f85e731bd2d4367c0 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:56:31.671170 20498 raft_consensus.cc:513] T 6f383be53b01468f85e731bd2d4367c0 P ff76b9242fda4d4586c2bb3da829b1ba [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:31.672186 20497 raft_consensus.cc:1235] T 6f383be53b01468f85e731bd2d4367c0 P ff76b9242fda4d4586c2bb3da829b1ba [term 2 FOLLOWER]: Rejecting Update request from peer c5bffb8e4c5e4e7a8b4e919c98c26e58 for earlier term 1. Current term is 2. Ops: []
I20250109 08:56:31.675335 20572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f383be53b01468f85e731bd2d4367c0" candidate_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b17b595d5da94e5dbf96bbef1ef3e668"
I20250109 08:56:31.675575 20756 consensus_queue.cc:1046] T 6f383be53b01468f85e731bd2d4367c0 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:56:31.676476 20756 raft_consensus.cc:3049] T 6f383be53b01468f85e731bd2d4367c0 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 1 LEADER]: Stepping down as leader of term 1
I20250109 08:56:31.676784 20756 raft_consensus.cc:738] T 6f383be53b01468f85e731bd2d4367c0 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c5bffb8e4c5e4e7a8b4e919c98c26e58, State: Running, Role: LEADER
I20250109 08:56:31.677618 20756 consensus_queue.cc:260] T 6f383be53b01468f85e731bd2d4367c0 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:31.678555 20756 raft_consensus.cc:3054] T 6f383be53b01468f85e731bd2d4367c0 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:56:31.679975 20646 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f383be53b01468f85e731bd2d4367c0" candidate_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58"
I20250109 08:56:31.681428 20696 raft_consensus.cc:988] T 4173445b18a24674ad20fe4b7e17c697 P c5bffb8e4c5e4e7a8b4e919c98c26e58: : Instructing follower ff76b9242fda4d4586c2bb3da829b1ba to start an election
I20250109 08:56:31.681870 20758 raft_consensus.cc:1076] T 4173445b18a24674ad20fe4b7e17c697 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 1 LEADER]: Signalling peer ff76b9242fda4d4586c2bb3da829b1ba to start an election
I20250109 08:56:31.683526 20496 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "4173445b18a24674ad20fe4b7e17c697"
dest_uuid: "ff76b9242fda4d4586c2bb3da829b1ba"
from {username='slave'} at 127.0.0.1:46182
I20250109 08:56:31.684047 20496 raft_consensus.cc:491] T 4173445b18a24674ad20fe4b7e17c697 P ff76b9242fda4d4586c2bb3da829b1ba [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:56:31.684396 20496 raft_consensus.cc:3054] T 4173445b18a24674ad20fe4b7e17c697 P ff76b9242fda4d4586c2bb3da829b1ba [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:56:31.702061 20498 leader_election.cc:290] T 6f383be53b01468f85e731bd2d4367c0 P ff76b9242fda4d4586c2bb3da829b1ba [CANDIDATE]: Term 2 election: Requested vote from peers b17b595d5da94e5dbf96bbef1ef3e668 (127.19.226.194:42855), c5bffb8e4c5e4e7a8b4e919c98c26e58 (127.19.226.195:45649)
I20250109 08:56:31.702697 20646 raft_consensus.cc:2463] T 6f383be53b01468f85e731bd2d4367c0 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff76b9242fda4d4586c2bb3da829b1ba in term 2.
I20250109 08:56:31.691013 20496 raft_consensus.cc:513] T 4173445b18a24674ad20fe4b7e17c697 P ff76b9242fda4d4586c2bb3da829b1ba [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:31.704731 20496 leader_election.cc:290] T 4173445b18a24674ad20fe4b7e17c697 P ff76b9242fda4d4586c2bb3da829b1ba [CANDIDATE]: Term 2 election: Requested vote from peers b17b595d5da94e5dbf96bbef1ef3e668 (127.19.226.194:42855), c5bffb8e4c5e4e7a8b4e919c98c26e58 (127.19.226.195:45649)
I20250109 08:56:31.706002 20461 leader_election.cc:304] T 6f383be53b01468f85e731bd2d4367c0 P ff76b9242fda4d4586c2bb3da829b1ba [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c5bffb8e4c5e4e7a8b4e919c98c26e58, ff76b9242fda4d4586c2bb3da829b1ba; no voters: b17b595d5da94e5dbf96bbef1ef3e668
I20250109 08:56:31.706681 20573 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4173445b18a24674ad20fe4b7e17c697" candidate_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b17b595d5da94e5dbf96bbef1ef3e668"
I20250109 08:56:31.707278 20573 raft_consensus.cc:3054] T 4173445b18a24674ad20fe4b7e17c697 P b17b595d5da94e5dbf96bbef1ef3e668 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:56:31.707288 20697 raft_consensus.cc:2798] T 6f383be53b01468f85e731bd2d4367c0 P ff76b9242fda4d4586c2bb3da829b1ba [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:56:31.707998 20646 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4173445b18a24674ad20fe4b7e17c697" candidate_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58"
I20250109 08:56:31.708165 20697 raft_consensus.cc:695] T 6f383be53b01468f85e731bd2d4367c0 P ff76b9242fda4d4586c2bb3da829b1ba [term 2 LEADER]: Becoming Leader. State: Replica: ff76b9242fda4d4586c2bb3da829b1ba, State: Running, Role: LEADER
I20250109 08:56:31.708786 20646 raft_consensus.cc:3049] T 4173445b18a24674ad20fe4b7e17c697 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 1 LEADER]: Stepping down as leader of term 1
I20250109 08:56:31.709203 20646 raft_consensus.cc:738] T 4173445b18a24674ad20fe4b7e17c697 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c5bffb8e4c5e4e7a8b4e919c98c26e58, State: Running, Role: LEADER
I20250109 08:56:31.709275 20697 consensus_queue.cc:237] T 6f383be53b01468f85e731bd2d4367c0 P ff76b9242fda4d4586c2bb3da829b1ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:31.710404 20646 consensus_queue.cc:260] T 4173445b18a24674ad20fe4b7e17c697 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:31.711863 20646 raft_consensus.cc:3054] T 4173445b18a24674ad20fe4b7e17c697 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:56:31.716135 20573 raft_consensus.cc:2463] T 4173445b18a24674ad20fe4b7e17c697 P b17b595d5da94e5dbf96bbef1ef3e668 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff76b9242fda4d4586c2bb3da829b1ba in term 2.
I20250109 08:56:31.717942 20462 leader_election.cc:304] T 4173445b18a24674ad20fe4b7e17c697 P ff76b9242fda4d4586c2bb3da829b1ba [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b17b595d5da94e5dbf96bbef1ef3e668, ff76b9242fda4d4586c2bb3da829b1ba; no voters:
I20250109 08:56:31.719945 20646 raft_consensus.cc:2463] T 4173445b18a24674ad20fe4b7e17c697 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff76b9242fda4d4586c2bb3da829b1ba in term 2.
I20250109 08:56:31.720721 20697 raft_consensus.cc:2798] T 4173445b18a24674ad20fe4b7e17c697 P ff76b9242fda4d4586c2bb3da829b1ba [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:56:31.721218 20697 raft_consensus.cc:695] T 4173445b18a24674ad20fe4b7e17c697 P ff76b9242fda4d4586c2bb3da829b1ba [term 2 LEADER]: Becoming Leader. State: Replica: ff76b9242fda4d4586c2bb3da829b1ba, State: Running, Role: LEADER
I20250109 08:56:31.721084 20391 catalog_manager.cc:5526] T 6f383be53b01468f85e731bd2d4367c0 P ff76b9242fda4d4586c2bb3da829b1ba reported cstate change: term changed from 1 to 2, leader changed from c5bffb8e4c5e4e7a8b4e919c98c26e58 (127.19.226.195) to ff76b9242fda4d4586c2bb3da829b1ba (127.19.226.193). New cstate: current_term: 2 leader_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } health_report { overall_health: UNKNOWN } } }
I20250109 08:56:31.722088 20697 consensus_queue.cc:237] T 4173445b18a24674ad20fe4b7e17c697 P ff76b9242fda4d4586c2bb3da829b1ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:31.725857 20770 raft_consensus.cc:988] T bdeeebda65be4889bf59888b7a7abfec P ff76b9242fda4d4586c2bb3da829b1ba: : Instructing follower b17b595d5da94e5dbf96bbef1ef3e668 to start an election
I20250109 08:56:31.726361 20697 raft_consensus.cc:1076] T bdeeebda65be4889bf59888b7a7abfec P ff76b9242fda4d4586c2bb3da829b1ba [term 1 LEADER]: Signalling peer b17b595d5da94e5dbf96bbef1ef3e668 to start an election
I20250109 08:56:31.727967 20573 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "bdeeebda65be4889bf59888b7a7abfec"
dest_uuid: "b17b595d5da94e5dbf96bbef1ef3e668"
from {username='slave'} at 127.0.0.1:49890
I20250109 08:56:31.728497 20573 raft_consensus.cc:491] T bdeeebda65be4889bf59888b7a7abfec P b17b595d5da94e5dbf96bbef1ef3e668 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:56:31.728785 20573 raft_consensus.cc:3054] T bdeeebda65be4889bf59888b7a7abfec P b17b595d5da94e5dbf96bbef1ef3e668 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:56:31.735580 20573 raft_consensus.cc:513] T bdeeebda65be4889bf59888b7a7abfec P b17b595d5da94e5dbf96bbef1ef3e668 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:31.736284 20391 catalog_manager.cc:5526] T 4173445b18a24674ad20fe4b7e17c697 P ff76b9242fda4d4586c2bb3da829b1ba reported cstate change: term changed from 1 to 2, leader changed from c5bffb8e4c5e4e7a8b4e919c98c26e58 (127.19.226.195) to ff76b9242fda4d4586c2bb3da829b1ba (127.19.226.193). New cstate: current_term: 2 leader_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } health_report { overall_health: UNKNOWN } } }
I20250109 08:56:31.738497 20573 leader_election.cc:290] T bdeeebda65be4889bf59888b7a7abfec P b17b595d5da94e5dbf96bbef1ef3e668 [CANDIDATE]: Term 2 election: Requested vote from peers ff76b9242fda4d4586c2bb3da829b1ba (127.19.226.193:42763), c5bffb8e4c5e4e7a8b4e919c98c26e58 (127.19.226.195:45649)
I20250109 08:56:31.738531 20496 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bdeeebda65be4889bf59888b7a7abfec" candidate_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ff76b9242fda4d4586c2bb3da829b1ba"
I20250109 08:56:31.739275 20496 raft_consensus.cc:3049] T bdeeebda65be4889bf59888b7a7abfec P ff76b9242fda4d4586c2bb3da829b1ba [term 1 LEADER]: Stepping down as leader of term 1
I20250109 08:56:31.739576 20646 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bdeeebda65be4889bf59888b7a7abfec" candidate_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58"
I20250109 08:56:31.739603 20496 raft_consensus.cc:738] T bdeeebda65be4889bf59888b7a7abfec P ff76b9242fda4d4586c2bb3da829b1ba [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ff76b9242fda4d4586c2bb3da829b1ba, State: Running, Role: LEADER
I20250109 08:56:31.740223 20646 raft_consensus.cc:3054] T bdeeebda65be4889bf59888b7a7abfec P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:56:31.740644 20496 consensus_queue.cc:260] T bdeeebda65be4889bf59888b7a7abfec P ff76b9242fda4d4586c2bb3da829b1ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:31.741784 20496 raft_consensus.cc:3054] T bdeeebda65be4889bf59888b7a7abfec P ff76b9242fda4d4586c2bb3da829b1ba [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:56:31.746742 20646 raft_consensus.cc:2463] T bdeeebda65be4889bf59888b7a7abfec P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b17b595d5da94e5dbf96bbef1ef3e668 in term 2.
I20250109 08:56:31.747777 20536 leader_election.cc:304] T bdeeebda65be4889bf59888b7a7abfec P b17b595d5da94e5dbf96bbef1ef3e668 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b17b595d5da94e5dbf96bbef1ef3e668, c5bffb8e4c5e4e7a8b4e919c98c26e58; no voters:
I20250109 08:56:31.748554 20698 raft_consensus.cc:2798] T bdeeebda65be4889bf59888b7a7abfec P b17b595d5da94e5dbf96bbef1ef3e668 [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:56:31.748957 20698 raft_consensus.cc:695] T bdeeebda65be4889bf59888b7a7abfec P b17b595d5da94e5dbf96bbef1ef3e668 [term 2 LEADER]: Becoming Leader. State: Replica: b17b595d5da94e5dbf96bbef1ef3e668, State: Running, Role: LEADER
I20250109 08:56:31.749558 20698 consensus_queue.cc:237] T bdeeebda65be4889bf59888b7a7abfec P b17b595d5da94e5dbf96bbef1ef3e668 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:31.751163 20496 raft_consensus.cc:2463] T bdeeebda65be4889bf59888b7a7abfec P ff76b9242fda4d4586c2bb3da829b1ba [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b17b595d5da94e5dbf96bbef1ef3e668 in term 2.
I20250109 08:56:31.756596 20391 catalog_manager.cc:5526] T bdeeebda65be4889bf59888b7a7abfec P b17b595d5da94e5dbf96bbef1ef3e668 reported cstate change: term changed from 1 to 2, leader changed from ff76b9242fda4d4586c2bb3da829b1ba (127.19.226.193) to b17b595d5da94e5dbf96bbef1ef3e668 (127.19.226.194). New cstate: current_term: 2 leader_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } health_report { overall_health: UNKNOWN } } }
I20250109 08:56:31.825671 20756 raft_consensus.cc:988] T 5378208cf9ad43129b6276fbb65af974 P c5bffb8e4c5e4e7a8b4e919c98c26e58: : Instructing follower b17b595d5da94e5dbf96bbef1ef3e668 to start an election
I20250109 08:56:31.826117 20758 raft_consensus.cc:1076] T 5378208cf9ad43129b6276fbb65af974 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 1 LEADER]: Signalling peer b17b595d5da94e5dbf96bbef1ef3e668 to start an election
I20250109 08:56:31.827415 20573 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "5378208cf9ad43129b6276fbb65af974"
dest_uuid: "b17b595d5da94e5dbf96bbef1ef3e668"
from {username='slave'} at 127.0.0.1:49904
I20250109 08:56:31.827872 20573 raft_consensus.cc:491] T 5378208cf9ad43129b6276fbb65af974 P b17b595d5da94e5dbf96bbef1ef3e668 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:56:31.828145 20573 raft_consensus.cc:3054] T 5378208cf9ad43129b6276fbb65af974 P b17b595d5da94e5dbf96bbef1ef3e668 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:56:31.833015 20573 raft_consensus.cc:513] T 5378208cf9ad43129b6276fbb65af974 P b17b595d5da94e5dbf96bbef1ef3e668 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:31.835207 20573 leader_election.cc:290] T 5378208cf9ad43129b6276fbb65af974 P b17b595d5da94e5dbf96bbef1ef3e668 [CANDIDATE]: Term 2 election: Requested vote from peers ff76b9242fda4d4586c2bb3da829b1ba (127.19.226.193:42763), c5bffb8e4c5e4e7a8b4e919c98c26e58 (127.19.226.195:45649)
I20250109 08:56:31.836344 20496 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5378208cf9ad43129b6276fbb65af974" candidate_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ff76b9242fda4d4586c2bb3da829b1ba"
I20250109 08:56:31.836614 20646 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5378208cf9ad43129b6276fbb65af974" candidate_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58"
I20250109 08:56:31.837182 20496 raft_consensus.cc:3054] T 5378208cf9ad43129b6276fbb65af974 P ff76b9242fda4d4586c2bb3da829b1ba [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:56:31.837281 20646 raft_consensus.cc:3049] T 5378208cf9ad43129b6276fbb65af974 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 1 LEADER]: Stepping down as leader of term 1
I20250109 08:56:31.837785 20646 raft_consensus.cc:738] T 5378208cf9ad43129b6276fbb65af974 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c5bffb8e4c5e4e7a8b4e919c98c26e58, State: Running, Role: LEADER
I20250109 08:56:31.838805 20646 consensus_queue.cc:260] T 5378208cf9ad43129b6276fbb65af974 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:31.840458 20646 raft_consensus.cc:3054] T 5378208cf9ad43129b6276fbb65af974 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:56:31.844949 20496 raft_consensus.cc:2463] T 5378208cf9ad43129b6276fbb65af974 P ff76b9242fda4d4586c2bb3da829b1ba [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b17b595d5da94e5dbf96bbef1ef3e668 in term 2.
I20250109 08:56:31.846385 20539 leader_election.cc:304] T 5378208cf9ad43129b6276fbb65af974 P b17b595d5da94e5dbf96bbef1ef3e668 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b17b595d5da94e5dbf96bbef1ef3e668, ff76b9242fda4d4586c2bb3da829b1ba; no voters:
I20250109 08:56:31.847229 20698 raft_consensus.cc:2798] T 5378208cf9ad43129b6276fbb65af974 P b17b595d5da94e5dbf96bbef1ef3e668 [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:56:31.847334 20646 raft_consensus.cc:2463] T 5378208cf9ad43129b6276fbb65af974 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b17b595d5da94e5dbf96bbef1ef3e668 in term 2.
I20250109 08:56:31.847724 20698 raft_consensus.cc:695] T 5378208cf9ad43129b6276fbb65af974 P b17b595d5da94e5dbf96bbef1ef3e668 [term 2 LEADER]: Becoming Leader. State: Replica: b17b595d5da94e5dbf96bbef1ef3e668, State: Running, Role: LEADER
I20250109 08:56:31.848564 20698 consensus_queue.cc:237] T 5378208cf9ad43129b6276fbb65af974 P b17b595d5da94e5dbf96bbef1ef3e668 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:31.855849 20391 catalog_manager.cc:5526] T 5378208cf9ad43129b6276fbb65af974 P b17b595d5da94e5dbf96bbef1ef3e668 reported cstate change: term changed from 1 to 2, leader changed from c5bffb8e4c5e4e7a8b4e919c98c26e58 (127.19.226.195) to b17b595d5da94e5dbf96bbef1ef3e668 (127.19.226.194). New cstate: current_term: 2 leader_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } health_report { overall_health: UNKNOWN } } }
I20250109 08:56:31.929730 20496 raft_consensus.cc:1270] T cfbc2774b91c40d59dc80655451f5bdd P ff76b9242fda4d4586c2bb3da829b1ba [term 2 FOLLOWER]: Refusing update from remote peer b17b595d5da94e5dbf96bbef1ef3e668: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:56:31.930938 20698 consensus_queue.cc:1035] T cfbc2774b91c40d59dc80655451f5bdd P b17b595d5da94e5dbf96bbef1ef3e668 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:56:31.939311 20646 raft_consensus.cc:1270] T cfbc2774b91c40d59dc80655451f5bdd P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 2 FOLLOWER]: Refusing update from remote peer b17b595d5da94e5dbf96bbef1ef3e668: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:56:31.942899 20698 consensus_queue.cc:1035] T cfbc2774b91c40d59dc80655451f5bdd P b17b595d5da94e5dbf96bbef1ef3e668 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:56:31.945747 20647 raft_consensus.cc:3054] T 74c881b69c3946429070c729ec66b0e4 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:56:31.952023 20647 raft_consensus.cc:1270] T 74c881b69c3946429070c729ec66b0e4 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 2 FOLLOWER]: Refusing update from remote peer b17b595d5da94e5dbf96bbef1ef3e668: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:56:31.953258 20698 consensus_queue.cc:1035] T 74c881b69c3946429070c729ec66b0e4 P b17b595d5da94e5dbf96bbef1ef3e668 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:56:31.960335 20496 raft_consensus.cc:1270] T 74c881b69c3946429070c729ec66b0e4 P ff76b9242fda4d4586c2bb3da829b1ba [term 2 FOLLOWER]: Refusing update from remote peer b17b595d5da94e5dbf96bbef1ef3e668: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:56:31.961743 20698 consensus_queue.cc:1035] T 74c881b69c3946429070c729ec66b0e4 P b17b595d5da94e5dbf96bbef1ef3e668 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:56:32.110335 20573 raft_consensus.cc:1270] T 4173445b18a24674ad20fe4b7e17c697 P b17b595d5da94e5dbf96bbef1ef3e668 [term 2 FOLLOWER]: Refusing update from remote peer ff76b9242fda4d4586c2bb3da829b1ba: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:56:32.111626 20697 consensus_queue.cc:1035] T 4173445b18a24674ad20fe4b7e17c697 P ff76b9242fda4d4586c2bb3da829b1ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:56:32.119954 20647 raft_consensus.cc:1270] T 4173445b18a24674ad20fe4b7e17c697 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 2 FOLLOWER]: Refusing update from remote peer ff76b9242fda4d4586c2bb3da829b1ba: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:56:32.121608 20770 consensus_queue.cc:1035] T 4173445b18a24674ad20fe4b7e17c697 P ff76b9242fda4d4586c2bb3da829b1ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:56:32.210503 20647 raft_consensus.cc:1270] T bdeeebda65be4889bf59888b7a7abfec P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 2 FOLLOWER]: Refusing update from remote peer b17b595d5da94e5dbf96bbef1ef3e668: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:56:32.211556 20698 consensus_queue.cc:1035] T bdeeebda65be4889bf59888b7a7abfec P b17b595d5da94e5dbf96bbef1ef3e668 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:56:32.218765 20496 raft_consensus.cc:1270] T bdeeebda65be4889bf59888b7a7abfec P ff76b9242fda4d4586c2bb3da829b1ba [term 2 FOLLOWER]: Refusing update from remote peer b17b595d5da94e5dbf96bbef1ef3e668: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:56:32.220180 20775 consensus_queue.cc:1035] T bdeeebda65be4889bf59888b7a7abfec P b17b595d5da94e5dbf96bbef1ef3e668 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:56:32.249804 20573 raft_consensus.cc:3054] T 6f383be53b01468f85e731bd2d4367c0 P b17b595d5da94e5dbf96bbef1ef3e668 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:56:32.254380 20496 raft_consensus.cc:1270] T 5378208cf9ad43129b6276fbb65af974 P ff76b9242fda4d4586c2bb3da829b1ba [term 2 FOLLOWER]: Refusing update from remote peer b17b595d5da94e5dbf96bbef1ef3e668: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:56:32.254885 20573 raft_consensus.cc:1270] T 6f383be53b01468f85e731bd2d4367c0 P b17b595d5da94e5dbf96bbef1ef3e668 [term 2 FOLLOWER]: Refusing update from remote peer ff76b9242fda4d4586c2bb3da829b1ba: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:56:32.255723 20775 consensus_queue.cc:1035] T 5378208cf9ad43129b6276fbb65af974 P b17b595d5da94e5dbf96bbef1ef3e668 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:56:32.256160 20770 consensus_queue.cc:1035] T 6f383be53b01468f85e731bd2d4367c0 P ff76b9242fda4d4586c2bb3da829b1ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:56:32.264528 20647 raft_consensus.cc:1270] T 6f383be53b01468f85e731bd2d4367c0 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 2 FOLLOWER]: Refusing update from remote peer ff76b9242fda4d4586c2bb3da829b1ba: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:56:32.265888 20716 consensus_queue.cc:1035] T 6f383be53b01468f85e731bd2d4367c0 P ff76b9242fda4d4586c2bb3da829b1ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:56:32.271886 20647 raft_consensus.cc:1270] T 5378208cf9ad43129b6276fbb65af974 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 2 FOLLOWER]: Refusing update from remote peer b17b595d5da94e5dbf96bbef1ef3e668: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:56:32.273511 20698 consensus_queue.cc:1035] T 5378208cf9ad43129b6276fbb65af974 P b17b595d5da94e5dbf96bbef1ef3e668 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:56:34.366879 20363 auto_leader_rebalancer-test.cc:269] The leader distribution is
I20250109 08:56:34.367589 20363 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250109 08:56:34.387041 20496 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "4173445b18a24674ad20fe4b7e17c697"
dest_uuid: "ff76b9242fda4d4586c2bb3da829b1ba"
mode: GRACEFUL
new_leader_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58"
from {username='slave'} at 127.0.0.1:46206
I20250109 08:56:34.387614 20496 raft_consensus.cc:604] T 4173445b18a24674ad20fe4b7e17c697 P ff76b9242fda4d4586c2bb3da829b1ba [term 2 LEADER]: Received request to transfer leadership to TS c5bffb8e4c5e4e7a8b4e919c98c26e58
I20250109 08:56:34.396723 20573 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "5378208cf9ad43129b6276fbb65af974"
dest_uuid: "b17b595d5da94e5dbf96bbef1ef3e668"
mode: GRACEFUL
new_leader_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58"
from {username='slave'} at 127.0.0.1:49918
I20250109 08:56:34.397217 20573 raft_consensus.cc:604] T 5378208cf9ad43129b6276fbb65af974 P b17b595d5da94e5dbf96bbef1ef3e668 [term 2 LEADER]: Received request to transfer leadership to TS c5bffb8e4c5e4e7a8b4e919c98c26e58
I20250109 08:56:34.398257 20363 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 2
I20250109 08:56:34.398762 20363 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250109 08:56:34.462720 20781 raft_consensus.cc:988] T 5378208cf9ad43129b6276fbb65af974 P b17b595d5da94e5dbf96bbef1ef3e668: : Instructing follower c5bffb8e4c5e4e7a8b4e919c98c26e58 to start an election
I20250109 08:56:34.463152 20774 raft_consensus.cc:1076] T 5378208cf9ad43129b6276fbb65af974 P b17b595d5da94e5dbf96bbef1ef3e668 [term 2 LEADER]: Signalling peer c5bffb8e4c5e4e7a8b4e919c98c26e58 to start an election
I20250109 08:56:34.464411 20647 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "5378208cf9ad43129b6276fbb65af974"
dest_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58"
from {username='slave'} at 127.0.0.1:49928
I20250109 08:56:34.464843 20647 raft_consensus.cc:491] T 5378208cf9ad43129b6276fbb65af974 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:56:34.465124 20647 raft_consensus.cc:3054] T 5378208cf9ad43129b6276fbb65af974 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 2 FOLLOWER]: Advancing to term 3
I20250109 08:56:34.469228 20647 raft_consensus.cc:513] T 5378208cf9ad43129b6276fbb65af974 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:34.470690 20647 leader_election.cc:290] T 5378208cf9ad43129b6276fbb65af974 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [CANDIDATE]: Term 3 election: Requested vote from peers ff76b9242fda4d4586c2bb3da829b1ba (127.19.226.193:42763), b17b595d5da94e5dbf96bbef1ef3e668 (127.19.226.194:42855)
I20250109 08:56:34.471350 20496 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5378208cf9ad43129b6276fbb65af974" candidate_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ff76b9242fda4d4586c2bb3da829b1ba"
I20250109 08:56:34.471678 20573 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5378208cf9ad43129b6276fbb65af974" candidate_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b17b595d5da94e5dbf96bbef1ef3e668"
I20250109 08:56:34.471843 20496 raft_consensus.cc:3054] T 5378208cf9ad43129b6276fbb65af974 P ff76b9242fda4d4586c2bb3da829b1ba [term 2 FOLLOWER]: Advancing to term 3
I20250109 08:56:34.472134 20573 raft_consensus.cc:3049] T 5378208cf9ad43129b6276fbb65af974 P b17b595d5da94e5dbf96bbef1ef3e668 [term 2 LEADER]: Stepping down as leader of term 2
I20250109 08:56:34.472414 20573 raft_consensus.cc:738] T 5378208cf9ad43129b6276fbb65af974 P b17b595d5da94e5dbf96bbef1ef3e668 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b17b595d5da94e5dbf96bbef1ef3e668, State: Running, Role: LEADER
I20250109 08:56:34.473052 20573 consensus_queue.cc:260] T 5378208cf9ad43129b6276fbb65af974 P b17b595d5da94e5dbf96bbef1ef3e668 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:34.473893 20573 raft_consensus.cc:3054] T 5378208cf9ad43129b6276fbb65af974 P b17b595d5da94e5dbf96bbef1ef3e668 [term 2 FOLLOWER]: Advancing to term 3
I20250109 08:56:34.476233 20496 raft_consensus.cc:2463] T 5378208cf9ad43129b6276fbb65af974 P ff76b9242fda4d4586c2bb3da829b1ba [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c5bffb8e4c5e4e7a8b4e919c98c26e58 in term 3.
I20250109 08:56:34.477218 20613 leader_election.cc:304] T 5378208cf9ad43129b6276fbb65af974 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c5bffb8e4c5e4e7a8b4e919c98c26e58, ff76b9242fda4d4586c2bb3da829b1ba; no voters:
I20250109 08:56:34.477942 20758 raft_consensus.cc:2798] T 5378208cf9ad43129b6276fbb65af974 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 3 FOLLOWER]: Leader election won for term 3
I20250109 08:56:34.478139 20573 raft_consensus.cc:2463] T 5378208cf9ad43129b6276fbb65af974 P b17b595d5da94e5dbf96bbef1ef3e668 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c5bffb8e4c5e4e7a8b4e919c98c26e58 in term 3.
I20250109 08:56:34.479008 20758 raft_consensus.cc:695] T 5378208cf9ad43129b6276fbb65af974 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 3 LEADER]: Becoming Leader. State: Replica: c5bffb8e4c5e4e7a8b4e919c98c26e58, State: Running, Role: LEADER
I20250109 08:56:34.479775 20758 consensus_queue.cc:237] T 5378208cf9ad43129b6276fbb65af974 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:34.485899 20392 catalog_manager.cc:5526] T 5378208cf9ad43129b6276fbb65af974 P c5bffb8e4c5e4e7a8b4e919c98c26e58 reported cstate change: term changed from 2 to 3, leader changed from b17b595d5da94e5dbf96bbef1ef3e668 (127.19.226.194) to c5bffb8e4c5e4e7a8b4e919c98c26e58 (127.19.226.195). New cstate: current_term: 3 leader_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } health_report { overall_health: HEALTHY } } }
I20250109 08:56:34.789310 20716 raft_consensus.cc:988] T 4173445b18a24674ad20fe4b7e17c697 P ff76b9242fda4d4586c2bb3da829b1ba: : Instructing follower c5bffb8e4c5e4e7a8b4e919c98c26e58 to start an election
I20250109 08:56:34.789710 20791 raft_consensus.cc:1076] T 4173445b18a24674ad20fe4b7e17c697 P ff76b9242fda4d4586c2bb3da829b1ba [term 2 LEADER]: Signalling peer c5bffb8e4c5e4e7a8b4e919c98c26e58 to start an election
I20250109 08:56:34.791023 20647 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "4173445b18a24674ad20fe4b7e17c697"
dest_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58"
from {username='slave'} at 127.0.0.1:49924
I20250109 08:56:34.791497 20647 raft_consensus.cc:491] T 4173445b18a24674ad20fe4b7e17c697 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:56:34.791765 20647 raft_consensus.cc:3054] T 4173445b18a24674ad20fe4b7e17c697 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 2 FOLLOWER]: Advancing to term 3
I20250109 08:56:34.796267 20647 raft_consensus.cc:513] T 4173445b18a24674ad20fe4b7e17c697 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:34.797708 20647 leader_election.cc:290] T 4173445b18a24674ad20fe4b7e17c697 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [CANDIDATE]: Term 3 election: Requested vote from peers ff76b9242fda4d4586c2bb3da829b1ba (127.19.226.193:42763), b17b595d5da94e5dbf96bbef1ef3e668 (127.19.226.194:42855)
I20250109 08:56:34.798458 20496 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4173445b18a24674ad20fe4b7e17c697" candidate_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ff76b9242fda4d4586c2bb3da829b1ba"
I20250109 08:56:34.798627 20573 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4173445b18a24674ad20fe4b7e17c697" candidate_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b17b595d5da94e5dbf96bbef1ef3e668"
I20250109 08:56:34.799050 20496 raft_consensus.cc:3049] T 4173445b18a24674ad20fe4b7e17c697 P ff76b9242fda4d4586c2bb3da829b1ba [term 2 LEADER]: Stepping down as leader of term 2
I20250109 08:56:34.799134 20573 raft_consensus.cc:3054] T 4173445b18a24674ad20fe4b7e17c697 P b17b595d5da94e5dbf96bbef1ef3e668 [term 2 FOLLOWER]: Advancing to term 3
I20250109 08:56:34.799422 20496 raft_consensus.cc:738] T 4173445b18a24674ad20fe4b7e17c697 P ff76b9242fda4d4586c2bb3da829b1ba [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ff76b9242fda4d4586c2bb3da829b1ba, State: Running, Role: LEADER
I20250109 08:56:34.800137 20496 consensus_queue.cc:260] T 4173445b18a24674ad20fe4b7e17c697 P ff76b9242fda4d4586c2bb3da829b1ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:34.801367 20496 raft_consensus.cc:3054] T 4173445b18a24674ad20fe4b7e17c697 P ff76b9242fda4d4586c2bb3da829b1ba [term 2 FOLLOWER]: Advancing to term 3
I20250109 08:56:34.805274 20573 raft_consensus.cc:2463] T 4173445b18a24674ad20fe4b7e17c697 P b17b595d5da94e5dbf96bbef1ef3e668 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c5bffb8e4c5e4e7a8b4e919c98c26e58 in term 3.
I20250109 08:56:34.806396 20611 leader_election.cc:304] T 4173445b18a24674ad20fe4b7e17c697 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b17b595d5da94e5dbf96bbef1ef3e668, c5bffb8e4c5e4e7a8b4e919c98c26e58; no voters:
I20250109 08:56:34.807041 20758 raft_consensus.cc:2798] T 4173445b18a24674ad20fe4b7e17c697 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 3 FOLLOWER]: Leader election won for term 3
I20250109 08:56:34.807017 20496 raft_consensus.cc:2463] T 4173445b18a24674ad20fe4b7e17c697 P ff76b9242fda4d4586c2bb3da829b1ba [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c5bffb8e4c5e4e7a8b4e919c98c26e58 in term 3.
I20250109 08:56:34.807636 20758 raft_consensus.cc:695] T 4173445b18a24674ad20fe4b7e17c697 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 3 LEADER]: Becoming Leader. State: Replica: c5bffb8e4c5e4e7a8b4e919c98c26e58, State: Running, Role: LEADER
I20250109 08:56:34.808491 20758 consensus_queue.cc:237] T 4173445b18a24674ad20fe4b7e17c697 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:34.817178 20392 catalog_manager.cc:5526] T 4173445b18a24674ad20fe4b7e17c697 P c5bffb8e4c5e4e7a8b4e919c98c26e58 reported cstate change: term changed from 2 to 3, leader changed from ff76b9242fda4d4586c2bb3da829b1ba (127.19.226.193) to c5bffb8e4c5e4e7a8b4e919c98c26e58 (127.19.226.195). New cstate: current_term: 3 leader_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } health_report { overall_health: HEALTHY } } }
I20250109 08:56:34.899374 20573 raft_consensus.cc:1270] T 5378208cf9ad43129b6276fbb65af974 P b17b595d5da94e5dbf96bbef1ef3e668 [term 3 FOLLOWER]: Refusing update from remote peer c5bffb8e4c5e4e7a8b4e919c98c26e58: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20250109 08:56:34.900665 20758 consensus_queue.cc:1035] T 5378208cf9ad43129b6276fbb65af974 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:56:34.911234 20496 raft_consensus.cc:1270] T 5378208cf9ad43129b6276fbb65af974 P ff76b9242fda4d4586c2bb3da829b1ba [term 3 FOLLOWER]: Refusing update from remote peer c5bffb8e4c5e4e7a8b4e919c98c26e58: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20250109 08:56:34.912742 20785 consensus_queue.cc:1035] T 5378208cf9ad43129b6276fbb65af974 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:56:35.271680 20573 raft_consensus.cc:1270] T 4173445b18a24674ad20fe4b7e17c697 P b17b595d5da94e5dbf96bbef1ef3e668 [term 3 FOLLOWER]: Refusing update from remote peer c5bffb8e4c5e4e7a8b4e919c98c26e58: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20250109 08:56:35.272822 20785 consensus_queue.cc:1035] T 4173445b18a24674ad20fe4b7e17c697 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:56:35.280848 20496 raft_consensus.cc:1270] T 4173445b18a24674ad20fe4b7e17c697 P ff76b9242fda4d4586c2bb3da829b1ba [term 3 FOLLOWER]: Refusing update from remote peer c5bffb8e4c5e4e7a8b4e919c98c26e58: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20250109 08:56:35.283849 20758 consensus_queue.cc:1035] T 4173445b18a24674ad20fe4b7e17c697 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:56:35.399523 20363 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250109 08:56:35.404235 20363 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250109 08:56:35.404667 20363 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250109 08:56:36.405481 20363 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250109 08:56:36.410575 20363 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250109 08:56:36.410995 20363 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250109 08:56:37.411733 20363 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250109 08:56:37.416503 20363 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250109 08:56:37.416901 20363 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250109 08:56:38.417928 20363 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250109 08:56:38.423267 20363 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250109 08:56:38.423681 20363 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250109 08:56:39.424451 20363 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250109 08:56:39.429685 20363 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250109 08:56:39.430142 20363 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250109 08:56:40.430910 20363 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250109 08:56:40.435428 20363 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250109 08:56:40.435848 20363 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250109 08:56:41.436643 20363 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250109 08:56:41.441536 20363 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250109 08:56:41.441972 20363 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250109 08:56:42.442726 20363 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250109 08:56:42.447521 20363 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250109 08:56:42.447861 20363 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250109 08:56:43.448489 20363 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250109 08:56:43.453009 20363 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250109 08:56:43.453342 20363 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250109 08:56:44.563969 20363 auto_leader_rebalancer-test.cc:288] The leader distribution is
I20250109 08:56:44.568006 20498 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "6b1d1214e23d47ff8ec02197a45d8484"
dest_uuid: "ff76b9242fda4d4586c2bb3da829b1ba"
mode: GRACEFUL
new_leader_uuid: "b17b595d5da94e5dbf96bbef1ef3e668"
from {username='slave'} at 127.0.0.1:46200
I20250109 08:56:44.568673 20498 raft_consensus.cc:604] T 6b1d1214e23d47ff8ec02197a45d8484 P ff76b9242fda4d4586c2bb3da829b1ba [term 1 LEADER]: Received request to transfer leadership to TS b17b595d5da94e5dbf96bbef1ef3e668
I20250109 08:56:44.573215 20646 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "4173445b18a24674ad20fe4b7e17c697"
dest_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58"
mode: GRACEFUL
new_leader_uuid: "b17b595d5da94e5dbf96bbef1ef3e668"
from {username='slave'} at 127.0.0.1:49942
I20250109 08:56:44.573765 20646 raft_consensus.cc:604] T 4173445b18a24674ad20fe4b7e17c697 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 3 LEADER]: Received request to transfer leadership to TS b17b595d5da94e5dbf96bbef1ef3e668
I20250109 08:56:44.577965 20498 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "bc8cfc045acc40efaa744f955d08ea97"
dest_uuid: "ff76b9242fda4d4586c2bb3da829b1ba"
mode: GRACEFUL
new_leader_uuid: "b17b595d5da94e5dbf96bbef1ef3e668"
from {username='slave'} at 127.0.0.1:46200
I20250109 08:56:44.578423 20498 raft_consensus.cc:604] T bc8cfc045acc40efaa744f955d08ea97 P ff76b9242fda4d4586c2bb3da829b1ba [term 1 LEADER]: Received request to transfer leadership to TS b17b595d5da94e5dbf96bbef1ef3e668
I20250109 08:56:44.582356 20498 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "6f383be53b01468f85e731bd2d4367c0"
dest_uuid: "ff76b9242fda4d4586c2bb3da829b1ba"
mode: GRACEFUL
new_leader_uuid: "b17b595d5da94e5dbf96bbef1ef3e668"
from {username='slave'} at 127.0.0.1:46200
I20250109 08:56:44.582791 20498 raft_consensus.cc:604] T 6f383be53b01468f85e731bd2d4367c0 P ff76b9242fda4d4586c2bb3da829b1ba [term 2 LEADER]: Received request to transfer leadership to TS b17b595d5da94e5dbf96bbef1ef3e668
I20250109 08:56:44.592903 20646 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "5378208cf9ad43129b6276fbb65af974"
dest_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58"
mode: GRACEFUL
new_leader_uuid: "b17b595d5da94e5dbf96bbef1ef3e668"
from {username='slave'} at 127.0.0.1:49942
I20250109 08:56:44.593359 20646 raft_consensus.cc:604] T 5378208cf9ad43129b6276fbb65af974 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 3 LEADER]: Received request to transfer leadership to TS b17b595d5da94e5dbf96bbef1ef3e668
I20250109 08:56:44.673269 20809 raft_consensus.cc:988] T 6f383be53b01468f85e731bd2d4367c0 P ff76b9242fda4d4586c2bb3da829b1ba: : Instructing follower b17b595d5da94e5dbf96bbef1ef3e668 to start an election
I20250109 08:56:44.673666 20809 raft_consensus.cc:1076] T 6f383be53b01468f85e731bd2d4367c0 P ff76b9242fda4d4586c2bb3da829b1ba [term 2 LEADER]: Signalling peer b17b595d5da94e5dbf96bbef1ef3e668 to start an election
I20250109 08:56:44.675124 20572 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6f383be53b01468f85e731bd2d4367c0"
dest_uuid: "b17b595d5da94e5dbf96bbef1ef3e668"
from {username='slave'} at 127.0.0.1:49890
I20250109 08:56:44.675628 20572 raft_consensus.cc:491] T 6f383be53b01468f85e731bd2d4367c0 P b17b595d5da94e5dbf96bbef1ef3e668 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:56:44.675967 20572 raft_consensus.cc:3054] T 6f383be53b01468f85e731bd2d4367c0 P b17b595d5da94e5dbf96bbef1ef3e668 [term 2 FOLLOWER]: Advancing to term 3
I20250109 08:56:44.680774 20572 raft_consensus.cc:513] T 6f383be53b01468f85e731bd2d4367c0 P b17b595d5da94e5dbf96bbef1ef3e668 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:44.682372 20572 leader_election.cc:290] T 6f383be53b01468f85e731bd2d4367c0 P b17b595d5da94e5dbf96bbef1ef3e668 [CANDIDATE]: Term 3 election: Requested vote from peers ff76b9242fda4d4586c2bb3da829b1ba (127.19.226.193:42763), c5bffb8e4c5e4e7a8b4e919c98c26e58 (127.19.226.195:45649)
I20250109 08:56:44.683296 20646 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f383be53b01468f85e731bd2d4367c0" candidate_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58"
I20250109 08:56:44.683198 20498 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f383be53b01468f85e731bd2d4367c0" candidate_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ff76b9242fda4d4586c2bb3da829b1ba"
I20250109 08:56:44.683970 20646 raft_consensus.cc:3054] T 6f383be53b01468f85e731bd2d4367c0 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 2 FOLLOWER]: Advancing to term 3
I20250109 08:56:44.684037 20498 raft_consensus.cc:3049] T 6f383be53b01468f85e731bd2d4367c0 P ff76b9242fda4d4586c2bb3da829b1ba [term 2 LEADER]: Stepping down as leader of term 2
I20250109 08:56:44.684459 20498 raft_consensus.cc:738] T 6f383be53b01468f85e731bd2d4367c0 P ff76b9242fda4d4586c2bb3da829b1ba [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ff76b9242fda4d4586c2bb3da829b1ba, State: Running, Role: LEADER
I20250109 08:56:44.685238 20498 consensus_queue.cc:260] T 6f383be53b01468f85e731bd2d4367c0 P ff76b9242fda4d4586c2bb3da829b1ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:44.686491 20498 raft_consensus.cc:3054] T 6f383be53b01468f85e731bd2d4367c0 P ff76b9242fda4d4586c2bb3da829b1ba [term 2 FOLLOWER]: Advancing to term 3
I20250109 08:56:44.690560 20646 raft_consensus.cc:2463] T 6f383be53b01468f85e731bd2d4367c0 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b17b595d5da94e5dbf96bbef1ef3e668 in term 3.
I20250109 08:56:44.691802 20536 leader_election.cc:304] T 6f383be53b01468f85e731bd2d4367c0 P b17b595d5da94e5dbf96bbef1ef3e668 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b17b595d5da94e5dbf96bbef1ef3e668, c5bffb8e4c5e4e7a8b4e919c98c26e58; no voters:
I20250109 08:56:44.692463 20498 raft_consensus.cc:2463] T 6f383be53b01468f85e731bd2d4367c0 P ff76b9242fda4d4586c2bb3da829b1ba [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b17b595d5da94e5dbf96bbef1ef3e668 in term 3.
I20250109 08:56:44.692566 20806 raft_consensus.cc:2798] T 6f383be53b01468f85e731bd2d4367c0 P b17b595d5da94e5dbf96bbef1ef3e668 [term 3 FOLLOWER]: Leader election won for term 3
I20250109 08:56:44.692991 20806 raft_consensus.cc:695] T 6f383be53b01468f85e731bd2d4367c0 P b17b595d5da94e5dbf96bbef1ef3e668 [term 3 LEADER]: Becoming Leader. State: Replica: b17b595d5da94e5dbf96bbef1ef3e668, State: Running, Role: LEADER
I20250109 08:56:44.694016 20806 consensus_queue.cc:237] T 6f383be53b01468f85e731bd2d4367c0 P b17b595d5da94e5dbf96bbef1ef3e668 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:44.702392 20392 catalog_manager.cc:5526] T 6f383be53b01468f85e731bd2d4367c0 P b17b595d5da94e5dbf96bbef1ef3e668 reported cstate change: term changed from 2 to 3, leader changed from ff76b9242fda4d4586c2bb3da829b1ba (127.19.226.193) to b17b595d5da94e5dbf96bbef1ef3e668 (127.19.226.194). New cstate: current_term: 3 leader_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } health_report { overall_health: UNKNOWN } } }
I20250109 08:56:44.719590 20807 raft_consensus.cc:988] T bc8cfc045acc40efaa744f955d08ea97 P ff76b9242fda4d4586c2bb3da829b1ba: : Instructing follower b17b595d5da94e5dbf96bbef1ef3e668 to start an election
I20250109 08:56:44.719966 20809 raft_consensus.cc:1076] T bc8cfc045acc40efaa744f955d08ea97 P ff76b9242fda4d4586c2bb3da829b1ba [term 1 LEADER]: Signalling peer b17b595d5da94e5dbf96bbef1ef3e668 to start an election
I20250109 08:56:44.721112 20572 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "bc8cfc045acc40efaa744f955d08ea97"
dest_uuid: "b17b595d5da94e5dbf96bbef1ef3e668"
from {username='slave'} at 127.0.0.1:49890
I20250109 08:56:44.721479 20572 raft_consensus.cc:491] T bc8cfc045acc40efaa744f955d08ea97 P b17b595d5da94e5dbf96bbef1ef3e668 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:56:44.721719 20572 raft_consensus.cc:3054] T bc8cfc045acc40efaa744f955d08ea97 P b17b595d5da94e5dbf96bbef1ef3e668 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:56:44.725386 20572 raft_consensus.cc:513] T bc8cfc045acc40efaa744f955d08ea97 P b17b595d5da94e5dbf96bbef1ef3e668 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:44.726563 20572 leader_election.cc:290] T bc8cfc045acc40efaa744f955d08ea97 P b17b595d5da94e5dbf96bbef1ef3e668 [CANDIDATE]: Term 2 election: Requested vote from peers ff76b9242fda4d4586c2bb3da829b1ba (127.19.226.193:42763), c5bffb8e4c5e4e7a8b4e919c98c26e58 (127.19.226.195:45649)
I20250109 08:56:44.727392 20498 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc8cfc045acc40efaa744f955d08ea97" candidate_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ff76b9242fda4d4586c2bb3da829b1ba"
I20250109 08:56:44.727421 20646 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc8cfc045acc40efaa744f955d08ea97" candidate_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58"
I20250109 08:56:44.728034 20498 raft_consensus.cc:3049] T bc8cfc045acc40efaa744f955d08ea97 P ff76b9242fda4d4586c2bb3da829b1ba [term 1 LEADER]: Stepping down as leader of term 1
I20250109 08:56:44.728106 20646 raft_consensus.cc:3054] T bc8cfc045acc40efaa744f955d08ea97 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:56:44.728299 20498 raft_consensus.cc:738] T bc8cfc045acc40efaa744f955d08ea97 P ff76b9242fda4d4586c2bb3da829b1ba [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ff76b9242fda4d4586c2bb3da829b1ba, State: Running, Role: LEADER
I20250109 08:56:44.728926 20498 consensus_queue.cc:260] T bc8cfc045acc40efaa744f955d08ea97 P ff76b9242fda4d4586c2bb3da829b1ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:44.729718 20498 raft_consensus.cc:3054] T bc8cfc045acc40efaa744f955d08ea97 P ff76b9242fda4d4586c2bb3da829b1ba [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:56:44.732264 20646 raft_consensus.cc:2463] T bc8cfc045acc40efaa744f955d08ea97 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b17b595d5da94e5dbf96bbef1ef3e668 in term 2.
I20250109 08:56:44.733134 20536 leader_election.cc:304] T bc8cfc045acc40efaa744f955d08ea97 P b17b595d5da94e5dbf96bbef1ef3e668 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b17b595d5da94e5dbf96bbef1ef3e668, c5bffb8e4c5e4e7a8b4e919c98c26e58; no voters:
I20250109 08:56:44.733556 20498 raft_consensus.cc:2463] T bc8cfc045acc40efaa744f955d08ea97 P ff76b9242fda4d4586c2bb3da829b1ba [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b17b595d5da94e5dbf96bbef1ef3e668 in term 2.
I20250109 08:56:44.733738 20806 raft_consensus.cc:2798] T bc8cfc045acc40efaa744f955d08ea97 P b17b595d5da94e5dbf96bbef1ef3e668 [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:56:44.734149 20806 raft_consensus.cc:695] T bc8cfc045acc40efaa744f955d08ea97 P b17b595d5da94e5dbf96bbef1ef3e668 [term 2 LEADER]: Becoming Leader. State: Replica: b17b595d5da94e5dbf96bbef1ef3e668, State: Running, Role: LEADER
I20250109 08:56:44.734908 20806 consensus_queue.cc:237] T bc8cfc045acc40efaa744f955d08ea97 P b17b595d5da94e5dbf96bbef1ef3e668 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:44.740690 20392 catalog_manager.cc:5526] T bc8cfc045acc40efaa744f955d08ea97 P b17b595d5da94e5dbf96bbef1ef3e668 reported cstate change: term changed from 1 to 2, leader changed from ff76b9242fda4d4586c2bb3da829b1ba (127.19.226.193) to b17b595d5da94e5dbf96bbef1ef3e668 (127.19.226.194). New cstate: current_term: 2 leader_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } health_report { overall_health: UNKNOWN } } }
I20250109 08:56:45.021777 20758 raft_consensus.cc:988] T 4173445b18a24674ad20fe4b7e17c697 P c5bffb8e4c5e4e7a8b4e919c98c26e58: : Instructing follower b17b595d5da94e5dbf96bbef1ef3e668 to start an election
I20250109 08:56:45.022280 20818 raft_consensus.cc:1076] T 4173445b18a24674ad20fe4b7e17c697 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 3 LEADER]: Signalling peer b17b595d5da94e5dbf96bbef1ef3e668 to start an election
I20250109 08:56:45.023711 20572 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "4173445b18a24674ad20fe4b7e17c697"
dest_uuid: "b17b595d5da94e5dbf96bbef1ef3e668"
from {username='slave'} at 127.0.0.1:49904
I20250109 08:56:45.024147 20572 raft_consensus.cc:491] T 4173445b18a24674ad20fe4b7e17c697 P b17b595d5da94e5dbf96bbef1ef3e668 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:56:45.024416 20572 raft_consensus.cc:3054] T 4173445b18a24674ad20fe4b7e17c697 P b17b595d5da94e5dbf96bbef1ef3e668 [term 3 FOLLOWER]: Advancing to term 4
I20250109 08:56:45.044808 20758 raft_consensus.cc:988] T 5378208cf9ad43129b6276fbb65af974 P c5bffb8e4c5e4e7a8b4e919c98c26e58: : Instructing follower b17b595d5da94e5dbf96bbef1ef3e668 to start an election
I20250109 08:56:45.045215 20758 raft_consensus.cc:1076] T 5378208cf9ad43129b6276fbb65af974 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 3 LEADER]: Signalling peer b17b595d5da94e5dbf96bbef1ef3e668 to start an election
I20250109 08:56:45.046517 20573 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "5378208cf9ad43129b6276fbb65af974"
dest_uuid: "b17b595d5da94e5dbf96bbef1ef3e668"
from {username='slave'} at 127.0.0.1:49904
I20250109 08:56:45.046949 20573 raft_consensus.cc:491] T 5378208cf9ad43129b6276fbb65af974 P b17b595d5da94e5dbf96bbef1ef3e668 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:56:45.047173 20573 raft_consensus.cc:3054] T 5378208cf9ad43129b6276fbb65af974 P b17b595d5da94e5dbf96bbef1ef3e668 [term 3 FOLLOWER]: Advancing to term 4
I20250109 08:56:45.052824 20572 raft_consensus.cc:513] T 4173445b18a24674ad20fe4b7e17c697 P b17b595d5da94e5dbf96bbef1ef3e668 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:45.054335 20572 leader_election.cc:290] T 4173445b18a24674ad20fe4b7e17c697 P b17b595d5da94e5dbf96bbef1ef3e668 [CANDIDATE]: Term 4 election: Requested vote from peers ff76b9242fda4d4586c2bb3da829b1ba (127.19.226.193:42763), c5bffb8e4c5e4e7a8b4e919c98c26e58 (127.19.226.195:45649)
I20250109 08:56:45.055173 20498 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4173445b18a24674ad20fe4b7e17c697" candidate_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "ff76b9242fda4d4586c2bb3da829b1ba"
I20250109 08:56:45.055761 20498 raft_consensus.cc:3054] T 4173445b18a24674ad20fe4b7e17c697 P ff76b9242fda4d4586c2bb3da829b1ba [term 3 FOLLOWER]: Advancing to term 4
I20250109 08:56:45.056190 20646 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4173445b18a24674ad20fe4b7e17c697" candidate_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58"
I20250109 08:56:45.056767 20646 raft_consensus.cc:3049] T 4173445b18a24674ad20fe4b7e17c697 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 3 LEADER]: Stepping down as leader of term 3
I20250109 08:56:45.057060 20646 raft_consensus.cc:738] T 4173445b18a24674ad20fe4b7e17c697 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: c5bffb8e4c5e4e7a8b4e919c98c26e58, State: Running, Role: LEADER
I20250109 08:56:45.057739 20646 consensus_queue.cc:260] T 4173445b18a24674ad20fe4b7e17c697 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:45.058631 20646 raft_consensus.cc:3054] T 4173445b18a24674ad20fe4b7e17c697 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 3 FOLLOWER]: Advancing to term 4
I20250109 08:56:45.092733 20646 raft_consensus.cc:2463] T 4173445b18a24674ad20fe4b7e17c697 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b17b595d5da94e5dbf96bbef1ef3e668 in term 4.
I20250109 08:56:45.092761 20498 raft_consensus.cc:2463] T 4173445b18a24674ad20fe4b7e17c697 P ff76b9242fda4d4586c2bb3da829b1ba [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b17b595d5da94e5dbf96bbef1ef3e668 in term 4.
I20250109 08:56:45.092794 20573 raft_consensus.cc:513] T 5378208cf9ad43129b6276fbb65af974 P b17b595d5da94e5dbf96bbef1ef3e668 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:45.094185 20536 leader_election.cc:304] T 4173445b18a24674ad20fe4b7e17c697 P b17b595d5da94e5dbf96bbef1ef3e668 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b17b595d5da94e5dbf96bbef1ef3e668, c5bffb8e4c5e4e7a8b4e919c98c26e58; no voters:
I20250109 08:56:45.094841 20573 leader_election.cc:290] T 5378208cf9ad43129b6276fbb65af974 P b17b595d5da94e5dbf96bbef1ef3e668 [CANDIDATE]: Term 4 election: Requested vote from peers ff76b9242fda4d4586c2bb3da829b1ba (127.19.226.193:42763), c5bffb8e4c5e4e7a8b4e919c98c26e58 (127.19.226.195:45649)
I20250109 08:56:45.095562 20806 raft_consensus.cc:2798] T 4173445b18a24674ad20fe4b7e17c697 P b17b595d5da94e5dbf96bbef1ef3e668 [term 4 FOLLOWER]: Leader election won for term 4
I20250109 08:56:45.095882 20498 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5378208cf9ad43129b6276fbb65af974" candidate_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "ff76b9242fda4d4586c2bb3da829b1ba"
I20250109 08:56:45.096438 20498 raft_consensus.cc:3054] T 5378208cf9ad43129b6276fbb65af974 P ff76b9242fda4d4586c2bb3da829b1ba [term 3 FOLLOWER]: Advancing to term 4
I20250109 08:56:45.096571 20646 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5378208cf9ad43129b6276fbb65af974" candidate_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58"
I20250109 08:56:45.096768 20806 raft_consensus.cc:695] T 4173445b18a24674ad20fe4b7e17c697 P b17b595d5da94e5dbf96bbef1ef3e668 [term 4 LEADER]: Becoming Leader. State: Replica: b17b595d5da94e5dbf96bbef1ef3e668, State: Running, Role: LEADER
I20250109 08:56:45.097218 20646 raft_consensus.cc:3049] T 5378208cf9ad43129b6276fbb65af974 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 3 LEADER]: Stepping down as leader of term 3
I20250109 08:56:45.097576 20646 raft_consensus.cc:738] T 5378208cf9ad43129b6276fbb65af974 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: c5bffb8e4c5e4e7a8b4e919c98c26e58, State: Running, Role: LEADER
I20250109 08:56:45.097751 20806 consensus_queue.cc:237] T 4173445b18a24674ad20fe4b7e17c697 P b17b595d5da94e5dbf96bbef1ef3e668 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:45.098321 20646 consensus_queue.cc:260] T 5378208cf9ad43129b6276fbb65af974 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:45.099478 20646 raft_consensus.cc:3054] T 5378208cf9ad43129b6276fbb65af974 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 3 FOLLOWER]: Advancing to term 4
I20250109 08:56:45.105038 20392 catalog_manager.cc:5526] T 4173445b18a24674ad20fe4b7e17c697 P b17b595d5da94e5dbf96bbef1ef3e668 reported cstate change: term changed from 3 to 4, leader changed from c5bffb8e4c5e4e7a8b4e919c98c26e58 (127.19.226.195) to b17b595d5da94e5dbf96bbef1ef3e668 (127.19.226.194). New cstate: current_term: 4 leader_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } health_report { overall_health: UNKNOWN } } }
I20250109 08:56:45.112730 20807 raft_consensus.cc:988] T 6b1d1214e23d47ff8ec02197a45d8484 P ff76b9242fda4d4586c2bb3da829b1ba: : Instructing follower b17b595d5da94e5dbf96bbef1ef3e668 to start an election
I20250109 08:56:45.113121 20809 raft_consensus.cc:1076] T 6b1d1214e23d47ff8ec02197a45d8484 P ff76b9242fda4d4586c2bb3da829b1ba [term 1 LEADER]: Signalling peer b17b595d5da94e5dbf96bbef1ef3e668 to start an election
I20250109 08:56:45.114346 20573 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6b1d1214e23d47ff8ec02197a45d8484"
dest_uuid: "b17b595d5da94e5dbf96bbef1ef3e668"
from {username='slave'} at 127.0.0.1:49890
I20250109 08:56:45.114758 20573 raft_consensus.cc:491] T 6b1d1214e23d47ff8ec02197a45d8484 P b17b595d5da94e5dbf96bbef1ef3e668 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:56:45.115026 20573 raft_consensus.cc:3054] T 6b1d1214e23d47ff8ec02197a45d8484 P b17b595d5da94e5dbf96bbef1ef3e668 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:56:45.120059 20647 raft_consensus.cc:1270] T bc8cfc045acc40efaa744f955d08ea97 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 2 FOLLOWER]: Refusing update from remote peer b17b595d5da94e5dbf96bbef1ef3e668: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:56:45.121085 20806 consensus_queue.cc:1035] T bc8cfc045acc40efaa744f955d08ea97 P b17b595d5da94e5dbf96bbef1ef3e668 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:56:45.130744 20496 raft_consensus.cc:1270] T bc8cfc045acc40efaa744f955d08ea97 P ff76b9242fda4d4586c2bb3da829b1ba [term 2 FOLLOWER]: Refusing update from remote peer b17b595d5da94e5dbf96bbef1ef3e668: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:56:45.132206 20806 consensus_queue.cc:1035] T bc8cfc045acc40efaa744f955d08ea97 P b17b595d5da94e5dbf96bbef1ef3e668 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:56:45.193791 20646 raft_consensus.cc:2463] T 5378208cf9ad43129b6276fbb65af974 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b17b595d5da94e5dbf96bbef1ef3e668 in term 4.
I20250109 08:56:45.193792 20498 raft_consensus.cc:2463] T 5378208cf9ad43129b6276fbb65af974 P ff76b9242fda4d4586c2bb3da829b1ba [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b17b595d5da94e5dbf96bbef1ef3e668 in term 4.
I20250109 08:56:45.193825 20573 raft_consensus.cc:513] T 6b1d1214e23d47ff8ec02197a45d8484 P b17b595d5da94e5dbf96bbef1ef3e668 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:45.195273 20536 leader_election.cc:304] T 5378208cf9ad43129b6276fbb65af974 P b17b595d5da94e5dbf96bbef1ef3e668 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b17b595d5da94e5dbf96bbef1ef3e668, c5bffb8e4c5e4e7a8b4e919c98c26e58; no voters:
I20250109 08:56:45.195802 20573 leader_election.cc:290] T 6b1d1214e23d47ff8ec02197a45d8484 P b17b595d5da94e5dbf96bbef1ef3e668 [CANDIDATE]: Term 2 election: Requested vote from peers ff76b9242fda4d4586c2bb3da829b1ba (127.19.226.193:42763), c5bffb8e4c5e4e7a8b4e919c98c26e58 (127.19.226.195:45649)
I20250109 08:56:45.196554 20808 raft_consensus.cc:2798] T 5378208cf9ad43129b6276fbb65af974 P b17b595d5da94e5dbf96bbef1ef3e668 [term 4 FOLLOWER]: Leader election won for term 4
I20250109 08:56:45.197005 20808 raft_consensus.cc:695] T 5378208cf9ad43129b6276fbb65af974 P b17b595d5da94e5dbf96bbef1ef3e668 [term 4 LEADER]: Becoming Leader. State: Replica: b17b595d5da94e5dbf96bbef1ef3e668, State: Running, Role: LEADER
I20250109 08:56:45.197098 20498 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b1d1214e23d47ff8ec02197a45d8484" candidate_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ff76b9242fda4d4586c2bb3da829b1ba"
I20250109 08:56:45.197348 20646 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b1d1214e23d47ff8ec02197a45d8484" candidate_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58"
I20250109 08:56:45.197661 20498 raft_consensus.cc:3049] T 6b1d1214e23d47ff8ec02197a45d8484 P ff76b9242fda4d4586c2bb3da829b1ba [term 1 LEADER]: Stepping down as leader of term 1
I20250109 08:56:45.197844 20646 raft_consensus.cc:3054] T 6b1d1214e23d47ff8ec02197a45d8484 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:56:45.197963 20498 raft_consensus.cc:738] T 6b1d1214e23d47ff8ec02197a45d8484 P ff76b9242fda4d4586c2bb3da829b1ba [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ff76b9242fda4d4586c2bb3da829b1ba, State: Running, Role: LEADER
I20250109 08:56:45.197777 20808 consensus_queue.cc:237] T 5378208cf9ad43129b6276fbb65af974 P b17b595d5da94e5dbf96bbef1ef3e668 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:45.198769 20498 consensus_queue.cc:260] T 6b1d1214e23d47ff8ec02197a45d8484 P ff76b9242fda4d4586c2bb3da829b1ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:45.199994 20498 raft_consensus.cc:3054] T 6b1d1214e23d47ff8ec02197a45d8484 P ff76b9242fda4d4586c2bb3da829b1ba [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:56:45.205833 20647 raft_consensus.cc:1270] T 6f383be53b01468f85e731bd2d4367c0 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 3 FOLLOWER]: Refusing update from remote peer b17b595d5da94e5dbf96bbef1ef3e668: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20250109 08:56:45.206970 20806 consensus_queue.cc:1035] T 6f383be53b01468f85e731bd2d4367c0 P b17b595d5da94e5dbf96bbef1ef3e668 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:56:45.206888 20395 catalog_manager.cc:5526] T 5378208cf9ad43129b6276fbb65af974 P b17b595d5da94e5dbf96bbef1ef3e668 reported cstate change: term changed from 3 to 4, leader changed from c5bffb8e4c5e4e7a8b4e919c98c26e58 (127.19.226.195) to b17b595d5da94e5dbf96bbef1ef3e668 (127.19.226.194). New cstate: current_term: 4 leader_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } health_report { overall_health: UNKNOWN } } }
I20250109 08:56:45.215138 20496 raft_consensus.cc:1270] T 6f383be53b01468f85e731bd2d4367c0 P ff76b9242fda4d4586c2bb3da829b1ba [term 3 FOLLOWER]: Refusing update from remote peer b17b595d5da94e5dbf96bbef1ef3e668: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20250109 08:56:45.216688 20806 consensus_queue.cc:1035] T 6f383be53b01468f85e731bd2d4367c0 P b17b595d5da94e5dbf96bbef1ef3e668 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20250109 08:56:45.243829 20594 debug-util.cc:398] Leaking SignalData structure 0x7b0800134060 after lost signal to thread 20366
W20250109 08:56:45.244625 20594 debug-util.cc:398] Leaking SignalData structure 0x7b0800134740 after lost signal to thread 20429
W20250109 08:56:45.245312 20594 debug-util.cc:398] Leaking SignalData structure 0x7b080009df00 after lost signal to thread 20522
W20250109 08:56:45.245961 20594 debug-util.cc:398] Leaking SignalData structure 0x7b08001eb780 after lost signal to thread 20597
W20250109 08:56:45.246582 20594 debug-util.cc:398] Leaking SignalData structure 0x7b08001f9060 after lost signal to thread 20671
I20250109 08:56:45.296030 20646 raft_consensus.cc:2463] T 6b1d1214e23d47ff8ec02197a45d8484 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b17b595d5da94e5dbf96bbef1ef3e668 in term 2.
I20250109 08:56:45.296030 20498 raft_consensus.cc:2463] T 6b1d1214e23d47ff8ec02197a45d8484 P ff76b9242fda4d4586c2bb3da829b1ba [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b17b595d5da94e5dbf96bbef1ef3e668 in term 2.
I20250109 08:56:45.297472 20536 leader_election.cc:304] T 6b1d1214e23d47ff8ec02197a45d8484 P b17b595d5da94e5dbf96bbef1ef3e668 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b17b595d5da94e5dbf96bbef1ef3e668, c5bffb8e4c5e4e7a8b4e919c98c26e58; no voters:
I20250109 08:56:45.298352 20806 raft_consensus.cc:2798] T 6b1d1214e23d47ff8ec02197a45d8484 P b17b595d5da94e5dbf96bbef1ef3e668 [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:56:45.298804 20806 raft_consensus.cc:695] T 6b1d1214e23d47ff8ec02197a45d8484 P b17b595d5da94e5dbf96bbef1ef3e668 [term 2 LEADER]: Becoming Leader. State: Replica: b17b595d5da94e5dbf96bbef1ef3e668, State: Running, Role: LEADER
I20250109 08:56:45.299603 20806 consensus_queue.cc:237] T 6b1d1214e23d47ff8ec02197a45d8484 P b17b595d5da94e5dbf96bbef1ef3e668 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:45.307336 20395 catalog_manager.cc:5526] T 6b1d1214e23d47ff8ec02197a45d8484 P b17b595d5da94e5dbf96bbef1ef3e668 reported cstate change: term changed from 1 to 2, leader changed from ff76b9242fda4d4586c2bb3da829b1ba (127.19.226.193) to b17b595d5da94e5dbf96bbef1ef3e668 (127.19.226.194). New cstate: current_term: 2 leader_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } health_report { overall_health: UNKNOWN } } }
I20250109 08:56:45.624874 20498 raft_consensus.cc:1270] T 5378208cf9ad43129b6276fbb65af974 P ff76b9242fda4d4586c2bb3da829b1ba [term 4 FOLLOWER]: Refusing update from remote peer b17b595d5da94e5dbf96bbef1ef3e668: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20250109 08:56:45.626101 20806 consensus_queue.cc:1035] T 5378208cf9ad43129b6276fbb65af974 P b17b595d5da94e5dbf96bbef1ef3e668 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250109 08:56:45.636273 20646 raft_consensus.cc:1270] T 5378208cf9ad43129b6276fbb65af974 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 4 FOLLOWER]: Refusing update from remote peer b17b595d5da94e5dbf96bbef1ef3e668: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20250109 08:56:45.637892 20827 consensus_queue.cc:1035] T 5378208cf9ad43129b6276fbb65af974 P b17b595d5da94e5dbf96bbef1ef3e668 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250109 08:56:45.678185 20646 raft_consensus.cc:1270] T 4173445b18a24674ad20fe4b7e17c697 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 4 FOLLOWER]: Refusing update from remote peer b17b595d5da94e5dbf96bbef1ef3e668: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20250109 08:56:45.679299 20498 raft_consensus.cc:1270] T 4173445b18a24674ad20fe4b7e17c697 P ff76b9242fda4d4586c2bb3da829b1ba [term 4 FOLLOWER]: Refusing update from remote peer b17b595d5da94e5dbf96bbef1ef3e668: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20250109 08:56:45.679627 20827 consensus_queue.cc:1035] T 4173445b18a24674ad20fe4b7e17c697 P b17b595d5da94e5dbf96bbef1ef3e668 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250109 08:56:45.680471 20808 consensus_queue.cc:1035] T 4173445b18a24674ad20fe4b7e17c697 P b17b595d5da94e5dbf96bbef1ef3e668 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250109 08:56:45.788157 20646 raft_consensus.cc:1270] T 6b1d1214e23d47ff8ec02197a45d8484 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 2 FOLLOWER]: Refusing update from remote peer b17b595d5da94e5dbf96bbef1ef3e668: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:56:45.789294 20827 consensus_queue.cc:1035] T 6b1d1214e23d47ff8ec02197a45d8484 P b17b595d5da94e5dbf96bbef1ef3e668 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:56:45.800257 20496 raft_consensus.cc:1270] T 6b1d1214e23d47ff8ec02197a45d8484 P ff76b9242fda4d4586c2bb3da829b1ba [term 2 FOLLOWER]: Refusing update from remote peer b17b595d5da94e5dbf96bbef1ef3e668: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:56:45.801550 20834 consensus_queue.cc:1035] T 6b1d1214e23d47ff8ec02197a45d8484 P b17b595d5da94e5dbf96bbef1ef3e668 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:56:47.621441 20363 auto_leader_rebalancer-test.cc:303] The leader distribution is
I20250109 08:56:47.622066 20363 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250109 08:56:47.628439 20573 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "5378208cf9ad43129b6276fbb65af974"
dest_uuid: "b17b595d5da94e5dbf96bbef1ef3e668"
mode: GRACEFUL
new_leader_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58"
from {username='slave'} at 127.0.0.1:49918
I20250109 08:56:47.628916 20573 raft_consensus.cc:604] T 5378208cf9ad43129b6276fbb65af974 P b17b595d5da94e5dbf96bbef1ef3e668 [term 4 LEADER]: Received request to transfer leadership to TS c5bffb8e4c5e4e7a8b4e919c98c26e58
I20250109 08:56:47.631023 20573 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "6f383be53b01468f85e731bd2d4367c0"
dest_uuid: "b17b595d5da94e5dbf96bbef1ef3e668"
mode: GRACEFUL
new_leader_uuid: "ff76b9242fda4d4586c2bb3da829b1ba"
from {username='slave'} at 127.0.0.1:49918
I20250109 08:56:47.631429 20573 raft_consensus.cc:604] T 6f383be53b01468f85e731bd2d4367c0 P b17b595d5da94e5dbf96bbef1ef3e668 [term 3 LEADER]: Received request to transfer leadership to TS ff76b9242fda4d4586c2bb3da829b1ba
I20250109 08:56:47.633276 20573 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "bc8cfc045acc40efaa744f955d08ea97"
dest_uuid: "b17b595d5da94e5dbf96bbef1ef3e668"
mode: GRACEFUL
new_leader_uuid: "ff76b9242fda4d4586c2bb3da829b1ba"
from {username='slave'} at 127.0.0.1:49918
I20250109 08:56:47.633680 20573 raft_consensus.cc:604] T bc8cfc045acc40efaa744f955d08ea97 P b17b595d5da94e5dbf96bbef1ef3e668 [term 2 LEADER]: Received request to transfer leadership to TS ff76b9242fda4d4586c2bb3da829b1ba
I20250109 08:56:47.635361 20573 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "bdeeebda65be4889bf59888b7a7abfec"
dest_uuid: "b17b595d5da94e5dbf96bbef1ef3e668"
mode: GRACEFUL
new_leader_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58"
from {username='slave'} at 127.0.0.1:49918
I20250109 08:56:47.635743 20573 raft_consensus.cc:604] T bdeeebda65be4889bf59888b7a7abfec P b17b595d5da94e5dbf96bbef1ef3e668 [term 2 LEADER]: Received request to transfer leadership to TS c5bffb8e4c5e4e7a8b4e919c98c26e58
I20250109 08:56:47.637492 20573 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "cfbc2774b91c40d59dc80655451f5bdd"
dest_uuid: "b17b595d5da94e5dbf96bbef1ef3e668"
mode: GRACEFUL
new_leader_uuid: "ff76b9242fda4d4586c2bb3da829b1ba"
from {username='slave'} at 127.0.0.1:49918
I20250109 08:56:47.637876 20573 raft_consensus.cc:604] T cfbc2774b91c40d59dc80655451f5bdd P b17b595d5da94e5dbf96bbef1ef3e668 [term 2 LEADER]: Received request to transfer leadership to TS ff76b9242fda4d4586c2bb3da829b1ba
I20250109 08:56:47.638643 20363 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 5
I20250109 08:56:47.639034 20363 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250109 08:56:47.797631 20838 raft_consensus.cc:988] T bdeeebda65be4889bf59888b7a7abfec P b17b595d5da94e5dbf96bbef1ef3e668: : Instructing follower c5bffb8e4c5e4e7a8b4e919c98c26e58 to start an election
I20250109 08:56:47.798312 20834 raft_consensus.cc:1076] T bdeeebda65be4889bf59888b7a7abfec P b17b595d5da94e5dbf96bbef1ef3e668 [term 2 LEADER]: Signalling peer c5bffb8e4c5e4e7a8b4e919c98c26e58 to start an election
I20250109 08:56:47.799805 20646 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "bdeeebda65be4889bf59888b7a7abfec"
dest_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58"
from {username='slave'} at 127.0.0.1:49928
I20250109 08:56:47.800427 20646 raft_consensus.cc:491] T bdeeebda65be4889bf59888b7a7abfec P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:56:47.800801 20646 raft_consensus.cc:3054] T bdeeebda65be4889bf59888b7a7abfec P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 2 FOLLOWER]: Advancing to term 3
I20250109 08:56:47.806165 20646 raft_consensus.cc:513] T bdeeebda65be4889bf59888b7a7abfec P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:47.808461 20646 leader_election.cc:290] T bdeeebda65be4889bf59888b7a7abfec P c5bffb8e4c5e4e7a8b4e919c98c26e58 [CANDIDATE]: Term 3 election: Requested vote from peers ff76b9242fda4d4586c2bb3da829b1ba (127.19.226.193:42763), b17b595d5da94e5dbf96bbef1ef3e668 (127.19.226.194:42855)
I20250109 08:56:47.810348 20498 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bdeeebda65be4889bf59888b7a7abfec" candidate_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ff76b9242fda4d4586c2bb3da829b1ba"
I20250109 08:56:47.810329 20573 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bdeeebda65be4889bf59888b7a7abfec" candidate_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b17b595d5da94e5dbf96bbef1ef3e668"
I20250109 08:56:47.811120 20498 raft_consensus.cc:3054] T bdeeebda65be4889bf59888b7a7abfec P ff76b9242fda4d4586c2bb3da829b1ba [term 2 FOLLOWER]: Advancing to term 3
I20250109 08:56:47.811120 20573 raft_consensus.cc:3049] T bdeeebda65be4889bf59888b7a7abfec P b17b595d5da94e5dbf96bbef1ef3e668 [term 2 LEADER]: Stepping down as leader of term 2
I20250109 08:56:47.811637 20573 raft_consensus.cc:738] T bdeeebda65be4889bf59888b7a7abfec P b17b595d5da94e5dbf96bbef1ef3e668 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b17b595d5da94e5dbf96bbef1ef3e668, State: Running, Role: LEADER
I20250109 08:56:47.812505 20573 consensus_queue.cc:260] T bdeeebda65be4889bf59888b7a7abfec P b17b595d5da94e5dbf96bbef1ef3e668 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:47.814152 20573 raft_consensus.cc:3054] T bdeeebda65be4889bf59888b7a7abfec P b17b595d5da94e5dbf96bbef1ef3e668 [term 2 FOLLOWER]: Advancing to term 3
I20250109 08:56:47.818912 20498 raft_consensus.cc:2463] T bdeeebda65be4889bf59888b7a7abfec P ff76b9242fda4d4586c2bb3da829b1ba [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c5bffb8e4c5e4e7a8b4e919c98c26e58 in term 3.
I20250109 08:56:47.820433 20613 leader_election.cc:304] T bdeeebda65be4889bf59888b7a7abfec P c5bffb8e4c5e4e7a8b4e919c98c26e58 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c5bffb8e4c5e4e7a8b4e919c98c26e58, ff76b9242fda4d4586c2bb3da829b1ba; no voters:
I20250109 08:56:47.821426 20758 raft_consensus.cc:2798] T bdeeebda65be4889bf59888b7a7abfec P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 3 FOLLOWER]: Leader election won for term 3
I20250109 08:56:47.821637 20573 raft_consensus.cc:2463] T bdeeebda65be4889bf59888b7a7abfec P b17b595d5da94e5dbf96bbef1ef3e668 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c5bffb8e4c5e4e7a8b4e919c98c26e58 in term 3.
I20250109 08:56:47.822650 20758 raft_consensus.cc:695] T bdeeebda65be4889bf59888b7a7abfec P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 3 LEADER]: Becoming Leader. State: Replica: c5bffb8e4c5e4e7a8b4e919c98c26e58, State: Running, Role: LEADER
I20250109 08:56:47.823441 20758 consensus_queue.cc:237] T bdeeebda65be4889bf59888b7a7abfec P c5bffb8e4c5e4e7a8b4e919c98c26e58 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:47.830613 20392 catalog_manager.cc:5526] T bdeeebda65be4889bf59888b7a7abfec P c5bffb8e4c5e4e7a8b4e919c98c26e58 reported cstate change: term changed from 2 to 3, leader changed from b17b595d5da94e5dbf96bbef1ef3e668 (127.19.226.194) to c5bffb8e4c5e4e7a8b4e919c98c26e58 (127.19.226.195). New cstate: current_term: 3 leader_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } health_report { overall_health: HEALTHY } } }
I20250109 08:56:47.881646 20838 raft_consensus.cc:988] T 6f383be53b01468f85e731bd2d4367c0 P b17b595d5da94e5dbf96bbef1ef3e668: : Instructing follower ff76b9242fda4d4586c2bb3da829b1ba to start an election
I20250109 08:56:47.882115 20834 raft_consensus.cc:1076] T 6f383be53b01468f85e731bd2d4367c0 P b17b595d5da94e5dbf96bbef1ef3e668 [term 3 LEADER]: Signalling peer ff76b9242fda4d4586c2bb3da829b1ba to start an election
I20250109 08:56:47.883721 20498 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6f383be53b01468f85e731bd2d4367c0"
dest_uuid: "ff76b9242fda4d4586c2bb3da829b1ba"
from {username='slave'} at 127.0.0.1:46196
I20250109 08:56:47.884238 20498 raft_consensus.cc:491] T 6f383be53b01468f85e731bd2d4367c0 P ff76b9242fda4d4586c2bb3da829b1ba [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:56:47.884536 20498 raft_consensus.cc:3054] T 6f383be53b01468f85e731bd2d4367c0 P ff76b9242fda4d4586c2bb3da829b1ba [term 3 FOLLOWER]: Advancing to term 4
I20250109 08:56:47.886554 20838 raft_consensus.cc:988] T bc8cfc045acc40efaa744f955d08ea97 P b17b595d5da94e5dbf96bbef1ef3e668: : Instructing follower ff76b9242fda4d4586c2bb3da829b1ba to start an election
I20250109 08:56:47.887054 20834 raft_consensus.cc:1076] T bc8cfc045acc40efaa744f955d08ea97 P b17b595d5da94e5dbf96bbef1ef3e668 [term 2 LEADER]: Signalling peer ff76b9242fda4d4586c2bb3da829b1ba to start an election
I20250109 08:56:47.889140 20496 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "bc8cfc045acc40efaa744f955d08ea97"
dest_uuid: "ff76b9242fda4d4586c2bb3da829b1ba"
from {username='slave'} at 127.0.0.1:46196
I20250109 08:56:47.889660 20496 raft_consensus.cc:491] T bc8cfc045acc40efaa744f955d08ea97 P ff76b9242fda4d4586c2bb3da829b1ba [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:56:47.890023 20496 raft_consensus.cc:3054] T bc8cfc045acc40efaa744f955d08ea97 P ff76b9242fda4d4586c2bb3da829b1ba [term 2 FOLLOWER]: Advancing to term 3
I20250109 08:56:47.890607 20498 raft_consensus.cc:513] T 6f383be53b01468f85e731bd2d4367c0 P ff76b9242fda4d4586c2bb3da829b1ba [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:47.892861 20498 leader_election.cc:290] T 6f383be53b01468f85e731bd2d4367c0 P ff76b9242fda4d4586c2bb3da829b1ba [CANDIDATE]: Term 4 election: Requested vote from peers b17b595d5da94e5dbf96bbef1ef3e668 (127.19.226.194:42855), c5bffb8e4c5e4e7a8b4e919c98c26e58 (127.19.226.195:45649)
I20250109 08:56:47.893579 20573 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f383be53b01468f85e731bd2d4367c0" candidate_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "b17b595d5da94e5dbf96bbef1ef3e668"
I20250109 08:56:47.893932 20646 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f383be53b01468f85e731bd2d4367c0" candidate_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58"
I20250109 08:56:47.894138 20573 raft_consensus.cc:3049] T 6f383be53b01468f85e731bd2d4367c0 P b17b595d5da94e5dbf96bbef1ef3e668 [term 3 LEADER]: Stepping down as leader of term 3
I20250109 08:56:47.894464 20573 raft_consensus.cc:738] T 6f383be53b01468f85e731bd2d4367c0 P b17b595d5da94e5dbf96bbef1ef3e668 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b17b595d5da94e5dbf96bbef1ef3e668, State: Running, Role: LEADER
I20250109 08:56:47.894474 20646 raft_consensus.cc:3054] T 6f383be53b01468f85e731bd2d4367c0 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 3 FOLLOWER]: Advancing to term 4
I20250109 08:56:47.895215 20573 consensus_queue.cc:260] T 6f383be53b01468f85e731bd2d4367c0 P b17b595d5da94e5dbf96bbef1ef3e668 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:47.896041 20573 raft_consensus.cc:3054] T 6f383be53b01468f85e731bd2d4367c0 P b17b595d5da94e5dbf96bbef1ef3e668 [term 3 FOLLOWER]: Advancing to term 4
I20250109 08:56:47.896822 20496 raft_consensus.cc:513] T bc8cfc045acc40efaa744f955d08ea97 P ff76b9242fda4d4586c2bb3da829b1ba [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:47.899585 20496 leader_election.cc:290] T bc8cfc045acc40efaa744f955d08ea97 P ff76b9242fda4d4586c2bb3da829b1ba [CANDIDATE]: Term 3 election: Requested vote from peers b17b595d5da94e5dbf96bbef1ef3e668 (127.19.226.194:42855), c5bffb8e4c5e4e7a8b4e919c98c26e58 (127.19.226.195:45649)
I20250109 08:56:47.900009 20647 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc8cfc045acc40efaa744f955d08ea97" candidate_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58"
I20250109 08:56:47.900027 20572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc8cfc045acc40efaa744f955d08ea97" candidate_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b17b595d5da94e5dbf96bbef1ef3e668"
I20250109 08:56:47.900820 20572 raft_consensus.cc:3049] T bc8cfc045acc40efaa744f955d08ea97 P b17b595d5da94e5dbf96bbef1ef3e668 [term 2 LEADER]: Stepping down as leader of term 2
I20250109 08:56:47.900852 20647 raft_consensus.cc:3054] T bc8cfc045acc40efaa744f955d08ea97 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 2 FOLLOWER]: Advancing to term 3
I20250109 08:56:47.901225 20572 raft_consensus.cc:738] T bc8cfc045acc40efaa744f955d08ea97 P b17b595d5da94e5dbf96bbef1ef3e668 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b17b595d5da94e5dbf96bbef1ef3e668, State: Running, Role: LEADER
I20250109 08:56:47.901927 20572 consensus_queue.cc:260] T bc8cfc045acc40efaa744f955d08ea97 P b17b595d5da94e5dbf96bbef1ef3e668 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:47.902403 20573 raft_consensus.cc:2463] T 6f383be53b01468f85e731bd2d4367c0 P b17b595d5da94e5dbf96bbef1ef3e668 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff76b9242fda4d4586c2bb3da829b1ba in term 4.
I20250109 08:56:47.903257 20572 raft_consensus.cc:3054] T bc8cfc045acc40efaa744f955d08ea97 P b17b595d5da94e5dbf96bbef1ef3e668 [term 2 FOLLOWER]: Advancing to term 3
I20250109 08:56:47.903411 20646 raft_consensus.cc:2463] T 6f383be53b01468f85e731bd2d4367c0 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff76b9242fda4d4586c2bb3da829b1ba in term 4.
I20250109 08:56:47.903721 20462 leader_election.cc:304] T 6f383be53b01468f85e731bd2d4367c0 P ff76b9242fda4d4586c2bb3da829b1ba [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b17b595d5da94e5dbf96bbef1ef3e668, ff76b9242fda4d4586c2bb3da829b1ba; no voters:
I20250109 08:56:47.906411 20838 raft_consensus.cc:988] T 5378208cf9ad43129b6276fbb65af974 P b17b595d5da94e5dbf96bbef1ef3e668: : Instructing follower c5bffb8e4c5e4e7a8b4e919c98c26e58 to start an election
I20250109 08:56:47.907032 20838 raft_consensus.cc:1076] T 5378208cf9ad43129b6276fbb65af974 P b17b595d5da94e5dbf96bbef1ef3e668 [term 4 LEADER]: Signalling peer c5bffb8e4c5e4e7a8b4e919c98c26e58 to start an election
I20250109 08:56:47.908210 20647 raft_consensus.cc:2463] T bc8cfc045acc40efaa744f955d08ea97 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff76b9242fda4d4586c2bb3da829b1ba in term 3.
I20250109 08:56:47.908867 20645 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "5378208cf9ad43129b6276fbb65af974"
dest_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58"
from {username='slave'} at 127.0.0.1:49928
I20250109 08:56:47.909368 20572 raft_consensus.cc:2463] T bc8cfc045acc40efaa744f955d08ea97 P b17b595d5da94e5dbf96bbef1ef3e668 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff76b9242fda4d4586c2bb3da829b1ba in term 3.
I20250109 08:56:47.909451 20645 raft_consensus.cc:491] T 5378208cf9ad43129b6276fbb65af974 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:56:47.909533 20461 leader_election.cc:304] T bc8cfc045acc40efaa744f955d08ea97 P ff76b9242fda4d4586c2bb3da829b1ba [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c5bffb8e4c5e4e7a8b4e919c98c26e58, ff76b9242fda4d4586c2bb3da829b1ba; no voters:
I20250109 08:56:47.910048 20645 raft_consensus.cc:3054] T 5378208cf9ad43129b6276fbb65af974 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 4 FOLLOWER]: Advancing to term 5
I20250109 08:56:47.915607 20845 raft_consensus.cc:2798] T 6f383be53b01468f85e731bd2d4367c0 P ff76b9242fda4d4586c2bb3da829b1ba [term 4 FOLLOWER]: Leader election won for term 4
I20250109 08:56:47.916548 20846 raft_consensus.cc:2798] T bc8cfc045acc40efaa744f955d08ea97 P ff76b9242fda4d4586c2bb3da829b1ba [term 3 FOLLOWER]: Leader election won for term 3
I20250109 08:56:47.917071 20845 raft_consensus.cc:695] T 6f383be53b01468f85e731bd2d4367c0 P ff76b9242fda4d4586c2bb3da829b1ba [term 4 LEADER]: Becoming Leader. State: Replica: ff76b9242fda4d4586c2bb3da829b1ba, State: Running, Role: LEADER
I20250109 08:56:47.917464 20645 raft_consensus.cc:513] T 5378208cf9ad43129b6276fbb65af974 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:47.918397 20846 raft_consensus.cc:695] T bc8cfc045acc40efaa744f955d08ea97 P ff76b9242fda4d4586c2bb3da829b1ba [term 3 LEADER]: Becoming Leader. State: Replica: ff76b9242fda4d4586c2bb3da829b1ba, State: Running, Role: LEADER
I20250109 08:56:47.918017 20845 consensus_queue.cc:237] T 6f383be53b01468f85e731bd2d4367c0 P ff76b9242fda4d4586c2bb3da829b1ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:47.919363 20846 consensus_queue.cc:237] T bc8cfc045acc40efaa744f955d08ea97 P ff76b9242fda4d4586c2bb3da829b1ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:47.920661 20496 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5378208cf9ad43129b6276fbb65af974" candidate_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "ff76b9242fda4d4586c2bb3da829b1ba"
I20250109 08:56:47.920718 20572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5378208cf9ad43129b6276fbb65af974" candidate_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "b17b595d5da94e5dbf96bbef1ef3e668"
I20250109 08:56:47.921360 20572 raft_consensus.cc:3049] T 5378208cf9ad43129b6276fbb65af974 P b17b595d5da94e5dbf96bbef1ef3e668 [term 4 LEADER]: Stepping down as leader of term 4
I20250109 08:56:47.921331 20496 raft_consensus.cc:3054] T 5378208cf9ad43129b6276fbb65af974 P ff76b9242fda4d4586c2bb3da829b1ba [term 4 FOLLOWER]: Advancing to term 5
I20250109 08:56:47.921775 20572 raft_consensus.cc:738] T 5378208cf9ad43129b6276fbb65af974 P b17b595d5da94e5dbf96bbef1ef3e668 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: b17b595d5da94e5dbf96bbef1ef3e668, State: Running, Role: LEADER
I20250109 08:56:47.922494 20572 consensus_queue.cc:260] T 5378208cf9ad43129b6276fbb65af974 P b17b595d5da94e5dbf96bbef1ef3e668 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:47.923554 20572 raft_consensus.cc:3054] T 5378208cf9ad43129b6276fbb65af974 P b17b595d5da94e5dbf96bbef1ef3e668 [term 4 FOLLOWER]: Advancing to term 5
I20250109 08:56:47.927703 20496 raft_consensus.cc:2463] T 5378208cf9ad43129b6276fbb65af974 P ff76b9242fda4d4586c2bb3da829b1ba [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c5bffb8e4c5e4e7a8b4e919c98c26e58 in term 5.
I20250109 08:56:47.929075 20613 leader_election.cc:304] T 5378208cf9ad43129b6276fbb65af974 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c5bffb8e4c5e4e7a8b4e919c98c26e58, ff76b9242fda4d4586c2bb3da829b1ba; no voters:
I20250109 08:56:47.929872 20758 raft_consensus.cc:2798] T 5378208cf9ad43129b6276fbb65af974 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 5 FOLLOWER]: Leader election won for term 5
I20250109 08:56:47.930073 20572 raft_consensus.cc:2463] T 5378208cf9ad43129b6276fbb65af974 P b17b595d5da94e5dbf96bbef1ef3e668 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c5bffb8e4c5e4e7a8b4e919c98c26e58 in term 5.
I20250109 08:56:47.930305 20758 raft_consensus.cc:695] T 5378208cf9ad43129b6276fbb65af974 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 5 LEADER]: Becoming Leader. State: Replica: c5bffb8e4c5e4e7a8b4e919c98c26e58, State: Running, Role: LEADER
I20250109 08:56:47.930485 20645 leader_election.cc:290] T 5378208cf9ad43129b6276fbb65af974 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [CANDIDATE]: Term 5 election: Requested vote from peers ff76b9242fda4d4586c2bb3da829b1ba (127.19.226.193:42763), b17b595d5da94e5dbf96bbef1ef3e668 (127.19.226.194:42855)
I20250109 08:56:47.931403 20758 consensus_queue.cc:237] T 5378208cf9ad43129b6276fbb65af974 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:47.934247 20392 catalog_manager.cc:5526] T 6f383be53b01468f85e731bd2d4367c0 P ff76b9242fda4d4586c2bb3da829b1ba reported cstate change: term changed from 3 to 4, leader changed from b17b595d5da94e5dbf96bbef1ef3e668 (127.19.226.194) to ff76b9242fda4d4586c2bb3da829b1ba (127.19.226.193). New cstate: current_term: 4 leader_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } health_report { overall_health: UNKNOWN } } }
I20250109 08:56:47.939189 20393 catalog_manager.cc:5526] T 5378208cf9ad43129b6276fbb65af974 P c5bffb8e4c5e4e7a8b4e919c98c26e58 reported cstate change: term changed from 4 to 5, leader changed from b17b595d5da94e5dbf96bbef1ef3e668 (127.19.226.194) to c5bffb8e4c5e4e7a8b4e919c98c26e58 (127.19.226.195). New cstate: current_term: 5 leader_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } health_report { overall_health: HEALTHY } } }
I20250109 08:56:47.948319 20392 catalog_manager.cc:5526] T bc8cfc045acc40efaa744f955d08ea97 P ff76b9242fda4d4586c2bb3da829b1ba reported cstate change: term changed from 2 to 3, leader changed from b17b595d5da94e5dbf96bbef1ef3e668 (127.19.226.194) to ff76b9242fda4d4586c2bb3da829b1ba (127.19.226.193). New cstate: current_term: 3 leader_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } health_report { overall_health: UNKNOWN } } }
I20250109 08:56:47.959388 20838 raft_consensus.cc:988] T cfbc2774b91c40d59dc80655451f5bdd P b17b595d5da94e5dbf96bbef1ef3e668: : Instructing follower ff76b9242fda4d4586c2bb3da829b1ba to start an election
I20250109 08:56:47.959909 20834 raft_consensus.cc:1076] T cfbc2774b91c40d59dc80655451f5bdd P b17b595d5da94e5dbf96bbef1ef3e668 [term 2 LEADER]: Signalling peer ff76b9242fda4d4586c2bb3da829b1ba to start an election
I20250109 08:56:47.961184 20496 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "cfbc2774b91c40d59dc80655451f5bdd"
dest_uuid: "ff76b9242fda4d4586c2bb3da829b1ba"
from {username='slave'} at 127.0.0.1:46196
I20250109 08:56:47.961566 20496 raft_consensus.cc:491] T cfbc2774b91c40d59dc80655451f5bdd P ff76b9242fda4d4586c2bb3da829b1ba [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:56:47.961800 20496 raft_consensus.cc:3054] T cfbc2774b91c40d59dc80655451f5bdd P ff76b9242fda4d4586c2bb3da829b1ba [term 2 FOLLOWER]: Advancing to term 3
I20250109 08:56:47.965492 20496 raft_consensus.cc:513] T cfbc2774b91c40d59dc80655451f5bdd P ff76b9242fda4d4586c2bb3da829b1ba [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:47.967197 20496 leader_election.cc:290] T cfbc2774b91c40d59dc80655451f5bdd P ff76b9242fda4d4586c2bb3da829b1ba [CANDIDATE]: Term 3 election: Requested vote from peers b17b595d5da94e5dbf96bbef1ef3e668 (127.19.226.194:42855), c5bffb8e4c5e4e7a8b4e919c98c26e58 (127.19.226.195:45649)
I20250109 08:56:47.967559 20572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfbc2774b91c40d59dc80655451f5bdd" candidate_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b17b595d5da94e5dbf96bbef1ef3e668"
I20250109 08:56:47.968060 20572 raft_consensus.cc:3049] T cfbc2774b91c40d59dc80655451f5bdd P b17b595d5da94e5dbf96bbef1ef3e668 [term 2 LEADER]: Stepping down as leader of term 2
I20250109 08:56:47.968258 20572 raft_consensus.cc:738] T cfbc2774b91c40d59dc80655451f5bdd P b17b595d5da94e5dbf96bbef1ef3e668 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b17b595d5da94e5dbf96bbef1ef3e668, State: Running, Role: LEADER
I20250109 08:56:47.968160 20645 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfbc2774b91c40d59dc80655451f5bdd" candidate_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58"
I20250109 08:56:47.968765 20645 raft_consensus.cc:3054] T cfbc2774b91c40d59dc80655451f5bdd P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 2 FOLLOWER]: Advancing to term 3
I20250109 08:56:47.968912 20572 consensus_queue.cc:260] T cfbc2774b91c40d59dc80655451f5bdd P b17b595d5da94e5dbf96bbef1ef3e668 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:47.969772 20572 raft_consensus.cc:3054] T cfbc2774b91c40d59dc80655451f5bdd P b17b595d5da94e5dbf96bbef1ef3e668 [term 2 FOLLOWER]: Advancing to term 3
I20250109 08:56:47.973093 20645 raft_consensus.cc:2463] T cfbc2774b91c40d59dc80655451f5bdd P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff76b9242fda4d4586c2bb3da829b1ba in term 3.
I20250109 08:56:47.973599 20572 raft_consensus.cc:2463] T cfbc2774b91c40d59dc80655451f5bdd P b17b595d5da94e5dbf96bbef1ef3e668 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff76b9242fda4d4586c2bb3da829b1ba in term 3.
I20250109 08:56:47.974100 20461 leader_election.cc:304] T cfbc2774b91c40d59dc80655451f5bdd P ff76b9242fda4d4586c2bb3da829b1ba [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c5bffb8e4c5e4e7a8b4e919c98c26e58, ff76b9242fda4d4586c2bb3da829b1ba; no voters:
I20250109 08:56:47.975030 20846 raft_consensus.cc:2798] T cfbc2774b91c40d59dc80655451f5bdd P ff76b9242fda4d4586c2bb3da829b1ba [term 3 FOLLOWER]: Leader election won for term 3
I20250109 08:56:47.975428 20846 raft_consensus.cc:695] T cfbc2774b91c40d59dc80655451f5bdd P ff76b9242fda4d4586c2bb3da829b1ba [term 3 LEADER]: Becoming Leader. State: Replica: ff76b9242fda4d4586c2bb3da829b1ba, State: Running, Role: LEADER
I20250109 08:56:47.976251 20846 consensus_queue.cc:237] T cfbc2774b91c40d59dc80655451f5bdd P ff76b9242fda4d4586c2bb3da829b1ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } }
I20250109 08:56:47.982172 20392 catalog_manager.cc:5526] T cfbc2774b91c40d59dc80655451f5bdd P ff76b9242fda4d4586c2bb3da829b1ba reported cstate change: term changed from 2 to 3, leader changed from b17b595d5da94e5dbf96bbef1ef3e668 (127.19.226.194) to ff76b9242fda4d4586c2bb3da829b1ba (127.19.226.193). New cstate: current_term: 3 leader_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 } health_report { overall_health: UNKNOWN } } }
I20250109 08:56:48.334988 20496 raft_consensus.cc:1270] T bdeeebda65be4889bf59888b7a7abfec P ff76b9242fda4d4586c2bb3da829b1ba [term 3 FOLLOWER]: Refusing update from remote peer c5bffb8e4c5e4e7a8b4e919c98c26e58: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20250109 08:56:48.336138 20758 consensus_queue.cc:1035] T bdeeebda65be4889bf59888b7a7abfec P c5bffb8e4c5e4e7a8b4e919c98c26e58 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:56:48.336326 20645 raft_consensus.cc:1270] T 6f383be53b01468f85e731bd2d4367c0 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 4 FOLLOWER]: Refusing update from remote peer ff76b9242fda4d4586c2bb3da829b1ba: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20250109 08:56:48.337600 20846 consensus_queue.cc:1035] T 6f383be53b01468f85e731bd2d4367c0 P ff76b9242fda4d4586c2bb3da829b1ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250109 08:56:48.350370 20647 raft_consensus.cc:1270] T bc8cfc045acc40efaa744f955d08ea97 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 3 FOLLOWER]: Refusing update from remote peer ff76b9242fda4d4586c2bb3da829b1ba: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20250109 08:56:48.352274 20846 consensus_queue.cc:1035] T bc8cfc045acc40efaa744f955d08ea97 P ff76b9242fda4d4586c2bb3da829b1ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:56:48.354657 20572 raft_consensus.cc:1270] T 6f383be53b01468f85e731bd2d4367c0 P b17b595d5da94e5dbf96bbef1ef3e668 [term 4 FOLLOWER]: Refusing update from remote peer ff76b9242fda4d4586c2bb3da829b1ba: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20250109 08:56:48.354879 20573 raft_consensus.cc:1270] T bdeeebda65be4889bf59888b7a7abfec P b17b595d5da94e5dbf96bbef1ef3e668 [term 3 FOLLOWER]: Refusing update from remote peer c5bffb8e4c5e4e7a8b4e919c98c26e58: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20250109 08:56:48.356050 20846 consensus_queue.cc:1035] T 6f383be53b01468f85e731bd2d4367c0 P ff76b9242fda4d4586c2bb3da829b1ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250109 08:56:48.356305 20758 consensus_queue.cc:1035] T bdeeebda65be4889bf59888b7a7abfec P c5bffb8e4c5e4e7a8b4e919c98c26e58 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:56:48.382797 20572 raft_consensus.cc:1270] T bc8cfc045acc40efaa744f955d08ea97 P b17b595d5da94e5dbf96bbef1ef3e668 [term 3 FOLLOWER]: Refusing update from remote peer ff76b9242fda4d4586c2bb3da829b1ba: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20250109 08:56:48.383852 20845 consensus_queue.cc:1035] T bc8cfc045acc40efaa744f955d08ea97 P ff76b9242fda4d4586c2bb3da829b1ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:56:48.439241 20645 raft_consensus.cc:1270] T cfbc2774b91c40d59dc80655451f5bdd P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 3 FOLLOWER]: Refusing update from remote peer ff76b9242fda4d4586c2bb3da829b1ba: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20250109 08:56:48.440462 20845 consensus_queue.cc:1035] T cfbc2774b91c40d59dc80655451f5bdd P ff76b9242fda4d4586c2bb3da829b1ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5bffb8e4c5e4e7a8b4e919c98c26e58" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 45649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:56:48.445441 20572 raft_consensus.cc:1270] T 5378208cf9ad43129b6276fbb65af974 P b17b595d5da94e5dbf96bbef1ef3e668 [term 5 FOLLOWER]: Refusing update from remote peer c5bffb8e4c5e4e7a8b4e919c98c26e58: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20250109 08:56:48.446923 20859 consensus_queue.cc:1035] T 5378208cf9ad43129b6276fbb65af974 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20250109 08:56:48.449131 20573 raft_consensus.cc:1270] T cfbc2774b91c40d59dc80655451f5bdd P b17b595d5da94e5dbf96bbef1ef3e668 [term 3 FOLLOWER]: Refusing update from remote peer ff76b9242fda4d4586c2bb3da829b1ba: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20250109 08:56:48.450593 20848 consensus_queue.cc:1035] T cfbc2774b91c40d59dc80655451f5bdd P ff76b9242fda4d4586c2bb3da829b1ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "b17b595d5da94e5dbf96bbef1ef3e668" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 42855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20250109 08:56:48.461992 20496 raft_consensus.cc:1270] T 5378208cf9ad43129b6276fbb65af974 P ff76b9242fda4d4586c2bb3da829b1ba [term 5 FOLLOWER]: Refusing update from remote peer c5bffb8e4c5e4e7a8b4e919c98c26e58: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20250109 08:56:48.463770 20839 consensus_queue.cc:1035] T 5378208cf9ad43129b6276fbb65af974 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff76b9242fda4d4586c2bb3da829b1ba" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250109 08:56:48.639829 20363 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250109 08:56:48.644569 20363 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250109 08:56:48.644961 20363 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250109 08:56:49.645747 20363 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250109 08:56:49.650674 20363 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250109 08:56:49.651106 20363 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250109 08:56:50.651880 20363 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250109 08:56:50.656723 20363 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250109 08:56:50.657160 20363 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250109 08:56:51.657907 20363 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250109 08:56:51.663033 20363 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250109 08:56:51.663435 20363 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250109 08:56:52.664268 20363 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250109 08:56:52.670166 20363 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250109 08:56:52.670707 20363 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250109 08:56:53.671590 20363 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250109 08:56:53.676348 20363 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250109 08:56:53.676782 20363 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250109 08:56:54.677495 20363 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250109 08:56:54.682435 20363 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250109 08:56:54.682849 20363 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250109 08:56:55.683576 20363 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250109 08:56:55.688396 20363 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250109 08:56:55.688804 20363 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250109 08:56:56.689569 20363 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250109 08:56:56.694288 20363 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250109 08:56:56.694682 20363 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250109 08:56:57.719555 20363 auto_leader_rebalancer-test.cc:314] The leader distribution is
I20250109 08:56:57.720089 20363 tablet_server.cc:178] TabletServer@127.19.226.193:0 shutting down...
I20250109 08:56:57.743077 20363 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250109 08:56:57.743752 20363 tablet_replica.cc:331] T 74c881b69c3946429070c729ec66b0e4 P ff76b9242fda4d4586c2bb3da829b1ba: stopping tablet replica
I20250109 08:56:57.744683 20363 raft_consensus.cc:2238] T 74c881b69c3946429070c729ec66b0e4 P ff76b9242fda4d4586c2bb3da829b1ba [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:56:57.745221 20363 raft_consensus.cc:2267] T 74c881b69c3946429070c729ec66b0e4 P ff76b9242fda4d4586c2bb3da829b1ba [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:56:57.748091 20363 tablet_replica.cc:331] T cfbc2774b91c40d59dc80655451f5bdd P ff76b9242fda4d4586c2bb3da829b1ba: stopping tablet replica
W20250109 08:56:57.748266 20613 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20250109 08:56:57.748929 20363 raft_consensus.cc:2238] T cfbc2774b91c40d59dc80655451f5bdd P ff76b9242fda4d4586c2bb3da829b1ba [term 3 LEADER]: Raft consensus shutting down.
I20250109 08:56:57.750654 20363 raft_consensus.cc:2267] T cfbc2774b91c40d59dc80655451f5bdd P ff76b9242fda4d4586c2bb3da829b1ba [term 3 FOLLOWER]: Raft consensus is shut down!
W20250109 08:56:57.751065 20613 consensus_peers.cc:487] T 5378208cf9ad43129b6276fbb65af974 P c5bffb8e4c5e4e7a8b4e919c98c26e58 -> Peer ff76b9242fda4d4586c2bb3da829b1ba (127.19.226.193:42763): Couldn't send request to peer ff76b9242fda4d4586c2bb3da829b1ba. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:56:57.752714 20363 tablet_replica.cc:331] T 6f383be53b01468f85e731bd2d4367c0 P ff76b9242fda4d4586c2bb3da829b1ba: stopping tablet replica
I20250109 08:56:57.753432 20363 raft_consensus.cc:2238] T 6f383be53b01468f85e731bd2d4367c0 P ff76b9242fda4d4586c2bb3da829b1ba [term 4 LEADER]: Raft consensus shutting down.
I20250109 08:56:57.754240 20363 raft_consensus.cc:2267] T 6f383be53b01468f85e731bd2d4367c0 P ff76b9242fda4d4586c2bb3da829b1ba [term 4 FOLLOWER]: Raft consensus is shut down!
W20250109 08:56:57.754848 20613 consensus_peers.cc:487] T db23d411a3a14ed69a1db0b2c8912807 P c5bffb8e4c5e4e7a8b4e919c98c26e58 -> Peer ff76b9242fda4d4586c2bb3da829b1ba (127.19.226.193:42763): Couldn't send request to peer ff76b9242fda4d4586c2bb3da829b1ba. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:56:57.755915 20363 tablet_replica.cc:331] T 5378208cf9ad43129b6276fbb65af974 P ff76b9242fda4d4586c2bb3da829b1ba: stopping tablet replica
I20250109 08:56:57.756454 20363 raft_consensus.cc:2238] T 5378208cf9ad43129b6276fbb65af974 P ff76b9242fda4d4586c2bb3da829b1ba [term 5 FOLLOWER]: Raft consensus shutting down.
I20250109 08:56:57.756786 20363 raft_consensus.cc:2267] T 5378208cf9ad43129b6276fbb65af974 P ff76b9242fda4d4586c2bb3da829b1ba [term 5 FOLLOWER]: Raft consensus is shut down!
I20250109 08:56:57.758198 20363 tablet_replica.cc:331] T 4173445b18a24674ad20fe4b7e17c697 P ff76b9242fda4d4586c2bb3da829b1ba: stopping tablet replica
I20250109 08:56:57.758605 20363 raft_consensus.cc:2238] T 4173445b18a24674ad20fe4b7e17c697 P ff76b9242fda4d4586c2bb3da829b1ba [term 4 FOLLOWER]: Raft consensus shutting down.
I20250109 08:56:57.758941 20363 raft_consensus.cc:2267] T 4173445b18a24674ad20fe4b7e17c697 P ff76b9242fda4d4586c2bb3da829b1ba [term 4 FOLLOWER]: Raft consensus is shut down!
I20250109 08:56:57.760406 20363 tablet_replica.cc:331] T bdeeebda65be4889bf59888b7a7abfec P ff76b9242fda4d4586c2bb3da829b1ba: stopping tablet replica
I20250109 08:56:57.760890 20363 raft_consensus.cc:2238] T bdeeebda65be4889bf59888b7a7abfec P ff76b9242fda4d4586c2bb3da829b1ba [term 3 FOLLOWER]: Raft consensus shutting down.
I20250109 08:56:57.761366 20363 raft_consensus.cc:2267] T bdeeebda65be4889bf59888b7a7abfec P ff76b9242fda4d4586c2bb3da829b1ba [term 3 FOLLOWER]: Raft consensus is shut down!
I20250109 08:56:57.763235 20363 tablet_replica.cc:331] T db23d411a3a14ed69a1db0b2c8912807 P ff76b9242fda4d4586c2bb3da829b1ba: stopping tablet replica
I20250109 08:56:57.763651 20363 raft_consensus.cc:2238] T db23d411a3a14ed69a1db0b2c8912807 P ff76b9242fda4d4586c2bb3da829b1ba [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:56:57.763986 20363 raft_consensus.cc:2267] T db23d411a3a14ed69a1db0b2c8912807 P ff76b9242fda4d4586c2bb3da829b1ba [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:56:57.765409 20363 tablet_replica.cc:331] T bc8cfc045acc40efaa744f955d08ea97 P ff76b9242fda4d4586c2bb3da829b1ba: stopping tablet replica
I20250109 08:56:57.765796 20363 raft_consensus.cc:2238] T bc8cfc045acc40efaa744f955d08ea97 P ff76b9242fda4d4586c2bb3da829b1ba [term 3 LEADER]: Raft consensus shutting down.
I20250109 08:56:57.766387 20363 raft_consensus.cc:2267] T bc8cfc045acc40efaa744f955d08ea97 P ff76b9242fda4d4586c2bb3da829b1ba [term 3 FOLLOWER]: Raft consensus is shut down!
I20250109 08:56:57.767820 20363 tablet_replica.cc:331] T 6b1d1214e23d47ff8ec02197a45d8484 P ff76b9242fda4d4586c2bb3da829b1ba: stopping tablet replica
I20250109 08:56:57.768216 20363 raft_consensus.cc:2238] T 6b1d1214e23d47ff8ec02197a45d8484 P ff76b9242fda4d4586c2bb3da829b1ba [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:56:57.768553 20363 raft_consensus.cc:2267] T 6b1d1214e23d47ff8ec02197a45d8484 P ff76b9242fda4d4586c2bb3da829b1ba [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:56:57.856992 20363 tablet_server.cc:195] TabletServer@127.19.226.193:0 shutdown complete.
W20250109 08:56:57.858542 20539 consensus_peers.cc:487] T 4173445b18a24674ad20fe4b7e17c697 P b17b595d5da94e5dbf96bbef1ef3e668 -> Peer ff76b9242fda4d4586c2bb3da829b1ba (127.19.226.193:42763): Couldn't send request to peer ff76b9242fda4d4586c2bb3da829b1ba. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.193:42763: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20250109 08:56:57.859127 20539 consensus_peers.cc:487] T 74c881b69c3946429070c729ec66b0e4 P b17b595d5da94e5dbf96bbef1ef3e668 -> Peer ff76b9242fda4d4586c2bb3da829b1ba (127.19.226.193:42763): Couldn't send request to peer ff76b9242fda4d4586c2bb3da829b1ba. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.193:42763: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250109 08:56:57.883131 20363 tablet_server.cc:178] TabletServer@127.19.226.194:0 shutting down...
I20250109 08:56:57.907534 20363 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250109 08:56:57.908226 20363 tablet_replica.cc:331] T 74c881b69c3946429070c729ec66b0e4 P b17b595d5da94e5dbf96bbef1ef3e668: stopping tablet replica
I20250109 08:56:57.908831 20363 raft_consensus.cc:2238] T 74c881b69c3946429070c729ec66b0e4 P b17b595d5da94e5dbf96bbef1ef3e668 [term 2 LEADER]: Raft consensus shutting down.
I20250109 08:56:57.909756 20363 raft_consensus.cc:2267] T 74c881b69c3946429070c729ec66b0e4 P b17b595d5da94e5dbf96bbef1ef3e668 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:56:57.911583 20363 tablet_replica.cc:331] T 5378208cf9ad43129b6276fbb65af974 P b17b595d5da94e5dbf96bbef1ef3e668: stopping tablet replica
I20250109 08:56:57.912091 20363 raft_consensus.cc:2238] T 5378208cf9ad43129b6276fbb65af974 P b17b595d5da94e5dbf96bbef1ef3e668 [term 5 FOLLOWER]: Raft consensus shutting down.
I20250109 08:56:57.912596 20363 raft_consensus.cc:2267] T 5378208cf9ad43129b6276fbb65af974 P b17b595d5da94e5dbf96bbef1ef3e668 [term 5 FOLLOWER]: Raft consensus is shut down!
I20250109 08:56:57.914139 20363 tablet_replica.cc:331] T 4173445b18a24674ad20fe4b7e17c697 P b17b595d5da94e5dbf96bbef1ef3e668: stopping tablet replica
I20250109 08:56:57.914620 20363 raft_consensus.cc:2238] T 4173445b18a24674ad20fe4b7e17c697 P b17b595d5da94e5dbf96bbef1ef3e668 [term 4 LEADER]: Raft consensus shutting down.
I20250109 08:56:57.915427 20363 raft_consensus.cc:2267] T 4173445b18a24674ad20fe4b7e17c697 P b17b595d5da94e5dbf96bbef1ef3e668 [term 4 FOLLOWER]: Raft consensus is shut down!
I20250109 08:56:57.917037 20363 tablet_replica.cc:331] T bdeeebda65be4889bf59888b7a7abfec P b17b595d5da94e5dbf96bbef1ef3e668: stopping tablet replica
I20250109 08:56:57.917539 20363 raft_consensus.cc:2238] T bdeeebda65be4889bf59888b7a7abfec P b17b595d5da94e5dbf96bbef1ef3e668 [term 3 FOLLOWER]: Raft consensus shutting down.
I20250109 08:56:57.917935 20363 raft_consensus.cc:2267] T bdeeebda65be4889bf59888b7a7abfec P b17b595d5da94e5dbf96bbef1ef3e668 [term 3 FOLLOWER]: Raft consensus is shut down!
I20250109 08:56:57.919413 20363 tablet_replica.cc:331] T db23d411a3a14ed69a1db0b2c8912807 P b17b595d5da94e5dbf96bbef1ef3e668: stopping tablet replica
I20250109 08:56:57.919842 20363 raft_consensus.cc:2238] T db23d411a3a14ed69a1db0b2c8912807 P b17b595d5da94e5dbf96bbef1ef3e668 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:56:57.920229 20363 raft_consensus.cc:2267] T db23d411a3a14ed69a1db0b2c8912807 P b17b595d5da94e5dbf96bbef1ef3e668 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:56:57.921689 20363 tablet_replica.cc:331] T bc8cfc045acc40efaa744f955d08ea97 P b17b595d5da94e5dbf96bbef1ef3e668: stopping tablet replica
I20250109 08:56:57.922108 20363 raft_consensus.cc:2238] T bc8cfc045acc40efaa744f955d08ea97 P b17b595d5da94e5dbf96bbef1ef3e668 [term 3 FOLLOWER]: Raft consensus shutting down.
I20250109 08:56:57.922539 20363 raft_consensus.cc:2267] T bc8cfc045acc40efaa744f955d08ea97 P b17b595d5da94e5dbf96bbef1ef3e668 [term 3 FOLLOWER]: Raft consensus is shut down!
I20250109 08:56:57.924088 20363 tablet_replica.cc:331] T cfbc2774b91c40d59dc80655451f5bdd P b17b595d5da94e5dbf96bbef1ef3e668: stopping tablet replica
I20250109 08:56:57.924567 20363 raft_consensus.cc:2238] T cfbc2774b91c40d59dc80655451f5bdd P b17b595d5da94e5dbf96bbef1ef3e668 [term 3 FOLLOWER]: Raft consensus shutting down.
I20250109 08:56:57.924993 20363 raft_consensus.cc:2267] T cfbc2774b91c40d59dc80655451f5bdd P b17b595d5da94e5dbf96bbef1ef3e668 [term 3 FOLLOWER]: Raft consensus is shut down!
I20250109 08:56:57.926451 20363 tablet_replica.cc:331] T 6f383be53b01468f85e731bd2d4367c0 P b17b595d5da94e5dbf96bbef1ef3e668: stopping tablet replica
I20250109 08:56:57.926842 20363 raft_consensus.cc:2238] T 6f383be53b01468f85e731bd2d4367c0 P b17b595d5da94e5dbf96bbef1ef3e668 [term 4 FOLLOWER]: Raft consensus shutting down.
I20250109 08:56:57.927273 20363 raft_consensus.cc:2267] T 6f383be53b01468f85e731bd2d4367c0 P b17b595d5da94e5dbf96bbef1ef3e668 [term 4 FOLLOWER]: Raft consensus is shut down!
I20250109 08:56:57.928742 20363 tablet_replica.cc:331] T 6b1d1214e23d47ff8ec02197a45d8484 P b17b595d5da94e5dbf96bbef1ef3e668: stopping tablet replica
I20250109 08:56:57.929131 20363 raft_consensus.cc:2238] T 6b1d1214e23d47ff8ec02197a45d8484 P b17b595d5da94e5dbf96bbef1ef3e668 [term 2 LEADER]: Raft consensus shutting down.
I20250109 08:56:57.929916 20363 raft_consensus.cc:2267] T 6b1d1214e23d47ff8ec02197a45d8484 P b17b595d5da94e5dbf96bbef1ef3e668 [term 2 FOLLOWER]: Raft consensus is shut down!
W20250109 08:56:57.946170 20613 consensus_peers.cc:487] T bdeeebda65be4889bf59888b7a7abfec P c5bffb8e4c5e4e7a8b4e919c98c26e58 -> Peer ff76b9242fda4d4586c2bb3da829b1ba (127.19.226.193:42763): Couldn't send request to peer ff76b9242fda4d4586c2bb3da829b1ba. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.193:42763: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250109 08:56:57.972532 20363 tablet_server.cc:195] TabletServer@127.19.226.194:0 shutdown complete.
I20250109 08:56:57.989734 20363 tablet_server.cc:178] TabletServer@127.19.226.195:0 shutting down...
I20250109 08:56:58.012619 20363 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250109 08:56:58.013509 20363 tablet_replica.cc:331] T 6f383be53b01468f85e731bd2d4367c0 P c5bffb8e4c5e4e7a8b4e919c98c26e58: stopping tablet replica
I20250109 08:56:58.014587 20363 raft_consensus.cc:2238] T 6f383be53b01468f85e731bd2d4367c0 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 4 FOLLOWER]: Raft consensus shutting down.
I20250109 08:56:58.015460 20363 raft_consensus.cc:2267] T 6f383be53b01468f85e731bd2d4367c0 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 4 FOLLOWER]: Raft consensus is shut down!
W20250109 08:56:58.016868 20611 consensus_peers.cc:487] T 5378208cf9ad43129b6276fbb65af974 P c5bffb8e4c5e4e7a8b4e919c98c26e58 -> Peer b17b595d5da94e5dbf96bbef1ef3e668 (127.19.226.194:42855): Couldn't send request to peer b17b595d5da94e5dbf96bbef1ef3e668. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.194:42855: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250109 08:56:58.017832 20363 tablet_replica.cc:331] T 74c881b69c3946429070c729ec66b0e4 P c5bffb8e4c5e4e7a8b4e919c98c26e58: stopping tablet replica
I20250109 08:56:58.018606 20363 raft_consensus.cc:2238] T 74c881b69c3946429070c729ec66b0e4 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:56:58.019065 20363 raft_consensus.cc:2267] T 74c881b69c3946429070c729ec66b0e4 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:56:58.021548 20363 tablet_replica.cc:331] T cfbc2774b91c40d59dc80655451f5bdd P c5bffb8e4c5e4e7a8b4e919c98c26e58: stopping tablet replica
I20250109 08:56:58.022066 20363 raft_consensus.cc:2238] T cfbc2774b91c40d59dc80655451f5bdd P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 3 FOLLOWER]: Raft consensus shutting down.
I20250109 08:56:58.022516 20363 raft_consensus.cc:2267] T cfbc2774b91c40d59dc80655451f5bdd P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 3 FOLLOWER]: Raft consensus is shut down!
I20250109 08:56:58.024092 20363 tablet_replica.cc:331] T bdeeebda65be4889bf59888b7a7abfec P c5bffb8e4c5e4e7a8b4e919c98c26e58: stopping tablet replica
I20250109 08:56:58.024657 20363 raft_consensus.cc:2238] T bdeeebda65be4889bf59888b7a7abfec P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 3 LEADER]: Raft consensus shutting down.
I20250109 08:56:58.025403 20363 raft_consensus.cc:2267] T bdeeebda65be4889bf59888b7a7abfec P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 3 FOLLOWER]: Raft consensus is shut down!
I20250109 08:56:58.026924 20363 tablet_replica.cc:331] T db23d411a3a14ed69a1db0b2c8912807 P c5bffb8e4c5e4e7a8b4e919c98c26e58: stopping tablet replica
I20250109 08:56:58.027390 20363 raft_consensus.cc:2238] T db23d411a3a14ed69a1db0b2c8912807 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 1 LEADER]: Raft consensus shutting down.
I20250109 08:56:58.028203 20363 raft_consensus.cc:2267] T db23d411a3a14ed69a1db0b2c8912807 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:56:58.029732 20363 tablet_replica.cc:331] T bc8cfc045acc40efaa744f955d08ea97 P c5bffb8e4c5e4e7a8b4e919c98c26e58: stopping tablet replica
I20250109 08:56:58.030176 20363 raft_consensus.cc:2238] T bc8cfc045acc40efaa744f955d08ea97 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 3 FOLLOWER]: Raft consensus shutting down.
I20250109 08:56:58.030615 20363 raft_consensus.cc:2267] T bc8cfc045acc40efaa744f955d08ea97 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 3 FOLLOWER]: Raft consensus is shut down!
I20250109 08:56:58.032099 20363 tablet_replica.cc:331] T 6b1d1214e23d47ff8ec02197a45d8484 P c5bffb8e4c5e4e7a8b4e919c98c26e58: stopping tablet replica
I20250109 08:56:58.032591 20363 raft_consensus.cc:2238] T 6b1d1214e23d47ff8ec02197a45d8484 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:56:58.032959 20363 raft_consensus.cc:2267] T 6b1d1214e23d47ff8ec02197a45d8484 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:56:58.034487 20363 tablet_replica.cc:331] T 5378208cf9ad43129b6276fbb65af974 P c5bffb8e4c5e4e7a8b4e919c98c26e58: stopping tablet replica
I20250109 08:56:58.034922 20363 raft_consensus.cc:2238] T 5378208cf9ad43129b6276fbb65af974 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 5 LEADER]: Raft consensus shutting down.
I20250109 08:56:58.035753 20363 raft_consensus.cc:2267] T 5378208cf9ad43129b6276fbb65af974 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 5 FOLLOWER]: Raft consensus is shut down!
I20250109 08:56:58.037281 20363 tablet_replica.cc:331] T 4173445b18a24674ad20fe4b7e17c697 P c5bffb8e4c5e4e7a8b4e919c98c26e58: stopping tablet replica
I20250109 08:56:58.037704 20363 raft_consensus.cc:2238] T 4173445b18a24674ad20fe4b7e17c697 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 4 FOLLOWER]: Raft consensus shutting down.
I20250109 08:56:58.038094 20363 raft_consensus.cc:2267] T 4173445b18a24674ad20fe4b7e17c697 P c5bffb8e4c5e4e7a8b4e919c98c26e58 [term 4 FOLLOWER]: Raft consensus is shut down!
I20250109 08:56:58.063741 20363 tablet_server.cc:195] TabletServer@127.19.226.195:0 shutdown complete.
I20250109 08:56:58.081286 20363 master.cc:537] Master@127.19.226.254:39769 shutting down...
I20250109 08:56:58.268123 20363 raft_consensus.cc:2238] T 00000000000000000000000000000000 P 756e7d8e818247aa9406c4a737856ce4 [term 1 LEADER]: Raft consensus shutting down.
I20250109 08:56:58.268762 20363 raft_consensus.cc:2267] T 00000000000000000000000000000000 P 756e7d8e818247aa9406c4a737856ce4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:56:58.269048 20363 tablet_replica.cc:331] T 00000000000000000000000000000000 P 756e7d8e818247aa9406c4a737856ce4: stopping tablet replica
I20250109 08:56:58.287962 20363 master.cc:559] Master@127.19.226.254:39769 shutdown complete.
b17b595d5da94e5dbf96bbef1ef3e668 4
c5bffb8e4c5e4e7a8b4e919c98c26e58 1
ff76b9242fda4d4586c2bb3da829b1ba 4
b17b595d5da94e5dbf96bbef1ef3e668 3
c5bffb8e4c5e4e7a8b4e919c98c26e58 3
ff76b9242fda4d4586c2bb3da829b1ba 3
b17b595d5da94e5dbf96bbef1ef3e668 8
c5bffb8e4c5e4e7a8b4e919c98c26e58 1
ff76b9242fda4d4586c2bb3da829b1ba 0
b17b595d5da94e5dbf96bbef1ef3e668 3
c5bffb8e4c5e4e7a8b4e919c98c26e58 3
ff76b9242fda4d4586c2bb3da829b1ba 3
[ OK ] LeaderRebalancerTest.FunctionalTestForDivided (30628 ms)
[ RUN ] LeaderRebalancerTest.FunctionalTestForNotDivided
I20250109 08:56:58.327615 20363 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.19.226.254:37551
I20250109 08:56:58.328786 20363 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250109 08:56:58.334080 20884 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250109 08:56:58.335281 20885 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250109 08:56:58.335601 20887 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250109 08:56:58.336810 20363 server_base.cc:1034] running on GCE node
I20250109 08:56:58.337706 20363 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250109 08:56:58.337891 20363 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250109 08:56:58.338035 20363 hybrid_clock.cc:648] HybridClock initialized: now 1736413018338019 us; error 0 us; skew 500 ppm
I20250109 08:56:58.338541 20363 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250109 08:56:58.340974 20363 webserver.cc:458] Webserver started at http://127.19.226.254:45623/ using document root <none> and password file <none>
I20250109 08:56:58.341452 20363 fs_manager.cc:362] Metadata directory not provided
I20250109 08:56:58.341624 20363 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250109 08:56:58.341881 20363 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250109 08:56:58.343014 20363 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1736412987538469-20363-0/minicluster-data/master-0-root/instance:
uuid: "8c3697782dab47b383365dc945f94c68"
format_stamp: "Formatted at 2025-01-09 08:56:58 on dist-test-slave-g8ds"
I20250109 08:56:58.347255 20363 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20250109 08:56:58.350371 20892 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250109 08:56:58.351047 20363 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20250109 08:56:58.351303 20363 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1736412987538469-20363-0/minicluster-data/master-0-root
uuid: "8c3697782dab47b383365dc945f94c68"
format_stamp: "Formatted at 2025-01-09 08:56:58 on dist-test-slave-g8ds"
I20250109 08:56:58.351565 20363 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1736412987538469-20363-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1736412987538469-20363-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1736412987538469-20363-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250109 08:56:58.389740 20363 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250109 08:56:58.390945 20363 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250109 08:56:58.425925 20363 rpc_server.cc:307] RPC server started. Bound to: 127.19.226.254:37551
I20250109 08:56:58.426016 20943 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.226.254:37551 every 8 connection(s)
I20250109 08:56:58.429579 20944 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:56:58.438884 20944 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c3697782dab47b383365dc945f94c68: Bootstrap starting.
I20250109 08:56:58.443118 20944 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8c3697782dab47b383365dc945f94c68: Neither blocks nor log segments found. Creating new log.
I20250109 08:56:58.447041 20944 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c3697782dab47b383365dc945f94c68: No bootstrap required, opened a new log
I20250109 08:56:58.449007 20944 raft_consensus.cc:357] T 00000000000000000000000000000000 P 8c3697782dab47b383365dc945f94c68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c3697782dab47b383365dc945f94c68" member_type: VOTER }
I20250109 08:56:58.449445 20944 raft_consensus.cc:383] T 00000000000000000000000000000000 P 8c3697782dab47b383365dc945f94c68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:56:58.449664 20944 raft_consensus.cc:738] T 00000000000000000000000000000000 P 8c3697782dab47b383365dc945f94c68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c3697782dab47b383365dc945f94c68, State: Initialized, Role: FOLLOWER
I20250109 08:56:58.450162 20944 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8c3697782dab47b383365dc945f94c68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c3697782dab47b383365dc945f94c68" member_type: VOTER }
I20250109 08:56:58.450587 20944 raft_consensus.cc:397] T 00000000000000000000000000000000 P 8c3697782dab47b383365dc945f94c68 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250109 08:56:58.450794 20944 raft_consensus.cc:491] T 00000000000000000000000000000000 P 8c3697782dab47b383365dc945f94c68 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250109 08:56:58.451035 20944 raft_consensus.cc:3054] T 00000000000000000000000000000000 P 8c3697782dab47b383365dc945f94c68 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:56:58.454874 20944 raft_consensus.cc:513] T 00000000000000000000000000000000 P 8c3697782dab47b383365dc945f94c68 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c3697782dab47b383365dc945f94c68" member_type: VOTER }
I20250109 08:56:58.455343 20944 leader_election.cc:304] T 00000000000000000000000000000000 P 8c3697782dab47b383365dc945f94c68 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8c3697782dab47b383365dc945f94c68; no voters:
I20250109 08:56:58.456359 20944 leader_election.cc:290] T 00000000000000000000000000000000 P 8c3697782dab47b383365dc945f94c68 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250109 08:56:58.456748 20947 raft_consensus.cc:2798] T 00000000000000000000000000000000 P 8c3697782dab47b383365dc945f94c68 [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:56:58.457916 20947 raft_consensus.cc:695] T 00000000000000000000000000000000 P 8c3697782dab47b383365dc945f94c68 [term 1 LEADER]: Becoming Leader. State: Replica: 8c3697782dab47b383365dc945f94c68, State: Running, Role: LEADER
I20250109 08:56:58.458667 20947 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8c3697782dab47b383365dc945f94c68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c3697782dab47b383365dc945f94c68" member_type: VOTER }
I20250109 08:56:58.459113 20944 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8c3697782dab47b383365dc945f94c68 [sys.catalog]: configured and running, proceeding with master startup.
I20250109 08:56:58.464648 20948 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c3697782dab47b383365dc945f94c68 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8c3697782dab47b383365dc945f94c68" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c3697782dab47b383365dc945f94c68" member_type: VOTER } }
I20250109 08:56:58.465200 20948 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c3697782dab47b383365dc945f94c68 [sys.catalog]: This master's current role is: LEADER
I20250109 08:56:58.465013 20949 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c3697782dab47b383365dc945f94c68 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8c3697782dab47b383365dc945f94c68. Latest consensus state: current_term: 1 leader_uuid: "8c3697782dab47b383365dc945f94c68" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c3697782dab47b383365dc945f94c68" member_type: VOTER } }
I20250109 08:56:58.465546 20949 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c3697782dab47b383365dc945f94c68 [sys.catalog]: This master's current role is: LEADER
I20250109 08:56:58.466976 20955 catalog_manager.cc:1476] Loading table and tablet metadata into memory...
I20250109 08:56:58.474110 20955 catalog_manager.cc:1485] Initializing Kudu cluster ID...
I20250109 08:56:58.474992 20363 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20250109 08:56:58.482111 20955 catalog_manager.cc:1348] Generated new cluster ID: 59a795d9621a4b89b92c5e707a1758f4
I20250109 08:56:58.482466 20955 catalog_manager.cc:1496] Initializing Kudu internal certificate authority...
I20250109 08:56:58.498857 20955 catalog_manager.cc:1371] Generated new certificate authority record
I20250109 08:56:58.500062 20955 catalog_manager.cc:1505] Loading token signing keys...
I20250109 08:56:58.511580 20955 catalog_manager.cc:5899] T 00000000000000000000000000000000 P 8c3697782dab47b383365dc945f94c68: Generated new TSK 0
I20250109 08:56:58.512140 20955 catalog_manager.cc:1515] Initializing in-progress tserver states...
I20250109 08:56:58.541358 20363 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250109 08:56:58.547170 20965 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250109 08:56:58.548520 20966 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250109 08:56:58.549592 20968 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250109 08:56:58.549934 20363 server_base.cc:1034] running on GCE node
I20250109 08:56:58.550752 20363 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250109 08:56:58.550944 20363 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250109 08:56:58.551087 20363 hybrid_clock.cc:648] HybridClock initialized: now 1736413018551070 us; error 0 us; skew 500 ppm
I20250109 08:56:58.551564 20363 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250109 08:56:58.553892 20363 webserver.cc:458] Webserver started at http://127.19.226.193:34243/ using document root <none> and password file <none>
I20250109 08:56:58.554368 20363 fs_manager.cc:362] Metadata directory not provided
I20250109 08:56:58.554540 20363 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250109 08:56:58.554777 20363 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250109 08:56:58.555809 20363 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1736412987538469-20363-0/minicluster-data/ts-0-root/instance:
uuid: "b6684bfcd7404fe68d8cb196153412d5"
format_stamp: "Formatted at 2025-01-09 08:56:58 on dist-test-slave-g8ds"
I20250109 08:56:58.560154 20363 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20250109 08:56:58.563311 20973 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250109 08:56:58.564054 20363 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20250109 08:56:58.564298 20363 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1736412987538469-20363-0/minicluster-data/ts-0-root
uuid: "b6684bfcd7404fe68d8cb196153412d5"
format_stamp: "Formatted at 2025-01-09 08:56:58 on dist-test-slave-g8ds"
I20250109 08:56:58.564589 20363 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1736412987538469-20363-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1736412987538469-20363-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1736412987538469-20363-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250109 08:56:58.575464 20363 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250109 08:56:58.576685 20363 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250109 08:56:58.578065 20363 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250109 08:56:58.580220 20363 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250109 08:56:58.580428 20363 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20250109 08:56:58.580655 20363 ts_tablet_manager.cc:610] Registered 0 tablets
I20250109 08:56:58.580790 20363 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250109 08:56:58.616812 20363 rpc_server.cc:307] RPC server started. Bound to: 127.19.226.193:34209
I20250109 08:56:58.616899 21035 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.226.193:34209 every 8 connection(s)
I20250109 08:56:58.621053 20363 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250109 08:56:58.628149 21040 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250109 08:56:58.630271 21041 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250109 08:56:58.631399 21043 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250109 08:56:58.632644 20363 server_base.cc:1034] running on GCE node
I20250109 08:56:58.633433 21036 heartbeater.cc:346] Connected to a master server at 127.19.226.254:37551
I20250109 08:56:58.633589 20363 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20250109 08:56:58.633731 21036 heartbeater.cc:463] Registering TS with master...
W20250109 08:56:58.633824 20363 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250109 08:56:58.634063 20363 hybrid_clock.cc:648] HybridClock initialized: now 1736413018634043 us; error 0 us; skew 500 ppm
I20250109 08:56:58.634512 21036 heartbeater.cc:510] Master 127.19.226.254:37551 requested a full tablet report, sending...
I20250109 08:56:58.634649 20363 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250109 08:56:58.636582 20909 ts_manager.cc:194] Registered new tserver with Master: b6684bfcd7404fe68d8cb196153412d5 (127.19.226.193:34209)
I20250109 08:56:58.637427 20363 webserver.cc:458] Webserver started at http://127.19.226.194:46457/ using document root <none> and password file <none>
I20250109 08:56:58.638065 20363 fs_manager.cc:362] Metadata directory not provided
I20250109 08:56:58.638310 20363 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250109 08:56:58.638600 20363 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250109 08:56:58.638875 20909 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:38546
I20250109 08:56:58.640020 20363 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1736412987538469-20363-0/minicluster-data/ts-1-root/instance:
uuid: "6e2837e083814e77b16db43b99d985ab"
format_stamp: "Formatted at 2025-01-09 08:56:58 on dist-test-slave-g8ds"
I20250109 08:56:58.644279 20363 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20250109 08:56:58.647425 21048 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250109 08:56:58.648133 20363 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20250109 08:56:58.648451 20363 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1736412987538469-20363-0/minicluster-data/ts-1-root
uuid: "6e2837e083814e77b16db43b99d985ab"
format_stamp: "Formatted at 2025-01-09 08:56:58 on dist-test-slave-g8ds"
I20250109 08:56:58.648715 20363 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1736412987538469-20363-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1736412987538469-20363-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1736412987538469-20363-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250109 08:56:58.659619 20363 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250109 08:56:58.660811 20363 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250109 08:56:58.662307 20363 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250109 08:56:58.664424 20363 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250109 08:56:58.664618 20363 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250109 08:56:58.664826 20363 ts_tablet_manager.cc:610] Registered 0 tablets
I20250109 08:56:58.664969 20363 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250109 08:56:58.700973 20363 rpc_server.cc:307] RPC server started. Bound to: 127.19.226.194:44083
I20250109 08:56:58.701062 21110 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.226.194:44083 every 8 connection(s)
I20250109 08:56:58.705277 20363 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250109 08:56:58.711930 21114 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250109 08:56:58.713176 21115 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250109 08:56:58.715952 20363 server_base.cc:1034] running on GCE node
W20250109 08:56:58.716580 21118 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250109 08:56:58.716800 21111 heartbeater.cc:346] Connected to a master server at 127.19.226.254:37551
I20250109 08:56:58.717204 21111 heartbeater.cc:463] Registering TS with master...
I20250109 08:56:58.717535 20363 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250109 08:56:58.717751 20363 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250109 08:56:58.717911 20363 hybrid_clock.cc:648] HybridClock initialized: now 1736413018717893 us; error 0 us; skew 500 ppm
I20250109 08:56:58.717866 21111 heartbeater.cc:510] Master 127.19.226.254:37551 requested a full tablet report, sending...
I20250109 08:56:58.718550 20363 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250109 08:56:58.719759 20909 ts_manager.cc:194] Registered new tserver with Master: 6e2837e083814e77b16db43b99d985ab (127.19.226.194:44083)
I20250109 08:56:58.721040 20909 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:36378
I20250109 08:56:58.721457 20363 webserver.cc:458] Webserver started at http://127.19.226.195:38411/ using document root <none> and password file <none>
I20250109 08:56:58.722029 20363 fs_manager.cc:362] Metadata directory not provided
I20250109 08:56:58.722200 20363 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250109 08:56:58.722468 20363 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250109 08:56:58.723548 20363 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1736412987538469-20363-0/minicluster-data/ts-2-root/instance:
uuid: "ead0b707c1014770b25c5a0db0454830"
format_stamp: "Formatted at 2025-01-09 08:56:58 on dist-test-slave-g8ds"
I20250109 08:56:58.727797 20363 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20250109 08:56:58.730904 21122 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250109 08:56:58.731686 20363 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20250109 08:56:58.731946 20363 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1736412987538469-20363-0/minicluster-data/ts-2-root
uuid: "ead0b707c1014770b25c5a0db0454830"
format_stamp: "Formatted at 2025-01-09 08:56:58 on dist-test-slave-g8ds"
I20250109 08:56:58.732203 20363 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1736412987538469-20363-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1736412987538469-20363-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1736412987538469-20363-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250109 08:56:58.772262 20363 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250109 08:56:58.773442 20363 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250109 08:56:58.774850 20363 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250109 08:56:58.776953 20363 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250109 08:56:58.777148 20363 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250109 08:56:58.777367 20363 ts_tablet_manager.cc:610] Registered 0 tablets
I20250109 08:56:58.777521 20363 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250109 08:56:58.814388 20363 rpc_server.cc:307] RPC server started. Bound to: 127.19.226.195:35749
I20250109 08:56:58.814471 21184 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.226.195:35749 every 8 connection(s)
I20250109 08:56:58.826828 21185 heartbeater.cc:346] Connected to a master server at 127.19.226.254:37551
I20250109 08:56:58.827234 21185 heartbeater.cc:463] Registering TS with master...
I20250109 08:56:58.827908 21185 heartbeater.cc:510] Master 127.19.226.254:37551 requested a full tablet report, sending...
I20250109 08:56:58.829638 20909 ts_manager.cc:194] Registered new tserver with Master: ead0b707c1014770b25c5a0db0454830 (127.19.226.195:35749)
I20250109 08:56:58.830622 20363 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.01327727s
I20250109 08:56:58.830988 20909 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:36380
I20250109 08:56:58.833220 20363 test_util.cc:274] Using random seed: 690733356
I20250109 08:56:58.854260 20909 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36386:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250109 08:56:58.857000 20909 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250109 08:56:58.921593 20999 tablet_service.cc:1467] Processing CreateTablet for tablet ec3e3f891e3c4a879077b9ee897e5d52 (DEFAULT_TABLE table=test-workload [id=dc620248ce2e48bf9fddfd949fb1adde]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20250109 08:56:58.923034 20999 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec3e3f891e3c4a879077b9ee897e5d52. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:56:58.921602 20998 tablet_service.cc:1467] Processing CreateTablet for tablet 163310a12b44475ca3727662dad194a6 (DEFAULT_TABLE table=test-workload [id=dc620248ce2e48bf9fddfd949fb1adde]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20250109 08:56:58.924052 20998 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 163310a12b44475ca3727662dad194a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:56:58.924455 20996 tablet_service.cc:1467] Processing CreateTablet for tablet af13c7179cc24b258e516eceb2b0cac0 (DEFAULT_TABLE table=test-workload [id=dc620248ce2e48bf9fddfd949fb1adde]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20250109 08:56:58.925135 21000 tablet_service.cc:1467] Processing CreateTablet for tablet 42af5f76a37645b5a3865739d54a6cb3 (DEFAULT_TABLE table=test-workload [id=dc620248ce2e48bf9fddfd949fb1adde]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20250109 08:56:58.925654 20996 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af13c7179cc24b258e516eceb2b0cac0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:56:58.926831 21001 tablet_service.cc:1467] Processing CreateTablet for tablet 502ebc25110a4c04a71b8b2e8eea07d4 (DEFAULT_TABLE table=test-workload [id=dc620248ce2e48bf9fddfd949fb1adde]), partition=RANGE (key) PARTITION VALUES < 214748364
I20250109 08:56:58.928001 21001 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 502ebc25110a4c04a71b8b2e8eea07d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:56:58.922761 20997 tablet_service.cc:1467] Processing CreateTablet for tablet 948e27b3034d44338e2b7d74f96374f7 (DEFAULT_TABLE table=test-workload [id=dc620248ce2e48bf9fddfd949fb1adde]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20250109 08:56:58.929433 20995 tablet_service.cc:1467] Processing CreateTablet for tablet 05148030cdd443048fe5899b03fd92fe (DEFAULT_TABLE table=test-workload [id=dc620248ce2e48bf9fddfd949fb1adde]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20250109 08:56:58.929898 20997 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 948e27b3034d44338e2b7d74f96374f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:56:58.930610 20995 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 05148030cdd443048fe5899b03fd92fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:56:58.933111 21000 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 42af5f76a37645b5a3865739d54a6cb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:56:58.937567 20994 tablet_service.cc:1467] Processing CreateTablet for tablet 166b00d1332d4f39b57ae9f58cb7fc5f (DEFAULT_TABLE table=test-workload [id=dc620248ce2e48bf9fddfd949fb1adde]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20250109 08:56:58.938778 20994 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 166b00d1332d4f39b57ae9f58cb7fc5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:56:58.942845 20993 tablet_service.cc:1467] Processing CreateTablet for tablet 53b47173fb6a4a358c8ad9c2df0f6f18 (DEFAULT_TABLE table=test-workload [id=dc620248ce2e48bf9fddfd949fb1adde]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20250109 08:56:58.943495 20992 tablet_service.cc:1467] Processing CreateTablet for tablet cb177a6fcf4144e2ae1a560eb627e276 (DEFAULT_TABLE table=test-workload [id=dc620248ce2e48bf9fddfd949fb1adde]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20250109 08:56:58.944797 20992 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb177a6fcf4144e2ae1a560eb627e276. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:56:58.963240 20993 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53b47173fb6a4a358c8ad9c2df0f6f18. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:56:58.987349 21150 tablet_service.cc:1467] Processing CreateTablet for tablet 502ebc25110a4c04a71b8b2e8eea07d4 (DEFAULT_TABLE table=test-workload [id=dc620248ce2e48bf9fddfd949fb1adde]), partition=RANGE (key) PARTITION VALUES < 214748364
I20250109 08:56:58.989559 21145 tablet_service.cc:1467] Processing CreateTablet for tablet af13c7179cc24b258e516eceb2b0cac0 (DEFAULT_TABLE table=test-workload [id=dc620248ce2e48bf9fddfd949fb1adde]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20250109 08:56:58.989802 21148 tablet_service.cc:1467] Processing CreateTablet for tablet ec3e3f891e3c4a879077b9ee897e5d52 (DEFAULT_TABLE table=test-workload [id=dc620248ce2e48bf9fddfd949fb1adde]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20250109 08:56:58.990406 21144 tablet_service.cc:1467] Processing CreateTablet for tablet 05148030cdd443048fe5899b03fd92fe (DEFAULT_TABLE table=test-workload [id=dc620248ce2e48bf9fddfd949fb1adde]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20250109 08:56:58.991096 21149 tablet_service.cc:1467] Processing CreateTablet for tablet 42af5f76a37645b5a3865739d54a6cb3 (DEFAULT_TABLE table=test-workload [id=dc620248ce2e48bf9fddfd949fb1adde]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20250109 08:56:58.991958 21147 tablet_service.cc:1467] Processing CreateTablet for tablet 163310a12b44475ca3727662dad194a6 (DEFAULT_TABLE table=test-workload [id=dc620248ce2e48bf9fddfd949fb1adde]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20250109 08:56:58.994237 21146 tablet_service.cc:1467] Processing CreateTablet for tablet 948e27b3034d44338e2b7d74f96374f7 (DEFAULT_TABLE table=test-workload [id=dc620248ce2e48bf9fddfd949fb1adde]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20250109 08:56:58.994350 21142 tablet_service.cc:1467] Processing CreateTablet for tablet 53b47173fb6a4a358c8ad9c2df0f6f18 (DEFAULT_TABLE table=test-workload [id=dc620248ce2e48bf9fddfd949fb1adde]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20250109 08:56:58.993057 21143 tablet_service.cc:1467] Processing CreateTablet for tablet 166b00d1332d4f39b57ae9f58cb7fc5f (DEFAULT_TABLE table=test-workload [id=dc620248ce2e48bf9fddfd949fb1adde]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20250109 08:56:58.995122 21141 tablet_service.cc:1467] Processing CreateTablet for tablet cb177a6fcf4144e2ae1a560eb627e276 (DEFAULT_TABLE table=test-workload [id=dc620248ce2e48bf9fddfd949fb1adde]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20250109 08:56:58.996758 21199 tablet_bootstrap.cc:492] T 502ebc25110a4c04a71b8b2e8eea07d4 P b6684bfcd7404fe68d8cb196153412d5: Bootstrap starting.
I20250109 08:56:58.997131 21150 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 502ebc25110a4c04a71b8b2e8eea07d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:56:58.997680 21149 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 42af5f76a37645b5a3865739d54a6cb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:56:58.998247 21146 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 948e27b3034d44338e2b7d74f96374f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:56:58.998889 21141 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb177a6fcf4144e2ae1a560eb627e276. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:56:58.999424 21147 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 163310a12b44475ca3727662dad194a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:56:59.003093 21199 tablet_bootstrap.cc:654] T 502ebc25110a4c04a71b8b2e8eea07d4 P b6684bfcd7404fe68d8cb196153412d5: Neither blocks nor log segments found. Creating new log.
I20250109 08:56:59.004705 21143 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 166b00d1332d4f39b57ae9f58cb7fc5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:56:59.008679 21142 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53b47173fb6a4a358c8ad9c2df0f6f18. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:56:59.012717 21148 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec3e3f891e3c4a879077b9ee897e5d52. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:56:59.016705 21145 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af13c7179cc24b258e516eceb2b0cac0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:56:59.017736 21144 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 05148030cdd443048fe5899b03fd92fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:56:59.019366 21199 tablet_bootstrap.cc:492] T 502ebc25110a4c04a71b8b2e8eea07d4 P b6684bfcd7404fe68d8cb196153412d5: No bootstrap required, opened a new log
I20250109 08:56:59.019829 21199 ts_tablet_manager.cc:1397] T 502ebc25110a4c04a71b8b2e8eea07d4 P b6684bfcd7404fe68d8cb196153412d5: Time spent bootstrapping tablet: real 0.023s user 0.011s sys 0.010s
I20250109 08:56:59.023221 21199 raft_consensus.cc:357] T 502ebc25110a4c04a71b8b2e8eea07d4 P b6684bfcd7404fe68d8cb196153412d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } }
I20250109 08:56:59.025449 21199 raft_consensus.cc:383] T 502ebc25110a4c04a71b8b2e8eea07d4 P b6684bfcd7404fe68d8cb196153412d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:56:59.025816 21199 raft_consensus.cc:738] T 502ebc25110a4c04a71b8b2e8eea07d4 P b6684bfcd7404fe68d8cb196153412d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6684bfcd7404fe68d8cb196153412d5, State: Initialized, Role: FOLLOWER
I20250109 08:56:59.026645 21199 consensus_queue.cc:260] T 502ebc25110a4c04a71b8b2e8eea07d4 P b6684bfcd7404fe68d8cb196153412d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } }
I20250109 08:56:59.035909 21199 ts_tablet_manager.cc:1428] T 502ebc25110a4c04a71b8b2e8eea07d4 P b6684bfcd7404fe68d8cb196153412d5: Time spent starting tablet: real 0.016s user 0.006s sys 0.009s
I20250109 08:56:59.036739 21036 heartbeater.cc:502] Master 127.19.226.254:37551 was elected leader, sending a full tablet report...
I20250109 08:56:59.039654 21199 tablet_bootstrap.cc:492] T 948e27b3034d44338e2b7d74f96374f7 P b6684bfcd7404fe68d8cb196153412d5: Bootstrap starting.
I20250109 08:56:59.045022 21199 tablet_bootstrap.cc:654] T 948e27b3034d44338e2b7d74f96374f7 P b6684bfcd7404fe68d8cb196153412d5: Neither blocks nor log segments found. Creating new log.
I20250109 08:56:59.072260 21199 tablet_bootstrap.cc:492] T 948e27b3034d44338e2b7d74f96374f7 P b6684bfcd7404fe68d8cb196153412d5: No bootstrap required, opened a new log
I20250109 08:56:59.072750 21199 ts_tablet_manager.cc:1397] T 948e27b3034d44338e2b7d74f96374f7 P b6684bfcd7404fe68d8cb196153412d5: Time spent bootstrapping tablet: real 0.033s user 0.010s sys 0.017s
I20250109 08:56:59.075299 21199 raft_consensus.cc:357] T 948e27b3034d44338e2b7d74f96374f7 P b6684bfcd7404fe68d8cb196153412d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:56:59.076031 21199 raft_consensus.cc:383] T 948e27b3034d44338e2b7d74f96374f7 P b6684bfcd7404fe68d8cb196153412d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:56:59.076311 21199 raft_consensus.cc:738] T 948e27b3034d44338e2b7d74f96374f7 P b6684bfcd7404fe68d8cb196153412d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6684bfcd7404fe68d8cb196153412d5, State: Initialized, Role: FOLLOWER
I20250109 08:56:59.077076 21199 consensus_queue.cc:260] T 948e27b3034d44338e2b7d74f96374f7 P b6684bfcd7404fe68d8cb196153412d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:56:59.084255 21199 ts_tablet_manager.cc:1428] T 948e27b3034d44338e2b7d74f96374f7 P b6684bfcd7404fe68d8cb196153412d5: Time spent starting tablet: real 0.011s user 0.005s sys 0.000s
I20250109 08:56:59.085134 21199 tablet_bootstrap.cc:492] T 166b00d1332d4f39b57ae9f58cb7fc5f P b6684bfcd7404fe68d8cb196153412d5: Bootstrap starting.
I20250109 08:56:59.091142 21199 tablet_bootstrap.cc:654] T 166b00d1332d4f39b57ae9f58cb7fc5f P b6684bfcd7404fe68d8cb196153412d5: Neither blocks nor log segments found. Creating new log.
I20250109 08:56:59.101814 21201 tablet_bootstrap.cc:492] T 42af5f76a37645b5a3865739d54a6cb3 P ead0b707c1014770b25c5a0db0454830: Bootstrap starting.
I20250109 08:56:59.103381 21076 tablet_service.cc:1467] Processing CreateTablet for tablet 502ebc25110a4c04a71b8b2e8eea07d4 (DEFAULT_TABLE table=test-workload [id=dc620248ce2e48bf9fddfd949fb1adde]), partition=RANGE (key) PARTITION VALUES < 214748364
I20250109 08:56:59.105628 21075 tablet_service.cc:1467] Processing CreateTablet for tablet 42af5f76a37645b5a3865739d54a6cb3 (DEFAULT_TABLE table=test-workload [id=dc620248ce2e48bf9fddfd949fb1adde]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20250109 08:56:59.106595 21201 tablet_bootstrap.cc:654] T 42af5f76a37645b5a3865739d54a6cb3 P ead0b707c1014770b25c5a0db0454830: Neither blocks nor log segments found. Creating new log.
I20250109 08:56:59.107110 21075 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 42af5f76a37645b5a3865739d54a6cb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:56:59.109627 21074 tablet_service.cc:1467] Processing CreateTablet for tablet ec3e3f891e3c4a879077b9ee897e5d52 (DEFAULT_TABLE table=test-workload [id=dc620248ce2e48bf9fddfd949fb1adde]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20250109 08:56:59.111068 21074 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec3e3f891e3c4a879077b9ee897e5d52. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:56:59.112968 21070 tablet_service.cc:1467] Processing CreateTablet for tablet 05148030cdd443048fe5899b03fd92fe (DEFAULT_TABLE table=test-workload [id=dc620248ce2e48bf9fddfd949fb1adde]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20250109 08:56:59.113942 21201 tablet_bootstrap.cc:492] T 42af5f76a37645b5a3865739d54a6cb3 P ead0b707c1014770b25c5a0db0454830: No bootstrap required, opened a new log
I20250109 08:56:59.114235 21070 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 05148030cdd443048fe5899b03fd92fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:56:59.114508 21201 ts_tablet_manager.cc:1397] T 42af5f76a37645b5a3865739d54a6cb3 P ead0b707c1014770b25c5a0db0454830: Time spent bootstrapping tablet: real 0.013s user 0.012s sys 0.000s
I20250109 08:56:59.117322 21201 raft_consensus.cc:357] T 42af5f76a37645b5a3865739d54a6cb3 P ead0b707c1014770b25c5a0db0454830 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:56:59.118166 21201 raft_consensus.cc:383] T 42af5f76a37645b5a3865739d54a6cb3 P ead0b707c1014770b25c5a0db0454830 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:56:59.118520 21201 raft_consensus.cc:738] T 42af5f76a37645b5a3865739d54a6cb3 P ead0b707c1014770b25c5a0db0454830 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ead0b707c1014770b25c5a0db0454830, State: Initialized, Role: FOLLOWER
I20250109 08:56:59.119201 21072 tablet_service.cc:1467] Processing CreateTablet for tablet 948e27b3034d44338e2b7d74f96374f7 (DEFAULT_TABLE table=test-workload [id=dc620248ce2e48bf9fddfd949fb1adde]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20250109 08:56:59.119299 21201 consensus_queue.cc:260] T 42af5f76a37645b5a3865739d54a6cb3 P ead0b707c1014770b25c5a0db0454830 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:56:59.121587 21073 tablet_service.cc:1467] Processing CreateTablet for tablet 163310a12b44475ca3727662dad194a6 (DEFAULT_TABLE table=test-workload [id=dc620248ce2e48bf9fddfd949fb1adde]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20250109 08:56:59.117537 21071 tablet_service.cc:1467] Processing CreateTablet for tablet af13c7179cc24b258e516eceb2b0cac0 (DEFAULT_TABLE table=test-workload [id=dc620248ce2e48bf9fddfd949fb1adde]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20250109 08:56:59.126199 21072 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 948e27b3034d44338e2b7d74f96374f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:56:59.128739 21071 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af13c7179cc24b258e516eceb2b0cac0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:56:59.130825 21073 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 163310a12b44475ca3727662dad194a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:56:59.131453 21185 heartbeater.cc:502] Master 127.19.226.254:37551 was elected leader, sending a full tablet report...
I20250109 08:56:59.132140 21201 ts_tablet_manager.cc:1428] T 42af5f76a37645b5a3865739d54a6cb3 P ead0b707c1014770b25c5a0db0454830: Time spent starting tablet: real 0.017s user 0.012s sys 0.004s
I20250109 08:56:59.132316 21207 tablet_bootstrap.cc:492] T ec3e3f891e3c4a879077b9ee897e5d52 P 6e2837e083814e77b16db43b99d985ab: Bootstrap starting.
I20250109 08:56:59.133349 21201 tablet_bootstrap.cc:492] T cb177a6fcf4144e2ae1a560eb627e276 P ead0b707c1014770b25c5a0db0454830: Bootstrap starting.
I20250109 08:56:59.134699 21076 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 502ebc25110a4c04a71b8b2e8eea07d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:56:59.139194 21201 tablet_bootstrap.cc:654] T cb177a6fcf4144e2ae1a560eb627e276 P ead0b707c1014770b25c5a0db0454830: Neither blocks nor log segments found. Creating new log.
I20250109 08:56:59.141579 21069 tablet_service.cc:1467] Processing CreateTablet for tablet 166b00d1332d4f39b57ae9f58cb7fc5f (DEFAULT_TABLE table=test-workload [id=dc620248ce2e48bf9fddfd949fb1adde]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20250109 08:56:59.142943 21069 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 166b00d1332d4f39b57ae9f58cb7fc5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:56:59.143597 21207 tablet_bootstrap.cc:654] T ec3e3f891e3c4a879077b9ee897e5d52 P 6e2837e083814e77b16db43b99d985ab: Neither blocks nor log segments found. Creating new log.
I20250109 08:56:59.145913 21074 tablet_service.cc:1467] Processing CreateTablet for tablet 53b47173fb6a4a358c8ad9c2df0f6f18 (DEFAULT_TABLE table=test-workload [id=dc620248ce2e48bf9fddfd949fb1adde]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20250109 08:56:59.146327 21068 tablet_service.cc:1467] Processing CreateTablet for tablet cb177a6fcf4144e2ae1a560eb627e276 (DEFAULT_TABLE table=test-workload [id=dc620248ce2e48bf9fddfd949fb1adde]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20250109 08:56:59.147177 21074 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53b47173fb6a4a358c8ad9c2df0f6f18. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:56:59.147953 21068 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb177a6fcf4144e2ae1a560eb627e276. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:56:59.168157 21207 tablet_bootstrap.cc:492] T ec3e3f891e3c4a879077b9ee897e5d52 P 6e2837e083814e77b16db43b99d985ab: No bootstrap required, opened a new log
I20250109 08:56:59.168730 21207 ts_tablet_manager.cc:1397] T ec3e3f891e3c4a879077b9ee897e5d52 P 6e2837e083814e77b16db43b99d985ab: Time spent bootstrapping tablet: real 0.037s user 0.007s sys 0.006s
I20250109 08:56:59.171865 21207 raft_consensus.cc:357] T ec3e3f891e3c4a879077b9ee897e5d52 P 6e2837e083814e77b16db43b99d985ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:56:59.175520 21207 raft_consensus.cc:383] T ec3e3f891e3c4a879077b9ee897e5d52 P 6e2837e083814e77b16db43b99d985ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:56:59.175873 21207 raft_consensus.cc:738] T ec3e3f891e3c4a879077b9ee897e5d52 P 6e2837e083814e77b16db43b99d985ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e2837e083814e77b16db43b99d985ab, State: Initialized, Role: FOLLOWER
I20250109 08:56:59.176775 21207 consensus_queue.cc:260] T ec3e3f891e3c4a879077b9ee897e5d52 P 6e2837e083814e77b16db43b99d985ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:56:59.185420 21111 heartbeater.cc:502] Master 127.19.226.254:37551 was elected leader, sending a full tablet report...
I20250109 08:56:59.186525 21207 ts_tablet_manager.cc:1428] T ec3e3f891e3c4a879077b9ee897e5d52 P 6e2837e083814e77b16db43b99d985ab: Time spent starting tablet: real 0.017s user 0.007s sys 0.005s
I20250109 08:56:59.189548 21202 raft_consensus.cc:491] T 948e27b3034d44338e2b7d74f96374f7 P b6684bfcd7404fe68d8cb196153412d5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:56:59.190068 21202 raft_consensus.cc:513] T 948e27b3034d44338e2b7d74f96374f7 P b6684bfcd7404fe68d8cb196153412d5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:56:59.194139 21202 leader_election.cc:290] T 948e27b3034d44338e2b7d74f96374f7 P b6684bfcd7404fe68d8cb196153412d5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e2837e083814e77b16db43b99d985ab (127.19.226.194:44083), ead0b707c1014770b25c5a0db0454830 (127.19.226.195:35749)
I20250109 08:56:59.200423 21207 tablet_bootstrap.cc:492] T cb177a6fcf4144e2ae1a560eb627e276 P 6e2837e083814e77b16db43b99d985ab: Bootstrap starting.
I20250109 08:56:59.262491 21207 tablet_bootstrap.cc:654] T cb177a6fcf4144e2ae1a560eb627e276 P 6e2837e083814e77b16db43b99d985ab: Neither blocks nor log segments found. Creating new log.
I20250109 08:56:59.300695 21202 raft_consensus.cc:491] T 502ebc25110a4c04a71b8b2e8eea07d4 P b6684bfcd7404fe68d8cb196153412d5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:56:59.303284 21202 raft_consensus.cc:513] T 502ebc25110a4c04a71b8b2e8eea07d4 P b6684bfcd7404fe68d8cb196153412d5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } }
I20250109 08:56:59.305262 21202 leader_election.cc:290] T 502ebc25110a4c04a71b8b2e8eea07d4 P b6684bfcd7404fe68d8cb196153412d5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ead0b707c1014770b25c5a0db0454830 (127.19.226.195:35749), 6e2837e083814e77b16db43b99d985ab (127.19.226.194:44083)
I20250109 08:56:59.315464 21160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "948e27b3034d44338e2b7d74f96374f7" candidate_uuid: "b6684bfcd7404fe68d8cb196153412d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ead0b707c1014770b25c5a0db0454830" is_pre_election: true
I20250109 08:56:59.315460 21159 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "502ebc25110a4c04a71b8b2e8eea07d4" candidate_uuid: "b6684bfcd7404fe68d8cb196153412d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ead0b707c1014770b25c5a0db0454830" is_pre_election: true
W20250109 08:56:59.317582 20974 leader_election.cc:343] T 948e27b3034d44338e2b7d74f96374f7 P b6684bfcd7404fe68d8cb196153412d5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ead0b707c1014770b25c5a0db0454830 (127.19.226.195:35749): Illegal state: must be running to vote when last-logged opid is not known
W20250109 08:56:59.318280 20974 leader_election.cc:343] T 502ebc25110a4c04a71b8b2e8eea07d4 P b6684bfcd7404fe68d8cb196153412d5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ead0b707c1014770b25c5a0db0454830 (127.19.226.195:35749): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:56:59.320770 21207 tablet_bootstrap.cc:492] T cb177a6fcf4144e2ae1a560eb627e276 P 6e2837e083814e77b16db43b99d985ab: No bootstrap required, opened a new log
I20250109 08:56:59.320799 21199 tablet_bootstrap.cc:492] T 166b00d1332d4f39b57ae9f58cb7fc5f P b6684bfcd7404fe68d8cb196153412d5: No bootstrap required, opened a new log
I20250109 08:56:59.321563 21199 ts_tablet_manager.cc:1397] T 166b00d1332d4f39b57ae9f58cb7fc5f P b6684bfcd7404fe68d8cb196153412d5: Time spent bootstrapping tablet: real 0.237s user 0.057s sys 0.099s
I20250109 08:56:59.321579 21207 ts_tablet_manager.cc:1397] T cb177a6fcf4144e2ae1a560eb627e276 P 6e2837e083814e77b16db43b99d985ab: Time spent bootstrapping tablet: real 0.134s user 0.009s sys 0.025s
I20250109 08:56:59.321738 21201 tablet_bootstrap.cc:492] T cb177a6fcf4144e2ae1a560eb627e276 P ead0b707c1014770b25c5a0db0454830: No bootstrap required, opened a new log
I20250109 08:56:59.322324 21201 ts_tablet_manager.cc:1397] T cb177a6fcf4144e2ae1a560eb627e276 P ead0b707c1014770b25c5a0db0454830: Time spent bootstrapping tablet: real 0.189s user 0.078s sys 0.102s
I20250109 08:56:59.322875 21085 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "502ebc25110a4c04a71b8b2e8eea07d4" candidate_uuid: "b6684bfcd7404fe68d8cb196153412d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e2837e083814e77b16db43b99d985ab" is_pre_election: true
I20250109 08:56:59.322353 21086 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "948e27b3034d44338e2b7d74f96374f7" candidate_uuid: "b6684bfcd7404fe68d8cb196153412d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e2837e083814e77b16db43b99d985ab" is_pre_election: true
W20250109 08:56:59.324275 20974 leader_election.cc:343] T 502ebc25110a4c04a71b8b2e8eea07d4 P b6684bfcd7404fe68d8cb196153412d5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6e2837e083814e77b16db43b99d985ab (127.19.226.194:44083): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:56:59.324761 20974 leader_election.cc:304] T 502ebc25110a4c04a71b8b2e8eea07d4 P b6684bfcd7404fe68d8cb196153412d5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b6684bfcd7404fe68d8cb196153412d5; no voters: 6e2837e083814e77b16db43b99d985ab, ead0b707c1014770b25c5a0db0454830
I20250109 08:56:59.324632 21207 raft_consensus.cc:357] T cb177a6fcf4144e2ae1a560eb627e276 P 6e2837e083814e77b16db43b99d985ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:56:59.324867 21201 raft_consensus.cc:357] T cb177a6fcf4144e2ae1a560eb627e276 P ead0b707c1014770b25c5a0db0454830 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:56:59.325389 21207 raft_consensus.cc:383] T cb177a6fcf4144e2ae1a560eb627e276 P 6e2837e083814e77b16db43b99d985ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:56:59.325559 21202 raft_consensus.cc:2743] T 502ebc25110a4c04a71b8b2e8eea07d4 P b6684bfcd7404fe68d8cb196153412d5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:56:59.325594 21201 raft_consensus.cc:383] T cb177a6fcf4144e2ae1a560eb627e276 P ead0b707c1014770b25c5a0db0454830 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:56:59.325788 21207 raft_consensus.cc:738] T cb177a6fcf4144e2ae1a560eb627e276 P 6e2837e083814e77b16db43b99d985ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e2837e083814e77b16db43b99d985ab, State: Initialized, Role: FOLLOWER
I20250109 08:56:59.326115 21201 raft_consensus.cc:738] T cb177a6fcf4144e2ae1a560eb627e276 P ead0b707c1014770b25c5a0db0454830 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ead0b707c1014770b25c5a0db0454830, State: Initialized, Role: FOLLOWER
I20250109 08:56:59.326603 21207 consensus_queue.cc:260] T cb177a6fcf4144e2ae1a560eb627e276 P 6e2837e083814e77b16db43b99d985ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
W20250109 08:56:59.326864 20974 leader_election.cc:343] T 948e27b3034d44338e2b7d74f96374f7 P b6684bfcd7404fe68d8cb196153412d5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6e2837e083814e77b16db43b99d985ab (127.19.226.194:44083): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:56:59.327517 20974 leader_election.cc:304] T 948e27b3034d44338e2b7d74f96374f7 P b6684bfcd7404fe68d8cb196153412d5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b6684bfcd7404fe68d8cb196153412d5; no voters: 6e2837e083814e77b16db43b99d985ab, ead0b707c1014770b25c5a0db0454830
I20250109 08:56:59.327147 21199 raft_consensus.cc:357] T 166b00d1332d4f39b57ae9f58cb7fc5f P b6684bfcd7404fe68d8cb196153412d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:56:59.328171 21199 raft_consensus.cc:383] T 166b00d1332d4f39b57ae9f58cb7fc5f P b6684bfcd7404fe68d8cb196153412d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:56:59.328250 21202 raft_consensus.cc:2743] T 948e27b3034d44338e2b7d74f96374f7 P b6684bfcd7404fe68d8cb196153412d5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:56:59.328625 21199 raft_consensus.cc:738] T 166b00d1332d4f39b57ae9f58cb7fc5f P b6684bfcd7404fe68d8cb196153412d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6684bfcd7404fe68d8cb196153412d5, State: Initialized, Role: FOLLOWER
I20250109 08:56:59.328931 21207 ts_tablet_manager.cc:1428] T cb177a6fcf4144e2ae1a560eb627e276 P 6e2837e083814e77b16db43b99d985ab: Time spent starting tablet: real 0.007s user 0.003s sys 0.003s
I20250109 08:56:59.326906 21201 consensus_queue.cc:260] T cb177a6fcf4144e2ae1a560eb627e276 P ead0b707c1014770b25c5a0db0454830 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:56:59.329910 21207 tablet_bootstrap.cc:492] T 53b47173fb6a4a358c8ad9c2df0f6f18 P 6e2837e083814e77b16db43b99d985ab: Bootstrap starting.
I20250109 08:56:59.329394 21199 consensus_queue.cc:260] T 166b00d1332d4f39b57ae9f58cb7fc5f P b6684bfcd7404fe68d8cb196153412d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:56:59.336143 21201 ts_tablet_manager.cc:1428] T cb177a6fcf4144e2ae1a560eb627e276 P ead0b707c1014770b25c5a0db0454830: Time spent starting tablet: real 0.013s user 0.003s sys 0.005s
I20250109 08:56:59.336301 21207 tablet_bootstrap.cc:654] T 53b47173fb6a4a358c8ad9c2df0f6f18 P 6e2837e083814e77b16db43b99d985ab: Neither blocks nor log segments found. Creating new log.
I20250109 08:56:59.337260 21201 tablet_bootstrap.cc:492] T af13c7179cc24b258e516eceb2b0cac0 P ead0b707c1014770b25c5a0db0454830: Bootstrap starting.
I20250109 08:56:59.339924 21206 raft_consensus.cc:491] T 42af5f76a37645b5a3865739d54a6cb3 P ead0b707c1014770b25c5a0db0454830 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:56:59.340598 21206 raft_consensus.cc:513] T 42af5f76a37645b5a3865739d54a6cb3 P ead0b707c1014770b25c5a0db0454830 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:56:59.342170 21199 ts_tablet_manager.cc:1428] T 166b00d1332d4f39b57ae9f58cb7fc5f P b6684bfcd7404fe68d8cb196153412d5: Time spent starting tablet: real 0.020s user 0.005s sys 0.001s
I20250109 08:56:59.344846 21199 tablet_bootstrap.cc:492] T ec3e3f891e3c4a879077b9ee897e5d52 P b6684bfcd7404fe68d8cb196153412d5: Bootstrap starting.
I20250109 08:56:59.348400 21206 leader_election.cc:290] T 42af5f76a37645b5a3865739d54a6cb3 P ead0b707c1014770b25c5a0db0454830 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b6684bfcd7404fe68d8cb196153412d5 (127.19.226.193:34209), 6e2837e083814e77b16db43b99d985ab (127.19.226.194:44083)
I20250109 08:56:59.354981 21201 tablet_bootstrap.cc:654] T af13c7179cc24b258e516eceb2b0cac0 P ead0b707c1014770b25c5a0db0454830: Neither blocks nor log segments found. Creating new log.
I20250109 08:56:59.357573 21199 tablet_bootstrap.cc:654] T ec3e3f891e3c4a879077b9ee897e5d52 P b6684bfcd7404fe68d8cb196153412d5: Neither blocks nor log segments found. Creating new log.
I20250109 08:56:59.364463 21207 tablet_bootstrap.cc:492] T 53b47173fb6a4a358c8ad9c2df0f6f18 P 6e2837e083814e77b16db43b99d985ab: No bootstrap required, opened a new log
I20250109 08:56:59.365010 21207 ts_tablet_manager.cc:1397] T 53b47173fb6a4a358c8ad9c2df0f6f18 P 6e2837e083814e77b16db43b99d985ab: Time spent bootstrapping tablet: real 0.035s user 0.010s sys 0.023s
I20250109 08:56:59.367954 21207 raft_consensus.cc:357] T 53b47173fb6a4a358c8ad9c2df0f6f18 P 6e2837e083814e77b16db43b99d985ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:56:59.368778 21207 raft_consensus.cc:383] T 53b47173fb6a4a358c8ad9c2df0f6f18 P 6e2837e083814e77b16db43b99d985ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:56:59.369067 21207 raft_consensus.cc:738] T 53b47173fb6a4a358c8ad9c2df0f6f18 P 6e2837e083814e77b16db43b99d985ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e2837e083814e77b16db43b99d985ab, State: Initialized, Role: FOLLOWER
I20250109 08:56:59.369773 21011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42af5f76a37645b5a3865739d54a6cb3" candidate_uuid: "ead0b707c1014770b25c5a0db0454830" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6684bfcd7404fe68d8cb196153412d5" is_pre_election: true
I20250109 08:56:59.369743 21207 consensus_queue.cc:260] T 53b47173fb6a4a358c8ad9c2df0f6f18 P 6e2837e083814e77b16db43b99d985ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
W20250109 08:56:59.371687 21125 leader_election.cc:343] T 42af5f76a37645b5a3865739d54a6cb3 P ead0b707c1014770b25c5a0db0454830 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b6684bfcd7404fe68d8cb196153412d5 (127.19.226.193:34209): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:56:59.372210 21207 ts_tablet_manager.cc:1428] T 53b47173fb6a4a358c8ad9c2df0f6f18 P 6e2837e083814e77b16db43b99d985ab: Time spent starting tablet: real 0.007s user 0.004s sys 0.000s
I20250109 08:56:59.373126 21207 tablet_bootstrap.cc:492] T 502ebc25110a4c04a71b8b2e8eea07d4 P 6e2837e083814e77b16db43b99d985ab: Bootstrap starting.
I20250109 08:56:59.379127 21207 tablet_bootstrap.cc:654] T 502ebc25110a4c04a71b8b2e8eea07d4 P 6e2837e083814e77b16db43b99d985ab: Neither blocks nor log segments found. Creating new log.
I20250109 08:56:59.385805 21086 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42af5f76a37645b5a3865739d54a6cb3" candidate_uuid: "ead0b707c1014770b25c5a0db0454830" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e2837e083814e77b16db43b99d985ab" is_pre_election: true
I20250109 08:56:59.386907 21199 tablet_bootstrap.cc:492] T ec3e3f891e3c4a879077b9ee897e5d52 P b6684bfcd7404fe68d8cb196153412d5: No bootstrap required, opened a new log
W20250109 08:56:59.387301 21123 leader_election.cc:343] T 42af5f76a37645b5a3865739d54a6cb3 P ead0b707c1014770b25c5a0db0454830 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6e2837e083814e77b16db43b99d985ab (127.19.226.194:44083): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:56:59.387447 21199 ts_tablet_manager.cc:1397] T ec3e3f891e3c4a879077b9ee897e5d52 P b6684bfcd7404fe68d8cb196153412d5: Time spent bootstrapping tablet: real 0.043s user 0.012s sys 0.000s
I20250109 08:56:59.387730 21123 leader_election.cc:304] T 42af5f76a37645b5a3865739d54a6cb3 P ead0b707c1014770b25c5a0db0454830 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ead0b707c1014770b25c5a0db0454830; no voters: 6e2837e083814e77b16db43b99d985ab, b6684bfcd7404fe68d8cb196153412d5
I20250109 08:56:59.388536 21206 raft_consensus.cc:2743] T 42af5f76a37645b5a3865739d54a6cb3 P ead0b707c1014770b25c5a0db0454830 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:56:59.389961 21199 raft_consensus.cc:357] T ec3e3f891e3c4a879077b9ee897e5d52 P b6684bfcd7404fe68d8cb196153412d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:56:59.390497 21199 raft_consensus.cc:383] T ec3e3f891e3c4a879077b9ee897e5d52 P b6684bfcd7404fe68d8cb196153412d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:56:59.390722 21199 raft_consensus.cc:738] T ec3e3f891e3c4a879077b9ee897e5d52 P b6684bfcd7404fe68d8cb196153412d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6684bfcd7404fe68d8cb196153412d5, State: Initialized, Role: FOLLOWER
I20250109 08:56:59.391489 21207 tablet_bootstrap.cc:492] T 502ebc25110a4c04a71b8b2e8eea07d4 P 6e2837e083814e77b16db43b99d985ab: No bootstrap required, opened a new log
I20250109 08:56:59.392000 21207 ts_tablet_manager.cc:1397] T 502ebc25110a4c04a71b8b2e8eea07d4 P 6e2837e083814e77b16db43b99d985ab: Time spent bootstrapping tablet: real 0.019s user 0.005s sys 0.012s
I20250109 08:56:59.391268 21199 consensus_queue.cc:260] T ec3e3f891e3c4a879077b9ee897e5d52 P b6684bfcd7404fe68d8cb196153412d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:56:59.393997 21201 tablet_bootstrap.cc:492] T af13c7179cc24b258e516eceb2b0cac0 P ead0b707c1014770b25c5a0db0454830: No bootstrap required, opened a new log
I20250109 08:56:59.394021 21199 ts_tablet_manager.cc:1428] T ec3e3f891e3c4a879077b9ee897e5d52 P b6684bfcd7404fe68d8cb196153412d5: Time spent starting tablet: real 0.006s user 0.003s sys 0.001s
I20250109 08:56:59.394666 21201 ts_tablet_manager.cc:1397] T af13c7179cc24b258e516eceb2b0cac0 P ead0b707c1014770b25c5a0db0454830: Time spent bootstrapping tablet: real 0.058s user 0.016s sys 0.021s
I20250109 08:56:59.394578 21207 raft_consensus.cc:357] T 502ebc25110a4c04a71b8b2e8eea07d4 P 6e2837e083814e77b16db43b99d985ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } }
I20250109 08:56:59.395123 21199 tablet_bootstrap.cc:492] T 42af5f76a37645b5a3865739d54a6cb3 P b6684bfcd7404fe68d8cb196153412d5: Bootstrap starting.
I20250109 08:56:59.395372 21207 raft_consensus.cc:383] T 502ebc25110a4c04a71b8b2e8eea07d4 P 6e2837e083814e77b16db43b99d985ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:56:59.395664 21207 raft_consensus.cc:738] T 502ebc25110a4c04a71b8b2e8eea07d4 P 6e2837e083814e77b16db43b99d985ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e2837e083814e77b16db43b99d985ab, State: Initialized, Role: FOLLOWER
I20250109 08:56:59.396313 21207 consensus_queue.cc:260] T 502ebc25110a4c04a71b8b2e8eea07d4 P 6e2837e083814e77b16db43b99d985ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } }
I20250109 08:56:59.398509 21201 raft_consensus.cc:357] T af13c7179cc24b258e516eceb2b0cac0 P ead0b707c1014770b25c5a0db0454830 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:56:59.399349 21201 raft_consensus.cc:383] T af13c7179cc24b258e516eceb2b0cac0 P ead0b707c1014770b25c5a0db0454830 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:56:59.398955 21207 ts_tablet_manager.cc:1428] T 502ebc25110a4c04a71b8b2e8eea07d4 P 6e2837e083814e77b16db43b99d985ab: Time spent starting tablet: real 0.007s user 0.004s sys 0.004s
I20250109 08:56:59.399713 21201 raft_consensus.cc:738] T af13c7179cc24b258e516eceb2b0cac0 P ead0b707c1014770b25c5a0db0454830 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ead0b707c1014770b25c5a0db0454830, State: Initialized, Role: FOLLOWER
I20250109 08:56:59.400475 21207 tablet_bootstrap.cc:492] T 948e27b3034d44338e2b7d74f96374f7 P 6e2837e083814e77b16db43b99d985ab: Bootstrap starting.
I20250109 08:56:59.400746 21201 consensus_queue.cc:260] T af13c7179cc24b258e516eceb2b0cac0 P ead0b707c1014770b25c5a0db0454830 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:56:59.403254 21201 ts_tablet_manager.cc:1428] T af13c7179cc24b258e516eceb2b0cac0 P ead0b707c1014770b25c5a0db0454830: Time spent starting tablet: real 0.008s user 0.003s sys 0.003s
I20250109 08:56:59.404165 21201 tablet_bootstrap.cc:492] T 53b47173fb6a4a358c8ad9c2df0f6f18 P ead0b707c1014770b25c5a0db0454830: Bootstrap starting.
I20250109 08:56:59.407202 21207 tablet_bootstrap.cc:654] T 948e27b3034d44338e2b7d74f96374f7 P 6e2837e083814e77b16db43b99d985ab: Neither blocks nor log segments found. Creating new log.
I20250109 08:56:59.410140 21201 tablet_bootstrap.cc:654] T 53b47173fb6a4a358c8ad9c2df0f6f18 P ead0b707c1014770b25c5a0db0454830: Neither blocks nor log segments found. Creating new log.
I20250109 08:56:59.413044 21199 tablet_bootstrap.cc:654] T 42af5f76a37645b5a3865739d54a6cb3 P b6684bfcd7404fe68d8cb196153412d5: Neither blocks nor log segments found. Creating new log.
I20250109 08:56:59.418434 21207 tablet_bootstrap.cc:492] T 948e27b3034d44338e2b7d74f96374f7 P 6e2837e083814e77b16db43b99d985ab: No bootstrap required, opened a new log
I20250109 08:56:59.418917 21207 ts_tablet_manager.cc:1397] T 948e27b3034d44338e2b7d74f96374f7 P 6e2837e083814e77b16db43b99d985ab: Time spent bootstrapping tablet: real 0.019s user 0.011s sys 0.004s
I20250109 08:56:59.421010 21199 tablet_bootstrap.cc:492] T 42af5f76a37645b5a3865739d54a6cb3 P b6684bfcd7404fe68d8cb196153412d5: No bootstrap required, opened a new log
I20250109 08:56:59.421450 21199 ts_tablet_manager.cc:1397] T 42af5f76a37645b5a3865739d54a6cb3 P b6684bfcd7404fe68d8cb196153412d5: Time spent bootstrapping tablet: real 0.026s user 0.011s sys 0.002s
I20250109 08:56:59.421554 21201 tablet_bootstrap.cc:492] T 53b47173fb6a4a358c8ad9c2df0f6f18 P ead0b707c1014770b25c5a0db0454830: No bootstrap required, opened a new log
I20250109 08:56:59.421361 21207 raft_consensus.cc:357] T 948e27b3034d44338e2b7d74f96374f7 P 6e2837e083814e77b16db43b99d985ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:56:59.421975 21201 ts_tablet_manager.cc:1397] T 53b47173fb6a4a358c8ad9c2df0f6f18 P ead0b707c1014770b25c5a0db0454830: Time spent bootstrapping tablet: real 0.018s user 0.011s sys 0.006s
I20250109 08:56:59.422166 21207 raft_consensus.cc:383] T 948e27b3034d44338e2b7d74f96374f7 P 6e2837e083814e77b16db43b99d985ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:56:59.422509 21207 raft_consensus.cc:738] T 948e27b3034d44338e2b7d74f96374f7 P 6e2837e083814e77b16db43b99d985ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e2837e083814e77b16db43b99d985ab, State: Initialized, Role: FOLLOWER
I20250109 08:56:59.423187 21207 consensus_queue.cc:260] T 948e27b3034d44338e2b7d74f96374f7 P 6e2837e083814e77b16db43b99d985ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:56:59.424327 21201 raft_consensus.cc:357] T 53b47173fb6a4a358c8ad9c2df0f6f18 P ead0b707c1014770b25c5a0db0454830 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:56:59.424916 21201 raft_consensus.cc:383] T 53b47173fb6a4a358c8ad9c2df0f6f18 P ead0b707c1014770b25c5a0db0454830 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:56:59.425150 21201 raft_consensus.cc:738] T 53b47173fb6a4a358c8ad9c2df0f6f18 P ead0b707c1014770b25c5a0db0454830 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ead0b707c1014770b25c5a0db0454830, State: Initialized, Role: FOLLOWER
I20250109 08:56:59.425421 21207 ts_tablet_manager.cc:1428] T 948e27b3034d44338e2b7d74f96374f7 P 6e2837e083814e77b16db43b99d985ab: Time spent starting tablet: real 0.006s user 0.007s sys 0.001s
I20250109 08:56:59.425689 21201 consensus_queue.cc:260] T 53b47173fb6a4a358c8ad9c2df0f6f18 P ead0b707c1014770b25c5a0db0454830 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:56:59.426307 21207 tablet_bootstrap.cc:492] T af13c7179cc24b258e516eceb2b0cac0 P 6e2837e083814e77b16db43b99d985ab: Bootstrap starting.
I20250109 08:56:59.427719 21201 ts_tablet_manager.cc:1428] T 53b47173fb6a4a358c8ad9c2df0f6f18 P ead0b707c1014770b25c5a0db0454830: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20250109 08:56:59.429394 21201 tablet_bootstrap.cc:492] T 05148030cdd443048fe5899b03fd92fe P ead0b707c1014770b25c5a0db0454830: Bootstrap starting.
I20250109 08:56:59.432540 21207 tablet_bootstrap.cc:654] T af13c7179cc24b258e516eceb2b0cac0 P 6e2837e083814e77b16db43b99d985ab: Neither blocks nor log segments found. Creating new log.
I20250109 08:56:59.424105 21199 raft_consensus.cc:357] T 42af5f76a37645b5a3865739d54a6cb3 P b6684bfcd7404fe68d8cb196153412d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:56:59.433732 21201 tablet_bootstrap.cc:654] T 05148030cdd443048fe5899b03fd92fe P ead0b707c1014770b25c5a0db0454830: Neither blocks nor log segments found. Creating new log.
I20250109 08:56:59.433990 21199 raft_consensus.cc:383] T 42af5f76a37645b5a3865739d54a6cb3 P b6684bfcd7404fe68d8cb196153412d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:56:59.434391 21199 raft_consensus.cc:738] T 42af5f76a37645b5a3865739d54a6cb3 P b6684bfcd7404fe68d8cb196153412d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6684bfcd7404fe68d8cb196153412d5, State: Initialized, Role: FOLLOWER
I20250109 08:56:59.435177 21199 consensus_queue.cc:260] T 42af5f76a37645b5a3865739d54a6cb3 P b6684bfcd7404fe68d8cb196153412d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:56:59.439855 21199 ts_tablet_manager.cc:1428] T 42af5f76a37645b5a3865739d54a6cb3 P b6684bfcd7404fe68d8cb196153412d5: Time spent starting tablet: real 0.018s user 0.003s sys 0.002s
I20250109 08:56:59.440927 21199 tablet_bootstrap.cc:492] T 53b47173fb6a4a358c8ad9c2df0f6f18 P b6684bfcd7404fe68d8cb196153412d5: Bootstrap starting.
I20250109 08:56:59.445674 21201 tablet_bootstrap.cc:492] T 05148030cdd443048fe5899b03fd92fe P ead0b707c1014770b25c5a0db0454830: No bootstrap required, opened a new log
I20250109 08:56:59.446136 21201 ts_tablet_manager.cc:1397] T 05148030cdd443048fe5899b03fd92fe P ead0b707c1014770b25c5a0db0454830: Time spent bootstrapping tablet: real 0.018s user 0.005s sys 0.009s
I20250109 08:56:59.447080 21199 tablet_bootstrap.cc:654] T 53b47173fb6a4a358c8ad9c2df0f6f18 P b6684bfcd7404fe68d8cb196153412d5: Neither blocks nor log segments found. Creating new log.
I20250109 08:56:59.448684 21201 raft_consensus.cc:357] T 05148030cdd443048fe5899b03fd92fe P ead0b707c1014770b25c5a0db0454830 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:56:59.449432 21201 raft_consensus.cc:383] T 05148030cdd443048fe5899b03fd92fe P ead0b707c1014770b25c5a0db0454830 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:56:59.449716 21201 raft_consensus.cc:738] T 05148030cdd443048fe5899b03fd92fe P ead0b707c1014770b25c5a0db0454830 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ead0b707c1014770b25c5a0db0454830, State: Initialized, Role: FOLLOWER
I20250109 08:56:59.450500 21201 consensus_queue.cc:260] T 05148030cdd443048fe5899b03fd92fe P ead0b707c1014770b25c5a0db0454830 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:56:59.453342 21201 ts_tablet_manager.cc:1428] T 05148030cdd443048fe5899b03fd92fe P ead0b707c1014770b25c5a0db0454830: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20250109 08:56:59.454393 21201 tablet_bootstrap.cc:492] T ec3e3f891e3c4a879077b9ee897e5d52 P ead0b707c1014770b25c5a0db0454830: Bootstrap starting.
I20250109 08:56:59.459838 21199 tablet_bootstrap.cc:492] T 53b47173fb6a4a358c8ad9c2df0f6f18 P b6684bfcd7404fe68d8cb196153412d5: No bootstrap required, opened a new log
I20250109 08:56:59.460311 21199 ts_tablet_manager.cc:1397] T 53b47173fb6a4a358c8ad9c2df0f6f18 P b6684bfcd7404fe68d8cb196153412d5: Time spent bootstrapping tablet: real 0.020s user 0.009s sys 0.007s
I20250109 08:56:59.461304 21201 tablet_bootstrap.cc:654] T ec3e3f891e3c4a879077b9ee897e5d52 P ead0b707c1014770b25c5a0db0454830: Neither blocks nor log segments found. Creating new log.
I20250109 08:56:59.462894 21199 raft_consensus.cc:357] T 53b47173fb6a4a358c8ad9c2df0f6f18 P b6684bfcd7404fe68d8cb196153412d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:56:59.463518 21199 raft_consensus.cc:383] T 53b47173fb6a4a358c8ad9c2df0f6f18 P b6684bfcd7404fe68d8cb196153412d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:56:59.463801 21199 raft_consensus.cc:738] T 53b47173fb6a4a358c8ad9c2df0f6f18 P b6684bfcd7404fe68d8cb196153412d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6684bfcd7404fe68d8cb196153412d5, State: Initialized, Role: FOLLOWER
I20250109 08:56:59.464597 21199 consensus_queue.cc:260] T 53b47173fb6a4a358c8ad9c2df0f6f18 P b6684bfcd7404fe68d8cb196153412d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:56:59.466157 21207 tablet_bootstrap.cc:492] T af13c7179cc24b258e516eceb2b0cac0 P 6e2837e083814e77b16db43b99d985ab: No bootstrap required, opened a new log
I20250109 08:56:59.466183 21201 tablet_bootstrap.cc:492] T ec3e3f891e3c4a879077b9ee897e5d52 P ead0b707c1014770b25c5a0db0454830: No bootstrap required, opened a new log
I20250109 08:56:59.466634 21199 ts_tablet_manager.cc:1428] T 53b47173fb6a4a358c8ad9c2df0f6f18 P b6684bfcd7404fe68d8cb196153412d5: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20250109 08:56:59.466769 21207 ts_tablet_manager.cc:1397] T af13c7179cc24b258e516eceb2b0cac0 P 6e2837e083814e77b16db43b99d985ab: Time spent bootstrapping tablet: real 0.041s user 0.018s sys 0.014s
I20250109 08:56:59.466827 21201 ts_tablet_manager.cc:1397] T ec3e3f891e3c4a879077b9ee897e5d52 P ead0b707c1014770b25c5a0db0454830: Time spent bootstrapping tablet: real 0.013s user 0.009s sys 0.001s
I20250109 08:56:59.467594 21199 tablet_bootstrap.cc:492] T 05148030cdd443048fe5899b03fd92fe P b6684bfcd7404fe68d8cb196153412d5: Bootstrap starting.
I20250109 08:56:59.469681 21201 raft_consensus.cc:357] T ec3e3f891e3c4a879077b9ee897e5d52 P ead0b707c1014770b25c5a0db0454830 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
W20250109 08:56:59.470481 20957 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20250109 08:56:59.470451 21201 raft_consensus.cc:383] T ec3e3f891e3c4a879077b9ee897e5d52 P ead0b707c1014770b25c5a0db0454830 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:56:59.470949 21201 raft_consensus.cc:738] T ec3e3f891e3c4a879077b9ee897e5d52 P ead0b707c1014770b25c5a0db0454830 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ead0b707c1014770b25c5a0db0454830, State: Initialized, Role: FOLLOWER
I20250109 08:56:59.471788 21201 consensus_queue.cc:260] T ec3e3f891e3c4a879077b9ee897e5d52 P ead0b707c1014770b25c5a0db0454830 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:56:59.472293 21207 raft_consensus.cc:357] T af13c7179cc24b258e516eceb2b0cac0 P 6e2837e083814e77b16db43b99d985ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:56:59.473028 21207 raft_consensus.cc:383] T af13c7179cc24b258e516eceb2b0cac0 P 6e2837e083814e77b16db43b99d985ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:56:59.473343 21207 raft_consensus.cc:738] T af13c7179cc24b258e516eceb2b0cac0 P 6e2837e083814e77b16db43b99d985ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e2837e083814e77b16db43b99d985ab, State: Initialized, Role: FOLLOWER
I20250109 08:56:59.473973 21199 tablet_bootstrap.cc:654] T 05148030cdd443048fe5899b03fd92fe P b6684bfcd7404fe68d8cb196153412d5: Neither blocks nor log segments found. Creating new log.
I20250109 08:56:59.474105 21207 consensus_queue.cc:260] T af13c7179cc24b258e516eceb2b0cac0 P 6e2837e083814e77b16db43b99d985ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:56:59.474704 21201 ts_tablet_manager.cc:1428] T ec3e3f891e3c4a879077b9ee897e5d52 P ead0b707c1014770b25c5a0db0454830: Time spent starting tablet: real 0.007s user 0.003s sys 0.003s
I20250109 08:56:59.475687 21201 tablet_bootstrap.cc:492] T 163310a12b44475ca3727662dad194a6 P ead0b707c1014770b25c5a0db0454830: Bootstrap starting.
I20250109 08:56:59.476907 21207 ts_tablet_manager.cc:1428] T af13c7179cc24b258e516eceb2b0cac0 P 6e2837e083814e77b16db43b99d985ab: Time spent starting tablet: real 0.010s user 0.007s sys 0.000s
I20250109 08:56:59.477857 21207 tablet_bootstrap.cc:492] T 166b00d1332d4f39b57ae9f58cb7fc5f P 6e2837e083814e77b16db43b99d985ab: Bootstrap starting.
I20250109 08:56:59.482015 21201 tablet_bootstrap.cc:654] T 163310a12b44475ca3727662dad194a6 P ead0b707c1014770b25c5a0db0454830: Neither blocks nor log segments found. Creating new log.
I20250109 08:56:59.483832 21207 tablet_bootstrap.cc:654] T 166b00d1332d4f39b57ae9f58cb7fc5f P 6e2837e083814e77b16db43b99d985ab: Neither blocks nor log segments found. Creating new log.
I20250109 08:56:59.492494 21207 tablet_bootstrap.cc:492] T 166b00d1332d4f39b57ae9f58cb7fc5f P 6e2837e083814e77b16db43b99d985ab: No bootstrap required, opened a new log
I20250109 08:56:59.493003 21207 ts_tablet_manager.cc:1397] T 166b00d1332d4f39b57ae9f58cb7fc5f P 6e2837e083814e77b16db43b99d985ab: Time spent bootstrapping tablet: real 0.015s user 0.014s sys 0.000s
I20250109 08:56:59.495741 21207 raft_consensus.cc:357] T 166b00d1332d4f39b57ae9f58cb7fc5f P 6e2837e083814e77b16db43b99d985ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:56:59.496568 21207 raft_consensus.cc:383] T 166b00d1332d4f39b57ae9f58cb7fc5f P 6e2837e083814e77b16db43b99d985ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:56:59.496868 21207 raft_consensus.cc:738] T 166b00d1332d4f39b57ae9f58cb7fc5f P 6e2837e083814e77b16db43b99d985ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e2837e083814e77b16db43b99d985ab, State: Initialized, Role: FOLLOWER
I20250109 08:56:59.498734 21207 consensus_queue.cc:260] T 166b00d1332d4f39b57ae9f58cb7fc5f P 6e2837e083814e77b16db43b99d985ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:56:59.500030 21201 tablet_bootstrap.cc:492] T 163310a12b44475ca3727662dad194a6 P ead0b707c1014770b25c5a0db0454830: No bootstrap required, opened a new log
I20250109 08:56:59.500587 21201 ts_tablet_manager.cc:1397] T 163310a12b44475ca3727662dad194a6 P ead0b707c1014770b25c5a0db0454830: Time spent bootstrapping tablet: real 0.025s user 0.021s sys 0.003s
I20250109 08:56:59.501173 21207 ts_tablet_manager.cc:1428] T 166b00d1332d4f39b57ae9f58cb7fc5f P 6e2837e083814e77b16db43b99d985ab: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20250109 08:56:59.502087 21207 tablet_bootstrap.cc:492] T 163310a12b44475ca3727662dad194a6 P 6e2837e083814e77b16db43b99d985ab: Bootstrap starting.
I20250109 08:56:59.503453 21201 raft_consensus.cc:357] T 163310a12b44475ca3727662dad194a6 P ead0b707c1014770b25c5a0db0454830 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:56:59.504252 21201 raft_consensus.cc:383] T 163310a12b44475ca3727662dad194a6 P ead0b707c1014770b25c5a0db0454830 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:56:59.504608 21201 raft_consensus.cc:738] T 163310a12b44475ca3727662dad194a6 P ead0b707c1014770b25c5a0db0454830 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ead0b707c1014770b25c5a0db0454830, State: Initialized, Role: FOLLOWER
I20250109 08:56:59.505452 21201 consensus_queue.cc:260] T 163310a12b44475ca3727662dad194a6 P ead0b707c1014770b25c5a0db0454830 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:56:59.508132 21201 ts_tablet_manager.cc:1428] T 163310a12b44475ca3727662dad194a6 P ead0b707c1014770b25c5a0db0454830: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20250109 08:56:59.508545 21207 tablet_bootstrap.cc:654] T 163310a12b44475ca3727662dad194a6 P 6e2837e083814e77b16db43b99d985ab: Neither blocks nor log segments found. Creating new log.
I20250109 08:56:59.509434 21201 tablet_bootstrap.cc:492] T 502ebc25110a4c04a71b8b2e8eea07d4 P ead0b707c1014770b25c5a0db0454830: Bootstrap starting.
I20250109 08:56:59.515926 21201 tablet_bootstrap.cc:654] T 502ebc25110a4c04a71b8b2e8eea07d4 P ead0b707c1014770b25c5a0db0454830: Neither blocks nor log segments found. Creating new log.
I20250109 08:56:59.519871 21207 tablet_bootstrap.cc:492] T 163310a12b44475ca3727662dad194a6 P 6e2837e083814e77b16db43b99d985ab: No bootstrap required, opened a new log
I20250109 08:56:59.520346 21207 ts_tablet_manager.cc:1397] T 163310a12b44475ca3727662dad194a6 P 6e2837e083814e77b16db43b99d985ab: Time spent bootstrapping tablet: real 0.018s user 0.016s sys 0.000s
I20250109 08:56:59.522859 21207 raft_consensus.cc:357] T 163310a12b44475ca3727662dad194a6 P 6e2837e083814e77b16db43b99d985ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:56:59.523594 21207 raft_consensus.cc:383] T 163310a12b44475ca3727662dad194a6 P 6e2837e083814e77b16db43b99d985ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:56:59.523859 21207 raft_consensus.cc:738] T 163310a12b44475ca3727662dad194a6 P 6e2837e083814e77b16db43b99d985ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e2837e083814e77b16db43b99d985ab, State: Initialized, Role: FOLLOWER
I20250109 08:56:59.524713 21207 consensus_queue.cc:260] T 163310a12b44475ca3727662dad194a6 P 6e2837e083814e77b16db43b99d985ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:56:59.527065 21207 ts_tablet_manager.cc:1428] T 163310a12b44475ca3727662dad194a6 P 6e2837e083814e77b16db43b99d985ab: Time spent starting tablet: real 0.006s user 0.003s sys 0.002s
I20250109 08:56:59.528015 21207 tablet_bootstrap.cc:492] T 42af5f76a37645b5a3865739d54a6cb3 P 6e2837e083814e77b16db43b99d985ab: Bootstrap starting.
I20250109 08:56:59.530905 21210 raft_consensus.cc:491] T 502ebc25110a4c04a71b8b2e8eea07d4 P 6e2837e083814e77b16db43b99d985ab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:56:59.531373 21210 raft_consensus.cc:513] T 502ebc25110a4c04a71b8b2e8eea07d4 P 6e2837e083814e77b16db43b99d985ab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } }
I20250109 08:56:59.541893 21210 leader_election.cc:290] T 502ebc25110a4c04a71b8b2e8eea07d4 P 6e2837e083814e77b16db43b99d985ab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b6684bfcd7404fe68d8cb196153412d5 (127.19.226.193:34209), ead0b707c1014770b25c5a0db0454830 (127.19.226.195:35749)
I20250109 08:56:59.542461 21199 tablet_bootstrap.cc:492] T 05148030cdd443048fe5899b03fd92fe P b6684bfcd7404fe68d8cb196153412d5: No bootstrap required, opened a new log
I20250109 08:56:59.542857 21199 ts_tablet_manager.cc:1397] T 05148030cdd443048fe5899b03fd92fe P b6684bfcd7404fe68d8cb196153412d5: Time spent bootstrapping tablet: real 0.075s user 0.024s sys 0.047s
I20250109 08:56:59.544765 21199 raft_consensus.cc:357] T 05148030cdd443048fe5899b03fd92fe P b6684bfcd7404fe68d8cb196153412d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:56:59.545285 21199 raft_consensus.cc:383] T 05148030cdd443048fe5899b03fd92fe P b6684bfcd7404fe68d8cb196153412d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:56:59.545512 21199 raft_consensus.cc:738] T 05148030cdd443048fe5899b03fd92fe P b6684bfcd7404fe68d8cb196153412d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6684bfcd7404fe68d8cb196153412d5, State: Initialized, Role: FOLLOWER
I20250109 08:56:59.546039 21199 consensus_queue.cc:260] T 05148030cdd443048fe5899b03fd92fe P b6684bfcd7404fe68d8cb196153412d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:56:59.548480 21199 ts_tablet_manager.cc:1428] T 05148030cdd443048fe5899b03fd92fe P b6684bfcd7404fe68d8cb196153412d5: Time spent starting tablet: real 0.005s user 0.007s sys 0.000s
I20250109 08:56:59.549767 21199 tablet_bootstrap.cc:492] T 163310a12b44475ca3727662dad194a6 P b6684bfcd7404fe68d8cb196153412d5: Bootstrap starting.
I20250109 08:56:59.555943 21199 tablet_bootstrap.cc:654] T 163310a12b44475ca3727662dad194a6 P b6684bfcd7404fe68d8cb196153412d5: Neither blocks nor log segments found. Creating new log.
I20250109 08:56:59.557283 21207 tablet_bootstrap.cc:654] T 42af5f76a37645b5a3865739d54a6cb3 P 6e2837e083814e77b16db43b99d985ab: Neither blocks nor log segments found. Creating new log.
I20250109 08:56:59.560990 21210 raft_consensus.cc:491] T ec3e3f891e3c4a879077b9ee897e5d52 P 6e2837e083814e77b16db43b99d985ab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:56:59.561920 21210 raft_consensus.cc:513] T ec3e3f891e3c4a879077b9ee897e5d52 P 6e2837e083814e77b16db43b99d985ab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:56:59.564018 21210 leader_election.cc:290] T ec3e3f891e3c4a879077b9ee897e5d52 P 6e2837e083814e77b16db43b99d985ab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b6684bfcd7404fe68d8cb196153412d5 (127.19.226.193:34209), ead0b707c1014770b25c5a0db0454830 (127.19.226.195:35749)
I20250109 08:56:59.564976 21201 tablet_bootstrap.cc:492] T 502ebc25110a4c04a71b8b2e8eea07d4 P ead0b707c1014770b25c5a0db0454830: No bootstrap required, opened a new log
I20250109 08:56:59.565225 21206 raft_consensus.cc:491] T 53b47173fb6a4a358c8ad9c2df0f6f18 P ead0b707c1014770b25c5a0db0454830 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:56:59.565440 21201 ts_tablet_manager.cc:1397] T 502ebc25110a4c04a71b8b2e8eea07d4 P ead0b707c1014770b25c5a0db0454830: Time spent bootstrapping tablet: real 0.056s user 0.030s sys 0.023s
I20250109 08:56:59.565680 21206 raft_consensus.cc:513] T 53b47173fb6a4a358c8ad9c2df0f6f18 P ead0b707c1014770b25c5a0db0454830 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:56:59.566783 21199 tablet_bootstrap.cc:492] T 163310a12b44475ca3727662dad194a6 P b6684bfcd7404fe68d8cb196153412d5: No bootstrap required, opened a new log
I20250109 08:56:59.567137 21199 ts_tablet_manager.cc:1397] T 163310a12b44475ca3727662dad194a6 P b6684bfcd7404fe68d8cb196153412d5: Time spent bootstrapping tablet: real 0.018s user 0.013s sys 0.002s
I20250109 08:56:59.567620 21206 leader_election.cc:290] T 53b47173fb6a4a358c8ad9c2df0f6f18 P ead0b707c1014770b25c5a0db0454830 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b6684bfcd7404fe68d8cb196153412d5 (127.19.226.193:34209), 6e2837e083814e77b16db43b99d985ab (127.19.226.194:44083)
I20250109 08:56:59.568029 21201 raft_consensus.cc:357] T 502ebc25110a4c04a71b8b2e8eea07d4 P ead0b707c1014770b25c5a0db0454830 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } }
I20250109 08:56:59.568405 21011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53b47173fb6a4a358c8ad9c2df0f6f18" candidate_uuid: "ead0b707c1014770b25c5a0db0454830" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6684bfcd7404fe68d8cb196153412d5" is_pre_election: true
I20250109 08:56:59.568750 21201 raft_consensus.cc:383] T 502ebc25110a4c04a71b8b2e8eea07d4 P ead0b707c1014770b25c5a0db0454830 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:56:59.569046 21201 raft_consensus.cc:738] T 502ebc25110a4c04a71b8b2e8eea07d4 P ead0b707c1014770b25c5a0db0454830 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ead0b707c1014770b25c5a0db0454830, State: Initialized, Role: FOLLOWER
I20250109 08:56:59.569236 21011 raft_consensus.cc:2463] T 53b47173fb6a4a358c8ad9c2df0f6f18 P b6684bfcd7404fe68d8cb196153412d5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ead0b707c1014770b25c5a0db0454830 in term 0.
I20250109 08:56:59.569397 21199 raft_consensus.cc:357] T 163310a12b44475ca3727662dad194a6 P b6684bfcd7404fe68d8cb196153412d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:56:59.570075 21199 raft_consensus.cc:383] T 163310a12b44475ca3727662dad194a6 P b6684bfcd7404fe68d8cb196153412d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:56:59.570379 21199 raft_consensus.cc:738] T 163310a12b44475ca3727662dad194a6 P b6684bfcd7404fe68d8cb196153412d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6684bfcd7404fe68d8cb196153412d5, State: Initialized, Role: FOLLOWER
I20250109 08:56:59.570755 21125 leader_election.cc:304] T 53b47173fb6a4a358c8ad9c2df0f6f18 P ead0b707c1014770b25c5a0db0454830 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b6684bfcd7404fe68d8cb196153412d5, ead0b707c1014770b25c5a0db0454830; no voters:
I20250109 08:56:59.571002 21199 consensus_queue.cc:260] T 163310a12b44475ca3727662dad194a6 P b6684bfcd7404fe68d8cb196153412d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:56:59.571686 21206 raft_consensus.cc:2798] T 53b47173fb6a4a358c8ad9c2df0f6f18 P ead0b707c1014770b25c5a0db0454830 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:56:59.572001 21206 raft_consensus.cc:491] T 53b47173fb6a4a358c8ad9c2df0f6f18 P ead0b707c1014770b25c5a0db0454830 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:56:59.572391 21206 raft_consensus.cc:3054] T 53b47173fb6a4a358c8ad9c2df0f6f18 P ead0b707c1014770b25c5a0db0454830 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:56:59.573156 21199 ts_tablet_manager.cc:1428] T 163310a12b44475ca3727662dad194a6 P b6684bfcd7404fe68d8cb196153412d5: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20250109 08:56:59.572664 21201 consensus_queue.cc:260] T 502ebc25110a4c04a71b8b2e8eea07d4 P ead0b707c1014770b25c5a0db0454830 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } }
I20250109 08:56:59.579329 21206 raft_consensus.cc:513] T 53b47173fb6a4a358c8ad9c2df0f6f18 P ead0b707c1014770b25c5a0db0454830 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:56:59.582041 21011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53b47173fb6a4a358c8ad9c2df0f6f18" candidate_uuid: "ead0b707c1014770b25c5a0db0454830" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6684bfcd7404fe68d8cb196153412d5"
I20250109 08:56:59.582413 21086 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53b47173fb6a4a358c8ad9c2df0f6f18" candidate_uuid: "ead0b707c1014770b25c5a0db0454830" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e2837e083814e77b16db43b99d985ab" is_pre_election: true
I20250109 08:56:59.583092 21086 raft_consensus.cc:2463] T 53b47173fb6a4a358c8ad9c2df0f6f18 P 6e2837e083814e77b16db43b99d985ab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ead0b707c1014770b25c5a0db0454830 in term 0.
I20250109 08:56:59.582617 21011 raft_consensus.cc:3054] T 53b47173fb6a4a358c8ad9c2df0f6f18 P b6684bfcd7404fe68d8cb196153412d5 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:56:59.583701 21206 leader_election.cc:290] T 53b47173fb6a4a358c8ad9c2df0f6f18 P ead0b707c1014770b25c5a0db0454830 [CANDIDATE]: Term 1 election: Requested vote from peers b6684bfcd7404fe68d8cb196153412d5 (127.19.226.193:34209), 6e2837e083814e77b16db43b99d985ab (127.19.226.194:44083)
I20250109 08:56:59.586616 21199 tablet_bootstrap.cc:492] T cb177a6fcf4144e2ae1a560eb627e276 P b6684bfcd7404fe68d8cb196153412d5: Bootstrap starting.
I20250109 08:56:59.591183 21011 raft_consensus.cc:2463] T 53b47173fb6a4a358c8ad9c2df0f6f18 P b6684bfcd7404fe68d8cb196153412d5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ead0b707c1014770b25c5a0db0454830 in term 1.
I20250109 08:56:59.591549 21199 tablet_bootstrap.cc:654] T cb177a6fcf4144e2ae1a560eb627e276 P b6684bfcd7404fe68d8cb196153412d5: Neither blocks nor log segments found. Creating new log.
I20250109 08:56:59.592476 21125 leader_election.cc:304] T 53b47173fb6a4a358c8ad9c2df0f6f18 P ead0b707c1014770b25c5a0db0454830 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b6684bfcd7404fe68d8cb196153412d5, ead0b707c1014770b25c5a0db0454830; no voters:
I20250109 08:56:59.593384 21206 raft_consensus.cc:491] T 502ebc25110a4c04a71b8b2e8eea07d4 P ead0b707c1014770b25c5a0db0454830 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:56:59.593858 21206 raft_consensus.cc:513] T 502ebc25110a4c04a71b8b2e8eea07d4 P ead0b707c1014770b25c5a0db0454830 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } }
I20250109 08:56:59.594978 21086 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53b47173fb6a4a358c8ad9c2df0f6f18" candidate_uuid: "ead0b707c1014770b25c5a0db0454830" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e2837e083814e77b16db43b99d985ab"
I20250109 08:56:59.595614 21086 raft_consensus.cc:3054] T 53b47173fb6a4a358c8ad9c2df0f6f18 P 6e2837e083814e77b16db43b99d985ab [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:56:59.597900 21011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "502ebc25110a4c04a71b8b2e8eea07d4" candidate_uuid: "ead0b707c1014770b25c5a0db0454830" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6684bfcd7404fe68d8cb196153412d5" is_pre_election: true
I20250109 08:56:59.598685 21011 raft_consensus.cc:2463] T 502ebc25110a4c04a71b8b2e8eea07d4 P b6684bfcd7404fe68d8cb196153412d5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ead0b707c1014770b25c5a0db0454830 in term 0.
I20250109 08:56:59.600651 21125 leader_election.cc:304] T 502ebc25110a4c04a71b8b2e8eea07d4 P ead0b707c1014770b25c5a0db0454830 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b6684bfcd7404fe68d8cb196153412d5, ead0b707c1014770b25c5a0db0454830; no voters:
I20250109 08:56:59.602762 21199 tablet_bootstrap.cc:492] T cb177a6fcf4144e2ae1a560eb627e276 P b6684bfcd7404fe68d8cb196153412d5: No bootstrap required, opened a new log
I20250109 08:56:59.602941 21086 raft_consensus.cc:2463] T 53b47173fb6a4a358c8ad9c2df0f6f18 P 6e2837e083814e77b16db43b99d985ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ead0b707c1014770b25c5a0db0454830 in term 1.
I20250109 08:56:59.603382 21199 ts_tablet_manager.cc:1397] T cb177a6fcf4144e2ae1a560eb627e276 P b6684bfcd7404fe68d8cb196153412d5: Time spent bootstrapping tablet: real 0.017s user 0.012s sys 0.004s
I20250109 08:56:59.605351 21086 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "502ebc25110a4c04a71b8b2e8eea07d4" candidate_uuid: "ead0b707c1014770b25c5a0db0454830" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e2837e083814e77b16db43b99d985ab" is_pre_election: true
I20250109 08:56:59.606045 21086 raft_consensus.cc:2463] T 502ebc25110a4c04a71b8b2e8eea07d4 P 6e2837e083814e77b16db43b99d985ab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ead0b707c1014770b25c5a0db0454830 in term 0.
I20250109 08:56:59.599548 21206 leader_election.cc:290] T 502ebc25110a4c04a71b8b2e8eea07d4 P ead0b707c1014770b25c5a0db0454830 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b6684bfcd7404fe68d8cb196153412d5 (127.19.226.193:34209), 6e2837e083814e77b16db43b99d985ab (127.19.226.194:44083)
I20250109 08:56:59.606200 21199 raft_consensus.cc:357] T cb177a6fcf4144e2ae1a560eb627e276 P b6684bfcd7404fe68d8cb196153412d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:56:59.607187 21199 raft_consensus.cc:383] T cb177a6fcf4144e2ae1a560eb627e276 P b6684bfcd7404fe68d8cb196153412d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:56:59.607532 21199 raft_consensus.cc:738] T cb177a6fcf4144e2ae1a560eb627e276 P b6684bfcd7404fe68d8cb196153412d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6684bfcd7404fe68d8cb196153412d5, State: Initialized, Role: FOLLOWER
I20250109 08:56:59.606755 21206 raft_consensus.cc:2798] T 53b47173fb6a4a358c8ad9c2df0f6f18 P ead0b707c1014770b25c5a0db0454830 [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:56:59.608426 21206 raft_consensus.cc:695] T 53b47173fb6a4a358c8ad9c2df0f6f18 P ead0b707c1014770b25c5a0db0454830 [term 1 LEADER]: Becoming Leader. State: Replica: ead0b707c1014770b25c5a0db0454830, State: Running, Role: LEADER
I20250109 08:56:59.610287 21206 consensus_queue.cc:237] T 53b47173fb6a4a358c8ad9c2df0f6f18 P ead0b707c1014770b25c5a0db0454830 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:56:59.609038 21199 consensus_queue.cc:260] T cb177a6fcf4144e2ae1a560eb627e276 P b6684bfcd7404fe68d8cb196153412d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:56:59.614733 21207 tablet_bootstrap.cc:492] T 42af5f76a37645b5a3865739d54a6cb3 P 6e2837e083814e77b16db43b99d985ab: No bootstrap required, opened a new log
I20250109 08:56:59.615206 21207 ts_tablet_manager.cc:1397] T 42af5f76a37645b5a3865739d54a6cb3 P 6e2837e083814e77b16db43b99d985ab: Time spent bootstrapping tablet: real 0.087s user 0.008s sys 0.000s
I20250109 08:56:59.616652 21199 ts_tablet_manager.cc:1428] T cb177a6fcf4144e2ae1a560eb627e276 P b6684bfcd7404fe68d8cb196153412d5: Time spent starting tablet: real 0.013s user 0.006s sys 0.001s
I20250109 08:56:59.617547 21199 tablet_bootstrap.cc:492] T af13c7179cc24b258e516eceb2b0cac0 P b6684bfcd7404fe68d8cb196153412d5: Bootstrap starting.
I20250109 08:56:59.624569 21199 tablet_bootstrap.cc:654] T af13c7179cc24b258e516eceb2b0cac0 P b6684bfcd7404fe68d8cb196153412d5: Neither blocks nor log segments found. Creating new log.
I20250109 08:56:59.627429 21011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "502ebc25110a4c04a71b8b2e8eea07d4" candidate_uuid: "6e2837e083814e77b16db43b99d985ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6684bfcd7404fe68d8cb196153412d5" is_pre_election: true
I20250109 08:56:59.627669 21010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec3e3f891e3c4a879077b9ee897e5d52" candidate_uuid: "6e2837e083814e77b16db43b99d985ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6684bfcd7404fe68d8cb196153412d5" is_pre_election: true
I20250109 08:56:59.628090 21011 raft_consensus.cc:2463] T 502ebc25110a4c04a71b8b2e8eea07d4 P b6684bfcd7404fe68d8cb196153412d5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e2837e083814e77b16db43b99d985ab in term 0.
I20250109 08:56:59.628347 21010 raft_consensus.cc:2463] T ec3e3f891e3c4a879077b9ee897e5d52 P b6684bfcd7404fe68d8cb196153412d5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e2837e083814e77b16db43b99d985ab in term 0.
I20250109 08:56:59.629623 21051 leader_election.cc:304] T 502ebc25110a4c04a71b8b2e8eea07d4 P 6e2837e083814e77b16db43b99d985ab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e2837e083814e77b16db43b99d985ab, b6684bfcd7404fe68d8cb196153412d5; no voters:
I20250109 08:56:59.630359 21238 raft_consensus.cc:2798] T 502ebc25110a4c04a71b8b2e8eea07d4 P ead0b707c1014770b25c5a0db0454830 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:56:59.630445 21210 raft_consensus.cc:2798] T 502ebc25110a4c04a71b8b2e8eea07d4 P 6e2837e083814e77b16db43b99d985ab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:56:59.630893 21238 raft_consensus.cc:491] T 502ebc25110a4c04a71b8b2e8eea07d4 P ead0b707c1014770b25c5a0db0454830 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:56:59.630954 21210 raft_consensus.cc:491] T 502ebc25110a4c04a71b8b2e8eea07d4 P 6e2837e083814e77b16db43b99d985ab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:56:59.631330 21238 raft_consensus.cc:3054] T 502ebc25110a4c04a71b8b2e8eea07d4 P ead0b707c1014770b25c5a0db0454830 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:56:59.634068 21159 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "502ebc25110a4c04a71b8b2e8eea07d4" candidate_uuid: "6e2837e083814e77b16db43b99d985ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ead0b707c1014770b25c5a0db0454830" is_pre_election: true
I20250109 08:56:59.635095 21160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec3e3f891e3c4a879077b9ee897e5d52" candidate_uuid: "6e2837e083814e77b16db43b99d985ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ead0b707c1014770b25c5a0db0454830" is_pre_election: true
I20250109 08:56:59.635751 21160 raft_consensus.cc:2463] T ec3e3f891e3c4a879077b9ee897e5d52 P ead0b707c1014770b25c5a0db0454830 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e2837e083814e77b16db43b99d985ab in term 0.
I20250109 08:56:59.637203 21049 leader_election.cc:304] T ec3e3f891e3c4a879077b9ee897e5d52 P 6e2837e083814e77b16db43b99d985ab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e2837e083814e77b16db43b99d985ab, ead0b707c1014770b25c5a0db0454830; no voters:
I20250109 08:56:59.631403 21210 raft_consensus.cc:3054] T 502ebc25110a4c04a71b8b2e8eea07d4 P 6e2837e083814e77b16db43b99d985ab [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:56:59.639303 21239 raft_consensus.cc:491] T 05148030cdd443048fe5899b03fd92fe P ead0b707c1014770b25c5a0db0454830 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:56:59.639467 21202 raft_consensus.cc:491] T 163310a12b44475ca3727662dad194a6 P b6684bfcd7404fe68d8cb196153412d5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:56:59.617725 21207 raft_consensus.cc:357] T 42af5f76a37645b5a3865739d54a6cb3 P 6e2837e083814e77b16db43b99d985ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:56:59.639762 21239 raft_consensus.cc:513] T 05148030cdd443048fe5899b03fd92fe P ead0b707c1014770b25c5a0db0454830 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:56:59.640530 21207 raft_consensus.cc:383] T 42af5f76a37645b5a3865739d54a6cb3 P 6e2837e083814e77b16db43b99d985ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:56:59.640058 21202 raft_consensus.cc:513] T 163310a12b44475ca3727662dad194a6 P b6684bfcd7404fe68d8cb196153412d5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:56:59.640811 21207 raft_consensus.cc:738] T 42af5f76a37645b5a3865739d54a6cb3 P 6e2837e083814e77b16db43b99d985ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e2837e083814e77b16db43b99d985ab, State: Initialized, Role: FOLLOWER
I20250109 08:56:59.642246 21202 leader_election.cc:290] T 163310a12b44475ca3727662dad194a6 P b6684bfcd7404fe68d8cb196153412d5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e2837e083814e77b16db43b99d985ab (127.19.226.194:44083), ead0b707c1014770b25c5a0db0454830 (127.19.226.195:35749)
I20250109 08:56:59.642105 21207 consensus_queue.cc:260] T 42af5f76a37645b5a3865739d54a6cb3 P 6e2837e083814e77b16db43b99d985ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:56:59.643602 21201 ts_tablet_manager.cc:1428] T 502ebc25110a4c04a71b8b2e8eea07d4 P ead0b707c1014770b25c5a0db0454830: Time spent starting tablet: real 0.078s user 0.018s sys 0.037s
I20250109 08:56:59.644351 21242 raft_consensus.cc:2798] T ec3e3f891e3c4a879077b9ee897e5d52 P 6e2837e083814e77b16db43b99d985ab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:56:59.644564 21201 tablet_bootstrap.cc:492] T 166b00d1332d4f39b57ae9f58cb7fc5f P ead0b707c1014770b25c5a0db0454830: Bootstrap starting.
I20250109 08:56:59.647521 21242 raft_consensus.cc:491] T ec3e3f891e3c4a879077b9ee897e5d52 P 6e2837e083814e77b16db43b99d985ab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:56:59.648319 21242 raft_consensus.cc:3054] T ec3e3f891e3c4a879077b9ee897e5d52 P 6e2837e083814e77b16db43b99d985ab [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:56:59.649554 21210 raft_consensus.cc:513] T 502ebc25110a4c04a71b8b2e8eea07d4 P 6e2837e083814e77b16db43b99d985ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } }
I20250109 08:56:59.639982 21238 raft_consensus.cc:513] T 502ebc25110a4c04a71b8b2e8eea07d4 P ead0b707c1014770b25c5a0db0454830 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } }
I20250109 08:56:59.645622 21086 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "163310a12b44475ca3727662dad194a6" candidate_uuid: "b6684bfcd7404fe68d8cb196153412d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e2837e083814e77b16db43b99d985ab" is_pre_election: true
I20250109 08:56:59.650652 21086 raft_consensus.cc:2463] T 163310a12b44475ca3727662dad194a6 P 6e2837e083814e77b16db43b99d985ab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b6684bfcd7404fe68d8cb196153412d5 in term 0.
I20250109 08:56:59.650923 21159 raft_consensus.cc:2388] T 502ebc25110a4c04a71b8b2e8eea07d4 P ead0b707c1014770b25c5a0db0454830 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e2837e083814e77b16db43b99d985ab in current term 1: Already voted for candidate ead0b707c1014770b25c5a0db0454830 in this term.
I20250109 08:56:59.651651 20974 leader_election.cc:304] T 163310a12b44475ca3727662dad194a6 P b6684bfcd7404fe68d8cb196153412d5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e2837e083814e77b16db43b99d985ab, b6684bfcd7404fe68d8cb196153412d5; no voters:
I20250109 08:56:59.652504 21202 raft_consensus.cc:2798] T 163310a12b44475ca3727662dad194a6 P b6684bfcd7404fe68d8cb196153412d5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:56:59.652894 21202 raft_consensus.cc:491] T 163310a12b44475ca3727662dad194a6 P b6684bfcd7404fe68d8cb196153412d5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:56:59.653213 21201 tablet_bootstrap.cc:654] T 166b00d1332d4f39b57ae9f58cb7fc5f P ead0b707c1014770b25c5a0db0454830: Neither blocks nor log segments found. Creating new log.
I20250109 08:56:59.653231 21202 raft_consensus.cc:3054] T 163310a12b44475ca3727662dad194a6 P b6684bfcd7404fe68d8cb196153412d5 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:56:59.653904 21086 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05148030cdd443048fe5899b03fd92fe" candidate_uuid: "ead0b707c1014770b25c5a0db0454830" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e2837e083814e77b16db43b99d985ab" is_pre_election: true
W20250109 08:56:59.655030 21123 leader_election.cc:343] T 05148030cdd443048fe5899b03fd92fe P ead0b707c1014770b25c5a0db0454830 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6e2837e083814e77b16db43b99d985ab (127.19.226.194:44083): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:56:59.646718 21207 ts_tablet_manager.cc:1428] T 42af5f76a37645b5a3865739d54a6cb3 P 6e2837e083814e77b16db43b99d985ab: Time spent starting tablet: real 0.031s user 0.008s sys 0.001s
I20250109 08:56:59.644834 21239 leader_election.cc:290] T 05148030cdd443048fe5899b03fd92fe P ead0b707c1014770b25c5a0db0454830 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b6684bfcd7404fe68d8cb196153412d5 (127.19.226.193:34209), 6e2837e083814e77b16db43b99d985ab (127.19.226.194:44083)
I20250109 08:56:59.646612 21199 tablet_bootstrap.cc:492] T af13c7179cc24b258e516eceb2b0cac0 P b6684bfcd7404fe68d8cb196153412d5: No bootstrap required, opened a new log
I20250109 08:56:59.656025 21199 ts_tablet_manager.cc:1397] T af13c7179cc24b258e516eceb2b0cac0 P b6684bfcd7404fe68d8cb196153412d5: Time spent bootstrapping tablet: real 0.039s user 0.006s sys 0.007s
W20250109 08:56:59.655277 21125 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.226.193:34209, user_credentials={real_user=slave}} blocked reactor thread for 54720.5us
I20250109 08:56:59.657248 21238 leader_election.cc:290] T 502ebc25110a4c04a71b8b2e8eea07d4 P ead0b707c1014770b25c5a0db0454830 [CANDIDATE]: Term 1 election: Requested vote from peers b6684bfcd7404fe68d8cb196153412d5 (127.19.226.193:34209), 6e2837e083814e77b16db43b99d985ab (127.19.226.194:44083)
I20250109 08:56:59.659466 21010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05148030cdd443048fe5899b03fd92fe" candidate_uuid: "ead0b707c1014770b25c5a0db0454830" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6684bfcd7404fe68d8cb196153412d5" is_pre_election: true
I20250109 08:56:59.660152 21010 raft_consensus.cc:2463] T 05148030cdd443048fe5899b03fd92fe P b6684bfcd7404fe68d8cb196153412d5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ead0b707c1014770b25c5a0db0454830 in term 0.
I20250109 08:56:59.662559 21086 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "502ebc25110a4c04a71b8b2e8eea07d4" candidate_uuid: "ead0b707c1014770b25c5a0db0454830" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e2837e083814e77b16db43b99d985ab"
I20250109 08:56:59.663281 21086 raft_consensus.cc:2388] T 502ebc25110a4c04a71b8b2e8eea07d4 P 6e2837e083814e77b16db43b99d985ab [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate ead0b707c1014770b25c5a0db0454830 in current term 1: Already voted for candidate 6e2837e083814e77b16db43b99d985ab in this term.
I20250109 08:56:59.663395 21125 leader_election.cc:304] T 05148030cdd443048fe5899b03fd92fe P ead0b707c1014770b25c5a0db0454830 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b6684bfcd7404fe68d8cb196153412d5, ead0b707c1014770b25c5a0db0454830; no voters: 6e2837e083814e77b16db43b99d985ab
I20250109 08:56:59.663811 21160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "502ebc25110a4c04a71b8b2e8eea07d4" candidate_uuid: "6e2837e083814e77b16db43b99d985ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ead0b707c1014770b25c5a0db0454830"
I20250109 08:56:59.664057 21011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "502ebc25110a4c04a71b8b2e8eea07d4" candidate_uuid: "ead0b707c1014770b25c5a0db0454830" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6684bfcd7404fe68d8cb196153412d5"
I20250109 08:56:59.664628 21011 raft_consensus.cc:3054] T 502ebc25110a4c04a71b8b2e8eea07d4 P b6684bfcd7404fe68d8cb196153412d5 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:56:59.664649 21160 raft_consensus.cc:2388] T 502ebc25110a4c04a71b8b2e8eea07d4 P ead0b707c1014770b25c5a0db0454830 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 6e2837e083814e77b16db43b99d985ab in current term 1: Already voted for candidate ead0b707c1014770b25c5a0db0454830 in this term.
I20250109 08:56:59.666033 21238 raft_consensus.cc:2798] T 05148030cdd443048fe5899b03fd92fe P ead0b707c1014770b25c5a0db0454830 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:56:59.671036 21010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "502ebc25110a4c04a71b8b2e8eea07d4" candidate_uuid: "6e2837e083814e77b16db43b99d985ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6684bfcd7404fe68d8cb196153412d5"
I20250109 08:56:59.666572 21210 leader_election.cc:290] T 502ebc25110a4c04a71b8b2e8eea07d4 P 6e2837e083814e77b16db43b99d985ab [CANDIDATE]: Term 1 election: Requested vote from peers b6684bfcd7404fe68d8cb196153412d5 (127.19.226.193:34209), ead0b707c1014770b25c5a0db0454830 (127.19.226.195:35749)
I20250109 08:56:59.677595 21159 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "163310a12b44475ca3727662dad194a6" candidate_uuid: "b6684bfcd7404fe68d8cb196153412d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ead0b707c1014770b25c5a0db0454830" is_pre_election: true
I20250109 08:56:59.675122 21011 raft_consensus.cc:2463] T 502ebc25110a4c04a71b8b2e8eea07d4 P b6684bfcd7404fe68d8cb196153412d5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ead0b707c1014770b25c5a0db0454830 in term 1.
I20250109 08:56:59.677963 21051 leader_election.cc:304] T 502ebc25110a4c04a71b8b2e8eea07d4 P 6e2837e083814e77b16db43b99d985ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e2837e083814e77b16db43b99d985ab; no voters: b6684bfcd7404fe68d8cb196153412d5, ead0b707c1014770b25c5a0db0454830
I20250109 08:56:59.679698 21159 raft_consensus.cc:2463] T 163310a12b44475ca3727662dad194a6 P ead0b707c1014770b25c5a0db0454830 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b6684bfcd7404fe68d8cb196153412d5 in term 0.
I20250109 08:56:59.679847 21210 raft_consensus.cc:2743] T 502ebc25110a4c04a71b8b2e8eea07d4 P 6e2837e083814e77b16db43b99d985ab [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20250109 08:56:59.675993 21238 raft_consensus.cc:491] T 05148030cdd443048fe5899b03fd92fe P ead0b707c1014770b25c5a0db0454830 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:56:59.681208 21238 raft_consensus.cc:3054] T 05148030cdd443048fe5899b03fd92fe P ead0b707c1014770b25c5a0db0454830 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:56:59.682793 21207 tablet_bootstrap.cc:492] T 05148030cdd443048fe5899b03fd92fe P 6e2837e083814e77b16db43b99d985ab: Bootstrap starting.
I20250109 08:56:59.677152 21202 raft_consensus.cc:513] T 163310a12b44475ca3727662dad194a6 P b6684bfcd7404fe68d8cb196153412d5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:56:59.685099 21199 raft_consensus.cc:357] T af13c7179cc24b258e516eceb2b0cac0 P b6684bfcd7404fe68d8cb196153412d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:56:59.685768 21199 raft_consensus.cc:383] T af13c7179cc24b258e516eceb2b0cac0 P b6684bfcd7404fe68d8cb196153412d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:56:59.686020 21199 raft_consensus.cc:738] T af13c7179cc24b258e516eceb2b0cac0 P b6684bfcd7404fe68d8cb196153412d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6684bfcd7404fe68d8cb196153412d5, State: Initialized, Role: FOLLOWER
I20250109 08:56:59.686609 21199 consensus_queue.cc:260] T af13c7179cc24b258e516eceb2b0cac0 P b6684bfcd7404fe68d8cb196153412d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:56:59.687323 21086 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "163310a12b44475ca3727662dad194a6" candidate_uuid: "b6684bfcd7404fe68d8cb196153412d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e2837e083814e77b16db43b99d985ab"
I20250109 08:56:59.687573 21159 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "163310a12b44475ca3727662dad194a6" candidate_uuid: "b6684bfcd7404fe68d8cb196153412d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ead0b707c1014770b25c5a0db0454830"
I20250109 08:56:59.688120 21159 raft_consensus.cc:3054] T 163310a12b44475ca3727662dad194a6 P ead0b707c1014770b25c5a0db0454830 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:56:59.690891 20906 catalog_manager.cc:5526] T 53b47173fb6a4a358c8ad9c2df0f6f18 P ead0b707c1014770b25c5a0db0454830 reported cstate change: term changed from 0 to 1, leader changed from <none> to ead0b707c1014770b25c5a0db0454830 (127.19.226.195). New cstate: current_term: 1 leader_uuid: "ead0b707c1014770b25c5a0db0454830" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } health_report { overall_health: HEALTHY } } }
I20250109 08:56:59.692767 21125 leader_election.cc:304] T 502ebc25110a4c04a71b8b2e8eea07d4 P ead0b707c1014770b25c5a0db0454830 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b6684bfcd7404fe68d8cb196153412d5, ead0b707c1014770b25c5a0db0454830; no voters: 6e2837e083814e77b16db43b99d985ab
I20250109 08:56:59.693109 21242 raft_consensus.cc:513] T ec3e3f891e3c4a879077b9ee897e5d52 P 6e2837e083814e77b16db43b99d985ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:56:59.694216 21086 raft_consensus.cc:3054] T 163310a12b44475ca3727662dad194a6 P 6e2837e083814e77b16db43b99d985ab [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:56:59.696054 21011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec3e3f891e3c4a879077b9ee897e5d52" candidate_uuid: "6e2837e083814e77b16db43b99d985ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6684bfcd7404fe68d8cb196153412d5"
I20250109 08:56:59.696831 21011 raft_consensus.cc:3054] T ec3e3f891e3c4a879077b9ee897e5d52 P b6684bfcd7404fe68d8cb196153412d5 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:56:59.697151 21207 tablet_bootstrap.cc:654] T 05148030cdd443048fe5899b03fd92fe P 6e2837e083814e77b16db43b99d985ab: Neither blocks nor log segments found. Creating new log.
I20250109 08:56:59.693184 21244 raft_consensus.cc:491] T 166b00d1332d4f39b57ae9f58cb7fc5f P b6684bfcd7404fe68d8cb196153412d5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:56:59.700683 21206 raft_consensus.cc:2798] T 502ebc25110a4c04a71b8b2e8eea07d4 P ead0b707c1014770b25c5a0db0454830 [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:56:59.700479 21244 raft_consensus.cc:513] T 166b00d1332d4f39b57ae9f58cb7fc5f P b6684bfcd7404fe68d8cb196153412d5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:56:59.703279 21242 leader_election.cc:290] T ec3e3f891e3c4a879077b9ee897e5d52 P 6e2837e083814e77b16db43b99d985ab [CANDIDATE]: Term 1 election: Requested vote from peers b6684bfcd7404fe68d8cb196153412d5 (127.19.226.193:34209), ead0b707c1014770b25c5a0db0454830 (127.19.226.195:35749)
I20250109 08:56:59.703534 21160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec3e3f891e3c4a879077b9ee897e5d52" candidate_uuid: "6e2837e083814e77b16db43b99d985ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ead0b707c1014770b25c5a0db0454830"
I20250109 08:56:59.704138 21160 raft_consensus.cc:3054] T ec3e3f891e3c4a879077b9ee897e5d52 P ead0b707c1014770b25c5a0db0454830 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:56:59.705885 21238 raft_consensus.cc:513] T 05148030cdd443048fe5899b03fd92fe P ead0b707c1014770b25c5a0db0454830 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:56:59.709916 21010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05148030cdd443048fe5899b03fd92fe" candidate_uuid: "ead0b707c1014770b25c5a0db0454830" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6684bfcd7404fe68d8cb196153412d5"
I20250109 08:56:59.710603 21010 raft_consensus.cc:3054] T 05148030cdd443048fe5899b03fd92fe P b6684bfcd7404fe68d8cb196153412d5 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:56:59.711714 21239 raft_consensus.cc:491] T cb177a6fcf4144e2ae1a560eb627e276 P ead0b707c1014770b25c5a0db0454830 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:56:59.711891 21244 leader_election.cc:290] T 166b00d1332d4f39b57ae9f58cb7fc5f P b6684bfcd7404fe68d8cb196153412d5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e2837e083814e77b16db43b99d985ab (127.19.226.194:44083), ead0b707c1014770b25c5a0db0454830 (127.19.226.195:35749)
I20250109 08:56:59.713915 21085 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "166b00d1332d4f39b57ae9f58cb7fc5f" candidate_uuid: "b6684bfcd7404fe68d8cb196153412d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e2837e083814e77b16db43b99d985ab" is_pre_election: true
I20250109 08:56:59.714619 21085 raft_consensus.cc:2463] T 166b00d1332d4f39b57ae9f58cb7fc5f P 6e2837e083814e77b16db43b99d985ab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b6684bfcd7404fe68d8cb196153412d5 in term 0.
I20250109 08:56:59.715864 20974 leader_election.cc:304] T 166b00d1332d4f39b57ae9f58cb7fc5f P b6684bfcd7404fe68d8cb196153412d5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e2837e083814e77b16db43b99d985ab, b6684bfcd7404fe68d8cb196153412d5; no voters:
I20250109 08:56:59.716336 21158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "166b00d1332d4f39b57ae9f58cb7fc5f" candidate_uuid: "b6684bfcd7404fe68d8cb196153412d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ead0b707c1014770b25c5a0db0454830" is_pre_election: true
I20250109 08:56:59.712188 21239 raft_consensus.cc:513] T cb177a6fcf4144e2ae1a560eb627e276 P ead0b707c1014770b25c5a0db0454830 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:56:59.701246 21206 raft_consensus.cc:695] T 502ebc25110a4c04a71b8b2e8eea07d4 P ead0b707c1014770b25c5a0db0454830 [term 1 LEADER]: Becoming Leader. State: Replica: ead0b707c1014770b25c5a0db0454830, State: Running, Role: LEADER
I20250109 08:56:59.693084 21159 raft_consensus.cc:2463] T 163310a12b44475ca3727662dad194a6 P ead0b707c1014770b25c5a0db0454830 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b6684bfcd7404fe68d8cb196153412d5 in term 1.
I20250109 08:56:59.692780 21202 leader_election.cc:290] T 163310a12b44475ca3727662dad194a6 P b6684bfcd7404fe68d8cb196153412d5 [CANDIDATE]: Term 1 election: Requested vote from peers 6e2837e083814e77b16db43b99d985ab (127.19.226.194:44083), ead0b707c1014770b25c5a0db0454830 (127.19.226.195:35749)
I20250109 08:56:59.725858 21084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb177a6fcf4144e2ae1a560eb627e276" candidate_uuid: "ead0b707c1014770b25c5a0db0454830" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e2837e083814e77b16db43b99d985ab" is_pre_election: true
W20250109 08:56:59.726194 20974 leader_election.cc:343] T 166b00d1332d4f39b57ae9f58cb7fc5f P b6684bfcd7404fe68d8cb196153412d5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ead0b707c1014770b25c5a0db0454830 (127.19.226.195:35749): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:56:59.726536 21084 raft_consensus.cc:2463] T cb177a6fcf4144e2ae1a560eb627e276 P 6e2837e083814e77b16db43b99d985ab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ead0b707c1014770b25c5a0db0454830 in term 0.
I20250109 08:56:59.720266 21085 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05148030cdd443048fe5899b03fd92fe" candidate_uuid: "ead0b707c1014770b25c5a0db0454830" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e2837e083814e77b16db43b99d985ab"
I20250109 08:56:59.727097 20974 leader_election.cc:304] T 163310a12b44475ca3727662dad194a6 P b6684bfcd7404fe68d8cb196153412d5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b6684bfcd7404fe68d8cb196153412d5, ead0b707c1014770b25c5a0db0454830; no voters:
I20250109 08:56:59.727962 21086 raft_consensus.cc:2463] T 163310a12b44475ca3727662dad194a6 P 6e2837e083814e77b16db43b99d985ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b6684bfcd7404fe68d8cb196153412d5 in term 1.
I20250109 08:56:59.723946 21239 leader_election.cc:290] T cb177a6fcf4144e2ae1a560eb627e276 P ead0b707c1014770b25c5a0db0454830 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b6684bfcd7404fe68d8cb196153412d5 (127.19.226.193:34209), 6e2837e083814e77b16db43b99d985ab (127.19.226.194:44083)
I20250109 08:56:59.724099 21011 raft_consensus.cc:2463] T ec3e3f891e3c4a879077b9ee897e5d52 P b6684bfcd7404fe68d8cb196153412d5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e2837e083814e77b16db43b99d985ab in term 1.
I20250109 08:56:59.724978 21206 consensus_queue.cc:237] T 502ebc25110a4c04a71b8b2e8eea07d4 P ead0b707c1014770b25c5a0db0454830 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } }
I20250109 08:56:59.727907 21202 raft_consensus.cc:2798] T 163310a12b44475ca3727662dad194a6 P b6684bfcd7404fe68d8cb196153412d5 [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:56:59.736799 21202 raft_consensus.cc:695] T 163310a12b44475ca3727662dad194a6 P b6684bfcd7404fe68d8cb196153412d5 [term 1 LEADER]: Becoming Leader. State: Replica: b6684bfcd7404fe68d8cb196153412d5, State: Running, Role: LEADER
I20250109 08:56:59.737457 21051 leader_election.cc:304] T ec3e3f891e3c4a879077b9ee897e5d52 P 6e2837e083814e77b16db43b99d985ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e2837e083814e77b16db43b99d985ab, b6684bfcd7404fe68d8cb196153412d5; no voters:
I20250109 08:56:59.720821 21238 leader_election.cc:290] T 05148030cdd443048fe5899b03fd92fe P ead0b707c1014770b25c5a0db0454830 [CANDIDATE]: Term 1 election: Requested vote from peers b6684bfcd7404fe68d8cb196153412d5 (127.19.226.193:34209), 6e2837e083814e77b16db43b99d985ab (127.19.226.194:44083)
I20250109 08:56:59.728721 21009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb177a6fcf4144e2ae1a560eb627e276" candidate_uuid: "ead0b707c1014770b25c5a0db0454830" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6684bfcd7404fe68d8cb196153412d5" is_pre_election: true
I20250109 08:56:59.729955 21242 raft_consensus.cc:491] T 948e27b3034d44338e2b7d74f96374f7 P 6e2837e083814e77b16db43b99d985ab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:56:59.729476 21123 leader_election.cc:304] T cb177a6fcf4144e2ae1a560eb627e276 P ead0b707c1014770b25c5a0db0454830 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e2837e083814e77b16db43b99d985ab, ead0b707c1014770b25c5a0db0454830; no voters:
I20250109 08:56:59.731695 21199 ts_tablet_manager.cc:1428] T af13c7179cc24b258e516eceb2b0cac0 P b6684bfcd7404fe68d8cb196153412d5: Time spent starting tablet: real 0.075s user 0.024s sys 0.005s
I20250109 08:56:59.730683 21244 raft_consensus.cc:2798] T 166b00d1332d4f39b57ae9f58cb7fc5f P b6684bfcd7404fe68d8cb196153412d5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:56:59.731812 21160 raft_consensus.cc:2463] T ec3e3f891e3c4a879077b9ee897e5d52 P ead0b707c1014770b25c5a0db0454830 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e2837e083814e77b16db43b99d985ab in term 1.
I20250109 08:56:59.739415 21210 raft_consensus.cc:2798] T ec3e3f891e3c4a879077b9ee897e5d52 P 6e2837e083814e77b16db43b99d985ab [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:56:59.740386 21210 raft_consensus.cc:695] T ec3e3f891e3c4a879077b9ee897e5d52 P 6e2837e083814e77b16db43b99d985ab [term 1 LEADER]: Becoming Leader. State: Replica: 6e2837e083814e77b16db43b99d985ab, State: Running, Role: LEADER
I20250109 08:56:59.738161 21202 consensus_queue.cc:237] T 163310a12b44475ca3727662dad194a6 P b6684bfcd7404fe68d8cb196153412d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:56:59.740789 21009 raft_consensus.cc:2463] T cb177a6fcf4144e2ae1a560eb627e276 P b6684bfcd7404fe68d8cb196153412d5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ead0b707c1014770b25c5a0db0454830 in term 0.
W20250109 08:56:59.741169 21123 leader_election.cc:343] T 05148030cdd443048fe5899b03fd92fe P ead0b707c1014770b25c5a0db0454830 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6e2837e083814e77b16db43b99d985ab (127.19.226.194:44083): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:56:59.740000 21242 raft_consensus.cc:513] T 948e27b3034d44338e2b7d74f96374f7 P 6e2837e083814e77b16db43b99d985ab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:56:59.741175 21210 consensus_queue.cc:237] T ec3e3f891e3c4a879077b9ee897e5d52 P 6e2837e083814e77b16db43b99d985ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:56:59.743361 21242 leader_election.cc:290] T 948e27b3034d44338e2b7d74f96374f7 P 6e2837e083814e77b16db43b99d985ab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b6684bfcd7404fe68d8cb196153412d5 (127.19.226.193:34209), ead0b707c1014770b25c5a0db0454830 (127.19.226.195:35749)
I20250109 08:56:59.743824 21239 raft_consensus.cc:491] T af13c7179cc24b258e516eceb2b0cac0 P ead0b707c1014770b25c5a0db0454830 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:56:59.744251 21239 raft_consensus.cc:513] T af13c7179cc24b258e516eceb2b0cac0 P ead0b707c1014770b25c5a0db0454830 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:56:59.746592 21238 raft_consensus.cc:2798] T cb177a6fcf4144e2ae1a560eb627e276 P ead0b707c1014770b25c5a0db0454830 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:56:59.745201 21009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "948e27b3034d44338e2b7d74f96374f7" candidate_uuid: "6e2837e083814e77b16db43b99d985ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6684bfcd7404fe68d8cb196153412d5" is_pre_election: true
I20250109 08:56:59.746932 21238 raft_consensus.cc:491] T cb177a6fcf4144e2ae1a560eb627e276 P ead0b707c1014770b25c5a0db0454830 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:56:59.742197 21244 raft_consensus.cc:491] T 166b00d1332d4f39b57ae9f58cb7fc5f P b6684bfcd7404fe68d8cb196153412d5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:56:59.747980 21239 leader_election.cc:290] T af13c7179cc24b258e516eceb2b0cac0 P ead0b707c1014770b25c5a0db0454830 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b6684bfcd7404fe68d8cb196153412d5 (127.19.226.193:34209), 6e2837e083814e77b16db43b99d985ab (127.19.226.194:44083)
I20250109 08:56:59.747474 21238 raft_consensus.cc:3054] T cb177a6fcf4144e2ae1a560eb627e276 P ead0b707c1014770b25c5a0db0454830 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:56:59.748246 21160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "948e27b3034d44338e2b7d74f96374f7" candidate_uuid: "6e2837e083814e77b16db43b99d985ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ead0b707c1014770b25c5a0db0454830" is_pre_election: true
I20250109 08:56:59.748499 21244 raft_consensus.cc:3054] T 166b00d1332d4f39b57ae9f58cb7fc5f P b6684bfcd7404fe68d8cb196153412d5 [term 0 FOLLOWER]: Advancing to term 1
W20250109 08:56:59.749577 21049 leader_election.cc:343] T 948e27b3034d44338e2b7d74f96374f7 P 6e2837e083814e77b16db43b99d985ab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ead0b707c1014770b25c5a0db0454830 (127.19.226.195:35749): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:56:59.751178 21011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af13c7179cc24b258e516eceb2b0cac0" candidate_uuid: "ead0b707c1014770b25c5a0db0454830" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6684bfcd7404fe68d8cb196153412d5" is_pre_election: true
I20250109 08:56:59.747527 21009 raft_consensus.cc:2463] T 948e27b3034d44338e2b7d74f96374f7 P b6684bfcd7404fe68d8cb196153412d5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e2837e083814e77b16db43b99d985ab in term 0.
I20250109 08:56:59.751838 21011 raft_consensus.cc:2463] T af13c7179cc24b258e516eceb2b0cac0 P b6684bfcd7404fe68d8cb196153412d5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ead0b707c1014770b25c5a0db0454830 in term 0.
I20250109 08:56:59.752677 21051 leader_election.cc:304] T 948e27b3034d44338e2b7d74f96374f7 P 6e2837e083814e77b16db43b99d985ab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e2837e083814e77b16db43b99d985ab, b6684bfcd7404fe68d8cb196153412d5; no voters: ead0b707c1014770b25c5a0db0454830
I20250109 08:56:59.753741 21125 leader_election.cc:304] T af13c7179cc24b258e516eceb2b0cac0 P ead0b707c1014770b25c5a0db0454830 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b6684bfcd7404fe68d8cb196153412d5, ead0b707c1014770b25c5a0db0454830; no voters:
I20250109 08:56:59.756597 21239 raft_consensus.cc:2798] T af13c7179cc24b258e516eceb2b0cac0 P ead0b707c1014770b25c5a0db0454830 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:56:59.757505 21239 raft_consensus.cc:491] T af13c7179cc24b258e516eceb2b0cac0 P ead0b707c1014770b25c5a0db0454830 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:56:59.757853 21239 raft_consensus.cc:3054] T af13c7179cc24b258e516eceb2b0cac0 P ead0b707c1014770b25c5a0db0454830 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:56:59.758117 21010 raft_consensus.cc:2463] T 05148030cdd443048fe5899b03fd92fe P b6684bfcd7404fe68d8cb196153412d5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ead0b707c1014770b25c5a0db0454830 in term 1.
I20250109 08:56:59.759182 21242 raft_consensus.cc:2798] T 948e27b3034d44338e2b7d74f96374f7 P 6e2837e083814e77b16db43b99d985ab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:56:59.762393 21242 raft_consensus.cc:491] T 948e27b3034d44338e2b7d74f96374f7 P 6e2837e083814e77b16db43b99d985ab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:56:59.762724 21242 raft_consensus.cc:3054] T 948e27b3034d44338e2b7d74f96374f7 P 6e2837e083814e77b16db43b99d985ab [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:56:59.752004 21086 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af13c7179cc24b258e516eceb2b0cac0" candidate_uuid: "ead0b707c1014770b25c5a0db0454830" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e2837e083814e77b16db43b99d985ab" is_pre_election: true
I20250109 08:56:59.764827 21086 raft_consensus.cc:2463] T af13c7179cc24b258e516eceb2b0cac0 P 6e2837e083814e77b16db43b99d985ab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ead0b707c1014770b25c5a0db0454830 in term 0.
I20250109 08:56:59.770687 21242 raft_consensus.cc:513] T 948e27b3034d44338e2b7d74f96374f7 P 6e2837e083814e77b16db43b99d985ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:56:59.775307 21011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "948e27b3034d44338e2b7d74f96374f7" candidate_uuid: "6e2837e083814e77b16db43b99d985ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6684bfcd7404fe68d8cb196153412d5"
I20250109 08:56:59.775931 21011 raft_consensus.cc:3054] T 948e27b3034d44338e2b7d74f96374f7 P b6684bfcd7404fe68d8cb196153412d5 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:56:59.778013 21239 raft_consensus.cc:513] T af13c7179cc24b258e516eceb2b0cac0 P ead0b707c1014770b25c5a0db0454830 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:56:59.781352 21125 leader_election.cc:304] T 05148030cdd443048fe5899b03fd92fe P ead0b707c1014770b25c5a0db0454830 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b6684bfcd7404fe68d8cb196153412d5, ead0b707c1014770b25c5a0db0454830; no voters: 6e2837e083814e77b16db43b99d985ab
I20250109 08:56:59.781848 21239 leader_election.cc:290] T af13c7179cc24b258e516eceb2b0cac0 P ead0b707c1014770b25c5a0db0454830 [CANDIDATE]: Term 1 election: Requested vote from peers b6684bfcd7404fe68d8cb196153412d5 (127.19.226.193:34209), 6e2837e083814e77b16db43b99d985ab (127.19.226.194:44083)
I20250109 08:56:59.782480 21239 raft_consensus.cc:2798] T 05148030cdd443048fe5899b03fd92fe P ead0b707c1014770b25c5a0db0454830 [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:56:59.782929 21239 raft_consensus.cc:695] T 05148030cdd443048fe5899b03fd92fe P ead0b707c1014770b25c5a0db0454830 [term 1 LEADER]: Becoming Leader. State: Replica: ead0b707c1014770b25c5a0db0454830, State: Running, Role: LEADER
I20250109 08:56:59.783819 21086 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af13c7179cc24b258e516eceb2b0cac0" candidate_uuid: "ead0b707c1014770b25c5a0db0454830" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e2837e083814e77b16db43b99d985ab"
I20250109 08:56:59.784615 21086 raft_consensus.cc:3054] T af13c7179cc24b258e516eceb2b0cac0 P 6e2837e083814e77b16db43b99d985ab [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:56:59.790058 21238 raft_consensus.cc:513] T cb177a6fcf4144e2ae1a560eb627e276 P ead0b707c1014770b25c5a0db0454830 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:56:59.785261 21010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af13c7179cc24b258e516eceb2b0cac0" candidate_uuid: "ead0b707c1014770b25c5a0db0454830" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6684bfcd7404fe68d8cb196153412d5"
I20250109 08:56:59.785920 21201 tablet_bootstrap.cc:492] T 166b00d1332d4f39b57ae9f58cb7fc5f P ead0b707c1014770b25c5a0db0454830: No bootstrap required, opened a new log
I20250109 08:56:59.791285 21201 ts_tablet_manager.cc:1397] T 166b00d1332d4f39b57ae9f58cb7fc5f P ead0b707c1014770b25c5a0db0454830: Time spent bootstrapping tablet: real 0.147s user 0.013s sys 0.005s
I20250109 08:56:59.795038 21201 raft_consensus.cc:357] T 166b00d1332d4f39b57ae9f58cb7fc5f P ead0b707c1014770b25c5a0db0454830 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:56:59.795869 21201 raft_consensus.cc:383] T 166b00d1332d4f39b57ae9f58cb7fc5f P ead0b707c1014770b25c5a0db0454830 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:56:59.796190 21201 raft_consensus.cc:738] T 166b00d1332d4f39b57ae9f58cb7fc5f P ead0b707c1014770b25c5a0db0454830 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ead0b707c1014770b25c5a0db0454830, State: Initialized, Role: FOLLOWER
I20250109 08:56:59.797235 21011 raft_consensus.cc:2463] T 948e27b3034d44338e2b7d74f96374f7 P b6684bfcd7404fe68d8cb196153412d5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e2837e083814e77b16db43b99d985ab in term 1.
I20250109 08:56:59.792961 21009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb177a6fcf4144e2ae1a560eb627e276" candidate_uuid: "ead0b707c1014770b25c5a0db0454830" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6684bfcd7404fe68d8cb196153412d5"
I20250109 08:56:59.798309 21009 raft_consensus.cc:3054] T cb177a6fcf4144e2ae1a560eb627e276 P b6684bfcd7404fe68d8cb196153412d5 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:56:59.796891 21010 raft_consensus.cc:3054] T af13c7179cc24b258e516eceb2b0cac0 P b6684bfcd7404fe68d8cb196153412d5 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:56:59.801534 21201 consensus_queue.cc:260] T 166b00d1332d4f39b57ae9f58cb7fc5f P ead0b707c1014770b25c5a0db0454830 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:56:59.783769 21239 consensus_queue.cc:237] T 05148030cdd443048fe5899b03fd92fe P ead0b707c1014770b25c5a0db0454830 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:56:59.804963 21236 raft_consensus.cc:491] T 166b00d1332d4f39b57ae9f58cb7fc5f P ead0b707c1014770b25c5a0db0454830 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:56:59.803107 21238 leader_election.cc:290] T cb177a6fcf4144e2ae1a560eb627e276 P ead0b707c1014770b25c5a0db0454830 [CANDIDATE]: Term 1 election: Requested vote from peers b6684bfcd7404fe68d8cb196153412d5 (127.19.226.193:34209), 6e2837e083814e77b16db43b99d985ab (127.19.226.194:44083)
I20250109 08:56:59.805433 21236 raft_consensus.cc:513] T 166b00d1332d4f39b57ae9f58cb7fc5f P ead0b707c1014770b25c5a0db0454830 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:56:59.798436 21051 leader_election.cc:304] T 948e27b3034d44338e2b7d74f96374f7 P 6e2837e083814e77b16db43b99d985ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e2837e083814e77b16db43b99d985ab, b6684bfcd7404fe68d8cb196153412d5; no voters:
I20250109 08:56:59.793903 21244 raft_consensus.cc:513] T 166b00d1332d4f39b57ae9f58cb7fc5f P b6684bfcd7404fe68d8cb196153412d5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:56:59.809978 21201 ts_tablet_manager.cc:1428] T 166b00d1332d4f39b57ae9f58cb7fc5f P ead0b707c1014770b25c5a0db0454830: Time spent starting tablet: real 0.018s user 0.006s sys 0.000s
I20250109 08:56:59.812990 21201 tablet_bootstrap.cc:492] T 948e27b3034d44338e2b7d74f96374f7 P ead0b707c1014770b25c5a0db0454830: Bootstrap starting.
I20250109 08:56:59.811818 21160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "948e27b3034d44338e2b7d74f96374f7" candidate_uuid: "6e2837e083814e77b16db43b99d985ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ead0b707c1014770b25c5a0db0454830"
I20250109 08:56:59.814527 21236 leader_election.cc:290] T 166b00d1332d4f39b57ae9f58cb7fc5f P ead0b707c1014770b25c5a0db0454830 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b6684bfcd7404fe68d8cb196153412d5 (127.19.226.193:34209), 6e2837e083814e77b16db43b99d985ab (127.19.226.194:44083)
I20250109 08:56:59.816970 21085 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb177a6fcf4144e2ae1a560eb627e276" candidate_uuid: "ead0b707c1014770b25c5a0db0454830" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e2837e083814e77b16db43b99d985ab"
I20250109 08:56:59.817602 21085 raft_consensus.cc:3054] T cb177a6fcf4144e2ae1a560eb627e276 P 6e2837e083814e77b16db43b99d985ab [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:56:59.820497 21207 tablet_bootstrap.cc:492] T 05148030cdd443048fe5899b03fd92fe P 6e2837e083814e77b16db43b99d985ab: No bootstrap required, opened a new log
I20250109 08:56:59.820922 21207 ts_tablet_manager.cc:1397] T 05148030cdd443048fe5899b03fd92fe P 6e2837e083814e77b16db43b99d985ab: Time spent bootstrapping tablet: real 0.138s user 0.012s sys 0.005s
I20250109 08:56:59.815889 21210 raft_consensus.cc:2798] T 948e27b3034d44338e2b7d74f96374f7 P 6e2837e083814e77b16db43b99d985ab [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:56:59.809132 21242 leader_election.cc:290] T 948e27b3034d44338e2b7d74f96374f7 P 6e2837e083814e77b16db43b99d985ab [CANDIDATE]: Term 1 election: Requested vote from peers b6684bfcd7404fe68d8cb196153412d5 (127.19.226.193:34209), ead0b707c1014770b25c5a0db0454830 (127.19.226.195:35749)
I20250109 08:56:59.826607 21207 raft_consensus.cc:357] T 05148030cdd443048fe5899b03fd92fe P 6e2837e083814e77b16db43b99d985ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:56:59.827405 21207 raft_consensus.cc:383] T 05148030cdd443048fe5899b03fd92fe P 6e2837e083814e77b16db43b99d985ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:56:59.827685 21207 raft_consensus.cc:738] T 05148030cdd443048fe5899b03fd92fe P 6e2837e083814e77b16db43b99d985ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e2837e083814e77b16db43b99d985ab, State: Initialized, Role: FOLLOWER
I20250109 08:56:59.828303 21207 consensus_queue.cc:260] T 05148030cdd443048fe5899b03fd92fe P 6e2837e083814e77b16db43b99d985ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
W20250109 08:56:59.825388 21049 leader_election.cc:343] T 948e27b3034d44338e2b7d74f96374f7 P 6e2837e083814e77b16db43b99d985ab [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ead0b707c1014770b25c5a0db0454830 (127.19.226.195:35749): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:56:59.835803 21084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "166b00d1332d4f39b57ae9f58cb7fc5f" candidate_uuid: "ead0b707c1014770b25c5a0db0454830" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e2837e083814e77b16db43b99d985ab" is_pre_election: true
I20250109 08:56:59.840844 21084 raft_consensus.cc:2463] T 166b00d1332d4f39b57ae9f58cb7fc5f P 6e2837e083814e77b16db43b99d985ab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ead0b707c1014770b25c5a0db0454830 in term 0.
I20250109 08:56:59.831379 21210 raft_consensus.cc:695] T 948e27b3034d44338e2b7d74f96374f7 P 6e2837e083814e77b16db43b99d985ab [term 1 LEADER]: Becoming Leader. State: Replica: 6e2837e083814e77b16db43b99d985ab, State: Running, Role: LEADER
I20250109 08:56:59.836159 21011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "166b00d1332d4f39b57ae9f58cb7fc5f" candidate_uuid: "ead0b707c1014770b25c5a0db0454830" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6684bfcd7404fe68d8cb196153412d5" is_pre_election: true
I20250109 08:56:59.842016 21123 leader_election.cc:304] T 166b00d1332d4f39b57ae9f58cb7fc5f P ead0b707c1014770b25c5a0db0454830 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e2837e083814e77b16db43b99d985ab, ead0b707c1014770b25c5a0db0454830; no voters:
I20250109 08:56:59.843366 21244 leader_election.cc:290] T 166b00d1332d4f39b57ae9f58cb7fc5f P b6684bfcd7404fe68d8cb196153412d5 [CANDIDATE]: Term 1 election: Requested vote from peers 6e2837e083814e77b16db43b99d985ab (127.19.226.194:44083), ead0b707c1014770b25c5a0db0454830 (127.19.226.195:35749)
I20250109 08:56:59.844405 21084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "166b00d1332d4f39b57ae9f58cb7fc5f" candidate_uuid: "b6684bfcd7404fe68d8cb196153412d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e2837e083814e77b16db43b99d985ab"
I20250109 08:56:59.857023 21236 raft_consensus.cc:2798] T 166b00d1332d4f39b57ae9f58cb7fc5f P ead0b707c1014770b25c5a0db0454830 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:56:59.857214 21084 raft_consensus.cc:3054] T 166b00d1332d4f39b57ae9f58cb7fc5f P 6e2837e083814e77b16db43b99d985ab [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:56:59.857376 21236 raft_consensus.cc:491] T 166b00d1332d4f39b57ae9f58cb7fc5f P ead0b707c1014770b25c5a0db0454830 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:56:59.857721 21236 raft_consensus.cc:3054] T 166b00d1332d4f39b57ae9f58cb7fc5f P ead0b707c1014770b25c5a0db0454830 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:56:59.859470 21011 raft_consensus.cc:2388] T 166b00d1332d4f39b57ae9f58cb7fc5f P b6684bfcd7404fe68d8cb196153412d5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ead0b707c1014770b25c5a0db0454830 in current term 1: Already voted for candidate b6684bfcd7404fe68d8cb196153412d5 in this term.
I20250109 08:56:59.846940 21160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "166b00d1332d4f39b57ae9f58cb7fc5f" candidate_uuid: "b6684bfcd7404fe68d8cb196153412d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ead0b707c1014770b25c5a0db0454830"
I20250109 08:56:59.849236 21085 raft_consensus.cc:2463] T cb177a6fcf4144e2ae1a560eb627e276 P 6e2837e083814e77b16db43b99d985ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ead0b707c1014770b25c5a0db0454830 in term 1.
I20250109 08:56:59.849381 21207 ts_tablet_manager.cc:1428] T 05148030cdd443048fe5899b03fd92fe P 6e2837e083814e77b16db43b99d985ab: Time spent starting tablet: real 0.028s user 0.006s sys 0.000s
I20250109 08:56:59.851790 21009 raft_consensus.cc:2463] T cb177a6fcf4144e2ae1a560eb627e276 P b6684bfcd7404fe68d8cb196153412d5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ead0b707c1014770b25c5a0db0454830 in term 1.
I20250109 08:56:59.861914 21125 leader_election.cc:304] T cb177a6fcf4144e2ae1a560eb627e276 P ead0b707c1014770b25c5a0db0454830 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b6684bfcd7404fe68d8cb196153412d5, ead0b707c1014770b25c5a0db0454830; no voters:
I20250109 08:56:59.851894 21010 raft_consensus.cc:2463] T af13c7179cc24b258e516eceb2b0cac0 P b6684bfcd7404fe68d8cb196153412d5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ead0b707c1014770b25c5a0db0454830 in term 1.
I20250109 08:56:59.853176 21086 raft_consensus.cc:2463] T af13c7179cc24b258e516eceb2b0cac0 P 6e2837e083814e77b16db43b99d985ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ead0b707c1014770b25c5a0db0454830 in term 1.
I20250109 08:56:59.860321 20909 catalog_manager.cc:5526] T ec3e3f891e3c4a879077b9ee897e5d52 P 6e2837e083814e77b16db43b99d985ab reported cstate change: term changed from 0 to 1, leader changed from <none> to 6e2837e083814e77b16db43b99d985ab (127.19.226.194). New cstate: current_term: 1 leader_uuid: "6e2837e083814e77b16db43b99d985ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } health_report { overall_health: UNKNOWN } } }
I20250109 08:56:59.865904 21210 consensus_queue.cc:237] T 948e27b3034d44338e2b7d74f96374f7 P 6e2837e083814e77b16db43b99d985ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:56:59.868335 21123 leader_election.cc:304] T af13c7179cc24b258e516eceb2b0cac0 P ead0b707c1014770b25c5a0db0454830 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e2837e083814e77b16db43b99d985ab, ead0b707c1014770b25c5a0db0454830; no voters:
I20250109 08:56:59.868693 21253 raft_consensus.cc:2798] T cb177a6fcf4144e2ae1a560eb627e276 P ead0b707c1014770b25c5a0db0454830 [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:56:59.869696 21238 raft_consensus.cc:2798] T af13c7179cc24b258e516eceb2b0cac0 P ead0b707c1014770b25c5a0db0454830 [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:56:59.874550 21201 tablet_bootstrap.cc:654] T 948e27b3034d44338e2b7d74f96374f7 P ead0b707c1014770b25c5a0db0454830: Neither blocks nor log segments found. Creating new log.
I20250109 08:56:59.878722 21253 raft_consensus.cc:695] T cb177a6fcf4144e2ae1a560eb627e276 P ead0b707c1014770b25c5a0db0454830 [term 1 LEADER]: Becoming Leader. State: Replica: ead0b707c1014770b25c5a0db0454830, State: Running, Role: LEADER
I20250109 08:56:59.881362 21084 raft_consensus.cc:2463] T 166b00d1332d4f39b57ae9f58cb7fc5f P 6e2837e083814e77b16db43b99d985ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b6684bfcd7404fe68d8cb196153412d5 in term 1.
I20250109 08:56:59.877514 21238 raft_consensus.cc:695] T af13c7179cc24b258e516eceb2b0cac0 P ead0b707c1014770b25c5a0db0454830 [term 1 LEADER]: Becoming Leader. State: Replica: ead0b707c1014770b25c5a0db0454830, State: Running, Role: LEADER
I20250109 08:56:59.883608 20907 catalog_manager.cc:5526] T 163310a12b44475ca3727662dad194a6 P b6684bfcd7404fe68d8cb196153412d5 reported cstate change: term changed from 0 to 1, leader changed from <none> to b6684bfcd7404fe68d8cb196153412d5 (127.19.226.193). New cstate: current_term: 1 leader_uuid: "b6684bfcd7404fe68d8cb196153412d5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } health_report { overall_health: UNKNOWN } } }
I20250109 08:56:59.888831 21236 raft_consensus.cc:513] T 166b00d1332d4f39b57ae9f58cb7fc5f P ead0b707c1014770b25c5a0db0454830 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:56:59.889926 21160 raft_consensus.cc:2388] T 166b00d1332d4f39b57ae9f58cb7fc5f P ead0b707c1014770b25c5a0db0454830 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate b6684bfcd7404fe68d8cb196153412d5 in current term 1: Already voted for candidate ead0b707c1014770b25c5a0db0454830 in this term.
I20250109 08:56:59.884919 21253 consensus_queue.cc:237] T cb177a6fcf4144e2ae1a560eb627e276 P ead0b707c1014770b25c5a0db0454830 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:56:59.887045 20974 leader_election.cc:304] T 166b00d1332d4f39b57ae9f58cb7fc5f P b6684bfcd7404fe68d8cb196153412d5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e2837e083814e77b16db43b99d985ab, b6684bfcd7404fe68d8cb196153412d5; no voters:
I20250109 08:56:59.891103 21244 raft_consensus.cc:2798] T 166b00d1332d4f39b57ae9f58cb7fc5f P b6684bfcd7404fe68d8cb196153412d5 [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:56:59.891541 21244 raft_consensus.cc:695] T 166b00d1332d4f39b57ae9f58cb7fc5f P b6684bfcd7404fe68d8cb196153412d5 [term 1 LEADER]: Becoming Leader. State: Replica: b6684bfcd7404fe68d8cb196153412d5, State: Running, Role: LEADER
I20250109 08:56:59.892278 21244 consensus_queue.cc:237] T 166b00d1332d4f39b57ae9f58cb7fc5f P b6684bfcd7404fe68d8cb196153412d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:56:59.899863 21010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "166b00d1332d4f39b57ae9f58cb7fc5f" candidate_uuid: "ead0b707c1014770b25c5a0db0454830" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6684bfcd7404fe68d8cb196153412d5"
I20250109 08:56:59.902319 20906 catalog_manager.cc:5526] T 502ebc25110a4c04a71b8b2e8eea07d4 P ead0b707c1014770b25c5a0db0454830 reported cstate change: term changed from 0 to 1, leader changed from <none> to ead0b707c1014770b25c5a0db0454830 (127.19.226.195). New cstate: current_term: 1 leader_uuid: "ead0b707c1014770b25c5a0db0454830" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } health_report { overall_health: UNKNOWN } } }
I20250109 08:56:59.897919 21238 consensus_queue.cc:237] T af13c7179cc24b258e516eceb2b0cac0 P ead0b707c1014770b25c5a0db0454830 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:56:59.907348 21236 leader_election.cc:290] T 166b00d1332d4f39b57ae9f58cb7fc5f P ead0b707c1014770b25c5a0db0454830 [CANDIDATE]: Term 1 election: Requested vote from peers b6684bfcd7404fe68d8cb196153412d5 (127.19.226.193:34209), 6e2837e083814e77b16db43b99d985ab (127.19.226.194:44083)
I20250109 08:56:59.907657 21084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "166b00d1332d4f39b57ae9f58cb7fc5f" candidate_uuid: "ead0b707c1014770b25c5a0db0454830" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e2837e083814e77b16db43b99d985ab"
I20250109 08:56:59.908401 21084 raft_consensus.cc:2388] T 166b00d1332d4f39b57ae9f58cb7fc5f P 6e2837e083814e77b16db43b99d985ab [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate ead0b707c1014770b25c5a0db0454830 in current term 1: Already voted for candidate b6684bfcd7404fe68d8cb196153412d5 in this term.
I20250109 08:56:59.909453 21123 leader_election.cc:304] T 166b00d1332d4f39b57ae9f58cb7fc5f P ead0b707c1014770b25c5a0db0454830 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ead0b707c1014770b25c5a0db0454830; no voters: 6e2837e083814e77b16db43b99d985ab, b6684bfcd7404fe68d8cb196153412d5
I20250109 08:56:59.910444 21236 raft_consensus.cc:2743] T 166b00d1332d4f39b57ae9f58cb7fc5f P ead0b707c1014770b25c5a0db0454830 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20250109 08:56:59.911993 20906 catalog_manager.cc:5526] T 05148030cdd443048fe5899b03fd92fe P ead0b707c1014770b25c5a0db0454830 reported cstate change: term changed from 0 to 1, leader changed from <none> to ead0b707c1014770b25c5a0db0454830 (127.19.226.195). New cstate: current_term: 1 leader_uuid: "ead0b707c1014770b25c5a0db0454830" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } health_report { overall_health: HEALTHY } } }
I20250109 08:56:59.926646 21201 tablet_bootstrap.cc:492] T 948e27b3034d44338e2b7d74f96374f7 P ead0b707c1014770b25c5a0db0454830: No bootstrap required, opened a new log
I20250109 08:56:59.927074 21201 ts_tablet_manager.cc:1397] T 948e27b3034d44338e2b7d74f96374f7 P ead0b707c1014770b25c5a0db0454830: Time spent bootstrapping tablet: real 0.114s user 0.019s sys 0.004s
I20250109 08:56:59.927546 20909 catalog_manager.cc:5526] T 166b00d1332d4f39b57ae9f58cb7fc5f P b6684bfcd7404fe68d8cb196153412d5 reported cstate change: term changed from 0 to 1, leader changed from <none> to b6684bfcd7404fe68d8cb196153412d5 (127.19.226.193). New cstate: current_term: 1 leader_uuid: "b6684bfcd7404fe68d8cb196153412d5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } health_report { overall_health: UNKNOWN } } }
I20250109 08:56:59.928828 20908 catalog_manager.cc:5526] T 948e27b3034d44338e2b7d74f96374f7 P 6e2837e083814e77b16db43b99d985ab reported cstate change: term changed from 0 to 1, leader changed from <none> to 6e2837e083814e77b16db43b99d985ab (127.19.226.194). New cstate: current_term: 1 leader_uuid: "6e2837e083814e77b16db43b99d985ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } health_report { overall_health: UNKNOWN } } }
I20250109 08:56:59.929353 21201 raft_consensus.cc:357] T 948e27b3034d44338e2b7d74f96374f7 P ead0b707c1014770b25c5a0db0454830 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:56:59.930024 21201 raft_consensus.cc:383] T 948e27b3034d44338e2b7d74f96374f7 P ead0b707c1014770b25c5a0db0454830 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:56:59.930274 21201 raft_consensus.cc:738] T 948e27b3034d44338e2b7d74f96374f7 P ead0b707c1014770b25c5a0db0454830 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ead0b707c1014770b25c5a0db0454830, State: Initialized, Role: FOLLOWER
I20250109 08:56:59.931021 21201 consensus_queue.cc:260] T 948e27b3034d44338e2b7d74f96374f7 P ead0b707c1014770b25c5a0db0454830 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:56:59.933001 21201 ts_tablet_manager.cc:1428] T 948e27b3034d44338e2b7d74f96374f7 P ead0b707c1014770b25c5a0db0454830: Time spent starting tablet: real 0.006s user 0.005s sys 0.002s
I20250109 08:56:59.932940 21244 raft_consensus.cc:491] T 42af5f76a37645b5a3865739d54a6cb3 P b6684bfcd7404fe68d8cb196153412d5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:56:59.933670 21244 raft_consensus.cc:513] T 42af5f76a37645b5a3865739d54a6cb3 P b6684bfcd7404fe68d8cb196153412d5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:56:59.935545 21244 leader_election.cc:290] T 42af5f76a37645b5a3865739d54a6cb3 P b6684bfcd7404fe68d8cb196153412d5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e2837e083814e77b16db43b99d985ab (127.19.226.194:44083), ead0b707c1014770b25c5a0db0454830 (127.19.226.195:35749)
I20250109 08:56:59.937376 21084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42af5f76a37645b5a3865739d54a6cb3" candidate_uuid: "b6684bfcd7404fe68d8cb196153412d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e2837e083814e77b16db43b99d985ab" is_pre_election: true
I20250109 08:56:59.937701 21160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42af5f76a37645b5a3865739d54a6cb3" candidate_uuid: "b6684bfcd7404fe68d8cb196153412d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ead0b707c1014770b25c5a0db0454830" is_pre_election: true
I20250109 08:56:59.938182 21084 raft_consensus.cc:2463] T 42af5f76a37645b5a3865739d54a6cb3 P 6e2837e083814e77b16db43b99d985ab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b6684bfcd7404fe68d8cb196153412d5 in term 0.
I20250109 08:56:59.938342 21160 raft_consensus.cc:2463] T 42af5f76a37645b5a3865739d54a6cb3 P ead0b707c1014770b25c5a0db0454830 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b6684bfcd7404fe68d8cb196153412d5 in term 0.
I20250109 08:56:59.939275 20974 leader_election.cc:304] T 42af5f76a37645b5a3865739d54a6cb3 P b6684bfcd7404fe68d8cb196153412d5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b6684bfcd7404fe68d8cb196153412d5, ead0b707c1014770b25c5a0db0454830; no voters:
I20250109 08:56:59.939949 21244 raft_consensus.cc:2798] T 42af5f76a37645b5a3865739d54a6cb3 P b6684bfcd7404fe68d8cb196153412d5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:56:59.940300 21244 raft_consensus.cc:491] T 42af5f76a37645b5a3865739d54a6cb3 P b6684bfcd7404fe68d8cb196153412d5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:56:59.940773 21244 raft_consensus.cc:3054] T 42af5f76a37645b5a3865739d54a6cb3 P b6684bfcd7404fe68d8cb196153412d5 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:56:59.940274 20907 catalog_manager.cc:5526] T cb177a6fcf4144e2ae1a560eb627e276 P ead0b707c1014770b25c5a0db0454830 reported cstate change: term changed from 0 to 1, leader changed from <none> to ead0b707c1014770b25c5a0db0454830 (127.19.226.195). New cstate: current_term: 1 leader_uuid: "ead0b707c1014770b25c5a0db0454830" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } health_report { overall_health: HEALTHY } } }
I20250109 08:56:59.942133 20907 catalog_manager.cc:5526] T af13c7179cc24b258e516eceb2b0cac0 P ead0b707c1014770b25c5a0db0454830 reported cstate change: term changed from 0 to 1, leader changed from <none> to ead0b707c1014770b25c5a0db0454830 (127.19.226.195). New cstate: current_term: 1 leader_uuid: "ead0b707c1014770b25c5a0db0454830" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } health_report { overall_health: HEALTHY } } }
I20250109 08:56:59.946398 21244 raft_consensus.cc:513] T 42af5f76a37645b5a3865739d54a6cb3 P b6684bfcd7404fe68d8cb196153412d5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:56:59.947847 21244 leader_election.cc:290] T 42af5f76a37645b5a3865739d54a6cb3 P b6684bfcd7404fe68d8cb196153412d5 [CANDIDATE]: Term 1 election: Requested vote from peers 6e2837e083814e77b16db43b99d985ab (127.19.226.194:44083), ead0b707c1014770b25c5a0db0454830 (127.19.226.195:35749)
I20250109 08:56:59.948865 21160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42af5f76a37645b5a3865739d54a6cb3" candidate_uuid: "b6684bfcd7404fe68d8cb196153412d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ead0b707c1014770b25c5a0db0454830"
I20250109 08:56:59.949455 21160 raft_consensus.cc:3054] T 42af5f76a37645b5a3865739d54a6cb3 P ead0b707c1014770b25c5a0db0454830 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:56:59.948865 21084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42af5f76a37645b5a3865739d54a6cb3" candidate_uuid: "b6684bfcd7404fe68d8cb196153412d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e2837e083814e77b16db43b99d985ab"
I20250109 08:56:59.949963 21084 raft_consensus.cc:3054] T 42af5f76a37645b5a3865739d54a6cb3 P 6e2837e083814e77b16db43b99d985ab [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:56:59.955629 21160 raft_consensus.cc:2463] T 42af5f76a37645b5a3865739d54a6cb3 P ead0b707c1014770b25c5a0db0454830 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b6684bfcd7404fe68d8cb196153412d5 in term 1.
I20250109 08:56:59.955716 21084 raft_consensus.cc:2463] T 42af5f76a37645b5a3865739d54a6cb3 P 6e2837e083814e77b16db43b99d985ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b6684bfcd7404fe68d8cb196153412d5 in term 1.
I20250109 08:56:59.956665 20974 leader_election.cc:304] T 42af5f76a37645b5a3865739d54a6cb3 P b6684bfcd7404fe68d8cb196153412d5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b6684bfcd7404fe68d8cb196153412d5, ead0b707c1014770b25c5a0db0454830; no voters:
I20250109 08:56:59.957223 21244 raft_consensus.cc:2798] T 42af5f76a37645b5a3865739d54a6cb3 P b6684bfcd7404fe68d8cb196153412d5 [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:56:59.957574 21244 raft_consensus.cc:695] T 42af5f76a37645b5a3865739d54a6cb3 P b6684bfcd7404fe68d8cb196153412d5 [term 1 LEADER]: Becoming Leader. State: Replica: b6684bfcd7404fe68d8cb196153412d5, State: Running, Role: LEADER
I20250109 08:56:59.958110 21244 consensus_queue.cc:237] T 42af5f76a37645b5a3865739d54a6cb3 P b6684bfcd7404fe68d8cb196153412d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:56:59.964622 20907 catalog_manager.cc:5526] T 42af5f76a37645b5a3865739d54a6cb3 P b6684bfcd7404fe68d8cb196153412d5 reported cstate change: term changed from 0 to 1, leader changed from <none> to b6684bfcd7404fe68d8cb196153412d5 (127.19.226.193). New cstate: current_term: 1 leader_uuid: "b6684bfcd7404fe68d8cb196153412d5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } health_report { overall_health: UNKNOWN } } }
I20250109 08:57:00.043000 21160 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "502ebc25110a4c04a71b8b2e8eea07d4"
dest_uuid: "ead0b707c1014770b25c5a0db0454830"
mode: GRACEFUL
new_leader_uuid: "b6684bfcd7404fe68d8cb196153412d5"
from {username='slave'} at 127.0.0.1:35984
I20250109 08:57:00.043534 21160 raft_consensus.cc:604] T 502ebc25110a4c04a71b8b2e8eea07d4 P ead0b707c1014770b25c5a0db0454830 [term 1 LEADER]: Received request to transfer leadership to TS b6684bfcd7404fe68d8cb196153412d5
I20250109 08:57:00.057788 21084 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "ec3e3f891e3c4a879077b9ee897e5d52"
dest_uuid: "6e2837e083814e77b16db43b99d985ab"
mode: GRACEFUL
new_leader_uuid: "b6684bfcd7404fe68d8cb196153412d5"
from {username='slave'} at 127.0.0.1:44084
I20250109 08:57:00.058246 21084 raft_consensus.cc:604] T ec3e3f891e3c4a879077b9ee897e5d52 P 6e2837e083814e77b16db43b99d985ab [term 1 LEADER]: Received request to transfer leadership to TS b6684bfcd7404fe68d8cb196153412d5
I20250109 08:57:00.065186 21084 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "948e27b3034d44338e2b7d74f96374f7"
dest_uuid: "6e2837e083814e77b16db43b99d985ab"
mode: GRACEFUL
new_leader_uuid: "b6684bfcd7404fe68d8cb196153412d5"
from {username='slave'} at 127.0.0.1:44084
I20250109 08:57:00.065703 21084 raft_consensus.cc:604] T 948e27b3034d44338e2b7d74f96374f7 P 6e2837e083814e77b16db43b99d985ab [term 1 LEADER]: Received request to transfer leadership to TS b6684bfcd7404fe68d8cb196153412d5
I20250109 08:57:00.069913 21160 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "af13c7179cc24b258e516eceb2b0cac0"
dest_uuid: "ead0b707c1014770b25c5a0db0454830"
mode: GRACEFUL
new_leader_uuid: "6e2837e083814e77b16db43b99d985ab"
from {username='slave'} at 127.0.0.1:35984
I20250109 08:57:00.070405 21160 raft_consensus.cc:604] T af13c7179cc24b258e516eceb2b0cac0 P ead0b707c1014770b25c5a0db0454830 [term 1 LEADER]: Received request to transfer leadership to TS 6e2837e083814e77b16db43b99d985ab
I20250109 08:57:00.074512 21160 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "05148030cdd443048fe5899b03fd92fe"
dest_uuid: "ead0b707c1014770b25c5a0db0454830"
mode: GRACEFUL
new_leader_uuid: "6e2837e083814e77b16db43b99d985ab"
from {username='slave'} at 127.0.0.1:35984
I20250109 08:57:00.074949 21160 raft_consensus.cc:604] T 05148030cdd443048fe5899b03fd92fe P ead0b707c1014770b25c5a0db0454830 [term 1 LEADER]: Received request to transfer leadership to TS 6e2837e083814e77b16db43b99d985ab
I20250109 08:57:00.085935 21010 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "166b00d1332d4f39b57ae9f58cb7fc5f"
dest_uuid: "b6684bfcd7404fe68d8cb196153412d5"
mode: GRACEFUL
new_leader_uuid: "6e2837e083814e77b16db43b99d985ab"
from {username='slave'} at 127.0.0.1:59352
I20250109 08:57:00.086408 21010 raft_consensus.cc:604] T 166b00d1332d4f39b57ae9f58cb7fc5f P b6684bfcd7404fe68d8cb196153412d5 [term 1 LEADER]: Received request to transfer leadership to TS 6e2837e083814e77b16db43b99d985ab
I20250109 08:57:00.090685 21160 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "53b47173fb6a4a358c8ad9c2df0f6f18"
dest_uuid: "ead0b707c1014770b25c5a0db0454830"
mode: GRACEFUL
new_leader_uuid: "6e2837e083814e77b16db43b99d985ab"
from {username='slave'} at 127.0.0.1:35984
I20250109 08:57:00.091132 21160 raft_consensus.cc:604] T 53b47173fb6a4a358c8ad9c2df0f6f18 P ead0b707c1014770b25c5a0db0454830 [term 1 LEADER]: Received request to transfer leadership to TS 6e2837e083814e77b16db43b99d985ab
I20250109 08:57:00.097587 21238 consensus_queue.cc:1035] T 53b47173fb6a4a358c8ad9c2df0f6f18 P ead0b707c1014770b25c5a0db0454830 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:57:00.108978 21238 consensus_queue.cc:1035] T 53b47173fb6a4a358c8ad9c2df0f6f18 P ead0b707c1014770b25c5a0db0454830 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:57:00.115460 21253 raft_consensus.cc:988] T 53b47173fb6a4a358c8ad9c2df0f6f18 P ead0b707c1014770b25c5a0db0454830: : Instructing follower 6e2837e083814e77b16db43b99d985ab to start an election
I20250109 08:57:00.115896 21238 raft_consensus.cc:1076] T 53b47173fb6a4a358c8ad9c2df0f6f18 P ead0b707c1014770b25c5a0db0454830 [term 1 LEADER]: Signalling peer 6e2837e083814e77b16db43b99d985ab to start an election
I20250109 08:57:00.117163 21084 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "53b47173fb6a4a358c8ad9c2df0f6f18"
dest_uuid: "6e2837e083814e77b16db43b99d985ab"
from {username='slave'} at 127.0.0.1:44082
I20250109 08:57:00.117630 21084 raft_consensus.cc:491] T 53b47173fb6a4a358c8ad9c2df0f6f18 P 6e2837e083814e77b16db43b99d985ab [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:57:00.117902 21084 raft_consensus.cc:3054] T 53b47173fb6a4a358c8ad9c2df0f6f18 P 6e2837e083814e77b16db43b99d985ab [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:57:00.122398 21084 raft_consensus.cc:513] T 53b47173fb6a4a358c8ad9c2df0f6f18 P 6e2837e083814e77b16db43b99d985ab [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:57:00.124811 21010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53b47173fb6a4a358c8ad9c2df0f6f18" candidate_uuid: "6e2837e083814e77b16db43b99d985ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b6684bfcd7404fe68d8cb196153412d5"
I20250109 08:57:00.125023 21160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53b47173fb6a4a358c8ad9c2df0f6f18" candidate_uuid: "6e2837e083814e77b16db43b99d985ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ead0b707c1014770b25c5a0db0454830"
I20250109 08:57:00.124480 21084 leader_election.cc:290] T 53b47173fb6a4a358c8ad9c2df0f6f18 P 6e2837e083814e77b16db43b99d985ab [CANDIDATE]: Term 2 election: Requested vote from peers b6684bfcd7404fe68d8cb196153412d5 (127.19.226.193:34209), ead0b707c1014770b25c5a0db0454830 (127.19.226.195:35749)
I20250109 08:57:00.125391 21010 raft_consensus.cc:3054] T 53b47173fb6a4a358c8ad9c2df0f6f18 P b6684bfcd7404fe68d8cb196153412d5 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:57:00.125540 21160 raft_consensus.cc:3049] T 53b47173fb6a4a358c8ad9c2df0f6f18 P ead0b707c1014770b25c5a0db0454830 [term 1 LEADER]: Stepping down as leader of term 1
I20250109 08:57:00.125916 21160 raft_consensus.cc:738] T 53b47173fb6a4a358c8ad9c2df0f6f18 P ead0b707c1014770b25c5a0db0454830 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ead0b707c1014770b25c5a0db0454830, State: Running, Role: LEADER
I20250109 08:57:00.126750 21160 consensus_queue.cc:260] T 53b47173fb6a4a358c8ad9c2df0f6f18 P ead0b707c1014770b25c5a0db0454830 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:57:00.127700 21160 raft_consensus.cc:3054] T 53b47173fb6a4a358c8ad9c2df0f6f18 P ead0b707c1014770b25c5a0db0454830 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:57:00.130903 21244 consensus_queue.cc:1035] T 163310a12b44475ca3727662dad194a6 P b6684bfcd7404fe68d8cb196153412d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:57:00.131618 21010 raft_consensus.cc:2463] T 53b47173fb6a4a358c8ad9c2df0f6f18 P b6684bfcd7404fe68d8cb196153412d5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e2837e083814e77b16db43b99d985ab in term 2.
I20250109 08:57:00.132426 21051 leader_election.cc:304] T 53b47173fb6a4a358c8ad9c2df0f6f18 P 6e2837e083814e77b16db43b99d985ab [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e2837e083814e77b16db43b99d985ab, b6684bfcd7404fe68d8cb196153412d5; no voters:
I20250109 08:57:00.133183 21210 raft_consensus.cc:2798] T 53b47173fb6a4a358c8ad9c2df0f6f18 P 6e2837e083814e77b16db43b99d985ab [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:57:00.133661 21210 raft_consensus.cc:695] T 53b47173fb6a4a358c8ad9c2df0f6f18 P 6e2837e083814e77b16db43b99d985ab [term 2 LEADER]: Becoming Leader. State: Replica: 6e2837e083814e77b16db43b99d985ab, State: Running, Role: LEADER
I20250109 08:57:00.134239 21160 raft_consensus.cc:2463] T 53b47173fb6a4a358c8ad9c2df0f6f18 P ead0b707c1014770b25c5a0db0454830 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e2837e083814e77b16db43b99d985ab in term 2.
I20250109 08:57:00.134461 21210 consensus_queue.cc:237] T 53b47173fb6a4a358c8ad9c2df0f6f18 P 6e2837e083814e77b16db43b99d985ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:57:00.143564 20908 catalog_manager.cc:5526] T 53b47173fb6a4a358c8ad9c2df0f6f18 P 6e2837e083814e77b16db43b99d985ab reported cstate change: term changed from 1 to 2, leader changed from ead0b707c1014770b25c5a0db0454830 (127.19.226.195) to 6e2837e083814e77b16db43b99d985ab (127.19.226.194). New cstate: current_term: 2 leader_uuid: "6e2837e083814e77b16db43b99d985ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } health_report { overall_health: UNKNOWN } } }
I20250109 08:57:00.145079 21202 consensus_queue.cc:1035] T 163310a12b44475ca3727662dad194a6 P b6684bfcd7404fe68d8cb196153412d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:57:00.183521 21210 raft_consensus.cc:491] T 05148030cdd443048fe5899b03fd92fe P 6e2837e083814e77b16db43b99d985ab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:00.183948 21210 raft_consensus.cc:513] T 05148030cdd443048fe5899b03fd92fe P 6e2837e083814e77b16db43b99d985ab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:57:00.185505 21210 leader_election.cc:290] T 05148030cdd443048fe5899b03fd92fe P 6e2837e083814e77b16db43b99d985ab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b6684bfcd7404fe68d8cb196153412d5 (127.19.226.193:34209), ead0b707c1014770b25c5a0db0454830 (127.19.226.195:35749)
I20250109 08:57:00.186293 21010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05148030cdd443048fe5899b03fd92fe" candidate_uuid: "6e2837e083814e77b16db43b99d985ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6684bfcd7404fe68d8cb196153412d5" is_pre_election: true
I20250109 08:57:00.186511 21160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05148030cdd443048fe5899b03fd92fe" candidate_uuid: "6e2837e083814e77b16db43b99d985ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ead0b707c1014770b25c5a0db0454830" is_pre_election: true
I20250109 08:57:00.187126 21010 raft_consensus.cc:2388] T 05148030cdd443048fe5899b03fd92fe P b6684bfcd7404fe68d8cb196153412d5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e2837e083814e77b16db43b99d985ab in current term 1: Already voted for candidate ead0b707c1014770b25c5a0db0454830 in this term.
I20250109 08:57:00.187961 21051 leader_election.cc:304] T 05148030cdd443048fe5899b03fd92fe P 6e2837e083814e77b16db43b99d985ab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e2837e083814e77b16db43b99d985ab; no voters: b6684bfcd7404fe68d8cb196153412d5, ead0b707c1014770b25c5a0db0454830
I20250109 08:57:00.188676 21210 raft_consensus.cc:3054] T 05148030cdd443048fe5899b03fd92fe P 6e2837e083814e77b16db43b99d985ab [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:00.192961 21210 raft_consensus.cc:2743] T 05148030cdd443048fe5899b03fd92fe P 6e2837e083814e77b16db43b99d985ab [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:57:00.198822 21238 consensus_queue.cc:1035] T 502ebc25110a4c04a71b8b2e8eea07d4 P ead0b707c1014770b25c5a0db0454830 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:57:00.207762 21236 consensus_queue.cc:1035] T 502ebc25110a4c04a71b8b2e8eea07d4 P ead0b707c1014770b25c5a0db0454830 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:57:00.214685 21253 raft_consensus.cc:988] T 502ebc25110a4c04a71b8b2e8eea07d4 P ead0b707c1014770b25c5a0db0454830: : Instructing follower b6684bfcd7404fe68d8cb196153412d5 to start an election
I20250109 08:57:00.215096 21253 raft_consensus.cc:1076] T 502ebc25110a4c04a71b8b2e8eea07d4 P ead0b707c1014770b25c5a0db0454830 [term 1 LEADER]: Signalling peer b6684bfcd7404fe68d8cb196153412d5 to start an election
I20250109 08:57:00.216481 21010 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "502ebc25110a4c04a71b8b2e8eea07d4"
dest_uuid: "b6684bfcd7404fe68d8cb196153412d5"
from {username='slave'} at 127.0.0.1:59340
I20250109 08:57:00.217003 21010 raft_consensus.cc:491] T 502ebc25110a4c04a71b8b2e8eea07d4 P b6684bfcd7404fe68d8cb196153412d5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:57:00.217353 21010 raft_consensus.cc:3054] T 502ebc25110a4c04a71b8b2e8eea07d4 P b6684bfcd7404fe68d8cb196153412d5 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:57:00.221714 21010 raft_consensus.cc:513] T 502ebc25110a4c04a71b8b2e8eea07d4 P b6684bfcd7404fe68d8cb196153412d5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } }
I20250109 08:57:00.223135 21010 leader_election.cc:290] T 502ebc25110a4c04a71b8b2e8eea07d4 P b6684bfcd7404fe68d8cb196153412d5 [CANDIDATE]: Term 2 election: Requested vote from peers ead0b707c1014770b25c5a0db0454830 (127.19.226.195:35749), 6e2837e083814e77b16db43b99d985ab (127.19.226.194:44083)
I20250109 08:57:00.223910 21160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "502ebc25110a4c04a71b8b2e8eea07d4" candidate_uuid: "b6684bfcd7404fe68d8cb196153412d5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ead0b707c1014770b25c5a0db0454830"
I20250109 08:57:00.224457 21160 raft_consensus.cc:3049] T 502ebc25110a4c04a71b8b2e8eea07d4 P ead0b707c1014770b25c5a0db0454830 [term 1 LEADER]: Stepping down as leader of term 1
I20250109 08:57:00.224458 21084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "502ebc25110a4c04a71b8b2e8eea07d4" candidate_uuid: "b6684bfcd7404fe68d8cb196153412d5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6e2837e083814e77b16db43b99d985ab"
I20250109 08:57:00.224889 21160 raft_consensus.cc:738] T 502ebc25110a4c04a71b8b2e8eea07d4 P ead0b707c1014770b25c5a0db0454830 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ead0b707c1014770b25c5a0db0454830, State: Running, Role: LEADER
I20250109 08:57:00.225176 21084 raft_consensus.cc:3054] T 502ebc25110a4c04a71b8b2e8eea07d4 P 6e2837e083814e77b16db43b99d985ab [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:57:00.225586 21160 consensus_queue.cc:260] T 502ebc25110a4c04a71b8b2e8eea07d4 P ead0b707c1014770b25c5a0db0454830 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } }
I20250109 08:57:00.226472 21160 raft_consensus.cc:3054] T 502ebc25110a4c04a71b8b2e8eea07d4 P ead0b707c1014770b25c5a0db0454830 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:57:00.230343 21084 raft_consensus.cc:2463] T 502ebc25110a4c04a71b8b2e8eea07d4 P 6e2837e083814e77b16db43b99d985ab [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b6684bfcd7404fe68d8cb196153412d5 in term 2.
I20250109 08:57:00.230438 21253 raft_consensus.cc:491] T 948e27b3034d44338e2b7d74f96374f7 P ead0b707c1014770b25c5a0db0454830 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:00.231012 21160 raft_consensus.cc:2463] T 502ebc25110a4c04a71b8b2e8eea07d4 P ead0b707c1014770b25c5a0db0454830 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b6684bfcd7404fe68d8cb196153412d5 in term 2.
I20250109 08:57:00.230909 21253 raft_consensus.cc:513] T 948e27b3034d44338e2b7d74f96374f7 P ead0b707c1014770b25c5a0db0454830 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:57:00.231222 20974 leader_election.cc:304] T 502ebc25110a4c04a71b8b2e8eea07d4 P b6684bfcd7404fe68d8cb196153412d5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e2837e083814e77b16db43b99d985ab, b6684bfcd7404fe68d8cb196153412d5; no voters:
I20250109 08:57:00.231933 21202 raft_consensus.cc:2798] T 502ebc25110a4c04a71b8b2e8eea07d4 P b6684bfcd7404fe68d8cb196153412d5 [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:57:00.232396 21202 raft_consensus.cc:695] T 502ebc25110a4c04a71b8b2e8eea07d4 P b6684bfcd7404fe68d8cb196153412d5 [term 2 LEADER]: Becoming Leader. State: Replica: b6684bfcd7404fe68d8cb196153412d5, State: Running, Role: LEADER
I20250109 08:57:00.232738 21253 leader_election.cc:290] T 948e27b3034d44338e2b7d74f96374f7 P ead0b707c1014770b25c5a0db0454830 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b6684bfcd7404fe68d8cb196153412d5 (127.19.226.193:34209), 6e2837e083814e77b16db43b99d985ab (127.19.226.194:44083)
I20250109 08:57:00.233413 21010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "948e27b3034d44338e2b7d74f96374f7" candidate_uuid: "ead0b707c1014770b25c5a0db0454830" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6684bfcd7404fe68d8cb196153412d5" is_pre_election: true
I20250109 08:57:00.233165 21202 consensus_queue.cc:237] T 502ebc25110a4c04a71b8b2e8eea07d4 P b6684bfcd7404fe68d8cb196153412d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } }
I20250109 08:57:00.233997 21084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "948e27b3034d44338e2b7d74f96374f7" candidate_uuid: "ead0b707c1014770b25c5a0db0454830" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e2837e083814e77b16db43b99d985ab" is_pre_election: true
I20250109 08:57:00.235898 21010 raft_consensus.cc:2388] T 948e27b3034d44338e2b7d74f96374f7 P b6684bfcd7404fe68d8cb196153412d5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ead0b707c1014770b25c5a0db0454830 in current term 1: Already voted for candidate 6e2837e083814e77b16db43b99d985ab in this term.
I20250109 08:57:00.236752 21125 leader_election.cc:304] T 948e27b3034d44338e2b7d74f96374f7 P ead0b707c1014770b25c5a0db0454830 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ead0b707c1014770b25c5a0db0454830; no voters: 6e2837e083814e77b16db43b99d985ab, b6684bfcd7404fe68d8cb196153412d5
I20250109 08:57:00.237334 21253 raft_consensus.cc:3054] T 948e27b3034d44338e2b7d74f96374f7 P ead0b707c1014770b25c5a0db0454830 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:00.238904 20909 catalog_manager.cc:5526] T 502ebc25110a4c04a71b8b2e8eea07d4 P b6684bfcd7404fe68d8cb196153412d5 reported cstate change: term changed from 1 to 2, leader changed from ead0b707c1014770b25c5a0db0454830 (127.19.226.195) to b6684bfcd7404fe68d8cb196153412d5 (127.19.226.193). New cstate: current_term: 2 leader_uuid: "b6684bfcd7404fe68d8cb196153412d5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } health_report { overall_health: UNKNOWN } } }
I20250109 08:57:00.241739 21253 raft_consensus.cc:2743] T 948e27b3034d44338e2b7d74f96374f7 P ead0b707c1014770b25c5a0db0454830 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:57:00.256299 21210 consensus_queue.cc:1035] T 948e27b3034d44338e2b7d74f96374f7 P 6e2837e083814e77b16db43b99d985ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:57:00.262284 21243 raft_consensus.cc:988] T 948e27b3034d44338e2b7d74f96374f7 P 6e2837e083814e77b16db43b99d985ab: : Instructing follower b6684bfcd7404fe68d8cb196153412d5 to start an election
I20250109 08:57:00.262748 21242 raft_consensus.cc:1076] T 948e27b3034d44338e2b7d74f96374f7 P 6e2837e083814e77b16db43b99d985ab [term 1 LEADER]: Signalling peer b6684bfcd7404fe68d8cb196153412d5 to start an election
I20250109 08:57:00.265321 21009 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "948e27b3034d44338e2b7d74f96374f7"
dest_uuid: "b6684bfcd7404fe68d8cb196153412d5"
from {username='slave'} at 127.0.0.1:59350
I20250109 08:57:00.265856 21009 raft_consensus.cc:491] T 948e27b3034d44338e2b7d74f96374f7 P b6684bfcd7404fe68d8cb196153412d5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:57:00.266220 21009 raft_consensus.cc:3054] T 948e27b3034d44338e2b7d74f96374f7 P b6684bfcd7404fe68d8cb196153412d5 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:57:00.266793 21210 consensus_queue.cc:1035] T 948e27b3034d44338e2b7d74f96374f7 P 6e2837e083814e77b16db43b99d985ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:57:00.274523 21009 raft_consensus.cc:513] T 948e27b3034d44338e2b7d74f96374f7 P b6684bfcd7404fe68d8cb196153412d5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:57:00.277209 21084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "948e27b3034d44338e2b7d74f96374f7" candidate_uuid: "b6684bfcd7404fe68d8cb196153412d5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6e2837e083814e77b16db43b99d985ab"
I20250109 08:57:00.277498 21160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "948e27b3034d44338e2b7d74f96374f7" candidate_uuid: "b6684bfcd7404fe68d8cb196153412d5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ead0b707c1014770b25c5a0db0454830"
I20250109 08:57:00.277793 21084 raft_consensus.cc:3049] T 948e27b3034d44338e2b7d74f96374f7 P 6e2837e083814e77b16db43b99d985ab [term 1 LEADER]: Stepping down as leader of term 1
I20250109 08:57:00.278012 21160 raft_consensus.cc:3054] T 948e27b3034d44338e2b7d74f96374f7 P ead0b707c1014770b25c5a0db0454830 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:57:00.278141 21084 raft_consensus.cc:738] T 948e27b3034d44338e2b7d74f96374f7 P 6e2837e083814e77b16db43b99d985ab [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6e2837e083814e77b16db43b99d985ab, State: Running, Role: LEADER
I20250109 08:57:00.278965 21084 consensus_queue.cc:260] T 948e27b3034d44338e2b7d74f96374f7 P 6e2837e083814e77b16db43b99d985ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:57:00.280047 21084 raft_consensus.cc:3054] T 948e27b3034d44338e2b7d74f96374f7 P 6e2837e083814e77b16db43b99d985ab [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:57:00.281507 21253 consensus_queue.cc:1035] T 05148030cdd443048fe5899b03fd92fe P ead0b707c1014770b25c5a0db0454830 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:57:00.283417 21009 leader_election.cc:290] T 948e27b3034d44338e2b7d74f96374f7 P b6684bfcd7404fe68d8cb196153412d5 [CANDIDATE]: Term 2 election: Requested vote from peers 6e2837e083814e77b16db43b99d985ab (127.19.226.194:44083), ead0b707c1014770b25c5a0db0454830 (127.19.226.195:35749)
I20250109 08:57:00.284538 21160 raft_consensus.cc:2463] T 948e27b3034d44338e2b7d74f96374f7 P ead0b707c1014770b25c5a0db0454830 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b6684bfcd7404fe68d8cb196153412d5 in term 2.
I20250109 08:57:00.286645 21084 raft_consensus.cc:2463] T 948e27b3034d44338e2b7d74f96374f7 P 6e2837e083814e77b16db43b99d985ab [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b6684bfcd7404fe68d8cb196153412d5 in term 2.
I20250109 08:57:00.287767 20974 leader_election.cc:304] T 948e27b3034d44338e2b7d74f96374f7 P b6684bfcd7404fe68d8cb196153412d5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e2837e083814e77b16db43b99d985ab, b6684bfcd7404fe68d8cb196153412d5; no voters:
I20250109 08:57:00.288586 21202 raft_consensus.cc:2798] T 948e27b3034d44338e2b7d74f96374f7 P b6684bfcd7404fe68d8cb196153412d5 [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:57:00.289012 21202 raft_consensus.cc:695] T 948e27b3034d44338e2b7d74f96374f7 P b6684bfcd7404fe68d8cb196153412d5 [term 2 LEADER]: Becoming Leader. State: Replica: b6684bfcd7404fe68d8cb196153412d5, State: Running, Role: LEADER
I20250109 08:57:00.289777 21202 consensus_queue.cc:237] T 948e27b3034d44338e2b7d74f96374f7 P b6684bfcd7404fe68d8cb196153412d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:57:00.294654 21256 consensus_queue.cc:1035] T 05148030cdd443048fe5899b03fd92fe P ead0b707c1014770b25c5a0db0454830 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250109 08:57:00.298811 21236 consensus_queue.cc:1035] T cb177a6fcf4144e2ae1a560eb627e276 P ead0b707c1014770b25c5a0db0454830 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:57:00.301723 20907 catalog_manager.cc:5526] T 948e27b3034d44338e2b7d74f96374f7 P b6684bfcd7404fe68d8cb196153412d5 reported cstate change: term changed from 1 to 2, leader changed from 6e2837e083814e77b16db43b99d985ab (127.19.226.194) to b6684bfcd7404fe68d8cb196153412d5 (127.19.226.193). New cstate: current_term: 2 leader_uuid: "b6684bfcd7404fe68d8cb196153412d5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } health_report { overall_health: UNKNOWN } } }
I20250109 08:57:00.311287 21239 consensus_queue.cc:1035] T cb177a6fcf4144e2ae1a560eb627e276 P ead0b707c1014770b25c5a0db0454830 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:57:00.312884 21239 raft_consensus.cc:988] T 05148030cdd443048fe5899b03fd92fe P ead0b707c1014770b25c5a0db0454830: : Instructing follower 6e2837e083814e77b16db43b99d985ab to start an election
I20250109 08:57:00.313232 21239 raft_consensus.cc:1076] T 05148030cdd443048fe5899b03fd92fe P ead0b707c1014770b25c5a0db0454830 [term 1 LEADER]: Signalling peer 6e2837e083814e77b16db43b99d985ab to start an election
I20250109 08:57:00.317610 21085 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "05148030cdd443048fe5899b03fd92fe"
dest_uuid: "6e2837e083814e77b16db43b99d985ab"
from {username='slave'} at 127.0.0.1:44082
I20250109 08:57:00.318158 21085 raft_consensus.cc:491] T 05148030cdd443048fe5899b03fd92fe P 6e2837e083814e77b16db43b99d985ab [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:57:00.318517 21085 raft_consensus.cc:3054] T 05148030cdd443048fe5899b03fd92fe P 6e2837e083814e77b16db43b99d985ab [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:57:00.323051 21210 consensus_queue.cc:1035] T ec3e3f891e3c4a879077b9ee897e5d52 P 6e2837e083814e77b16db43b99d985ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:57:00.325431 21085 raft_consensus.cc:513] T 05148030cdd443048fe5899b03fd92fe P 6e2837e083814e77b16db43b99d985ab [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:57:00.327932 21009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05148030cdd443048fe5899b03fd92fe" candidate_uuid: "6e2837e083814e77b16db43b99d985ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b6684bfcd7404fe68d8cb196153412d5"
I20250109 08:57:00.328431 21009 raft_consensus.cc:3054] T 05148030cdd443048fe5899b03fd92fe P b6684bfcd7404fe68d8cb196153412d5 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:57:00.329481 21085 leader_election.cc:290] T 05148030cdd443048fe5899b03fd92fe P 6e2837e083814e77b16db43b99d985ab [CANDIDATE]: Term 2 election: Requested vote from peers b6684bfcd7404fe68d8cb196153412d5 (127.19.226.193:34209), ead0b707c1014770b25c5a0db0454830 (127.19.226.195:35749)
I20250109 08:57:00.330235 21160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05148030cdd443048fe5899b03fd92fe" candidate_uuid: "6e2837e083814e77b16db43b99d985ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ead0b707c1014770b25c5a0db0454830"
I20250109 08:57:00.330793 21160 raft_consensus.cc:3049] T 05148030cdd443048fe5899b03fd92fe P ead0b707c1014770b25c5a0db0454830 [term 1 LEADER]: Stepping down as leader of term 1
I20250109 08:57:00.331185 21160 raft_consensus.cc:738] T 05148030cdd443048fe5899b03fd92fe P ead0b707c1014770b25c5a0db0454830 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ead0b707c1014770b25c5a0db0454830, State: Running, Role: LEADER
I20250109 08:57:00.330828 21242 raft_consensus.cc:988] T ec3e3f891e3c4a879077b9ee897e5d52 P 6e2837e083814e77b16db43b99d985ab: : Instructing follower b6684bfcd7404fe68d8cb196153412d5 to start an election
I20250109 08:57:00.332015 21242 raft_consensus.cc:1076] T ec3e3f891e3c4a879077b9ee897e5d52 P 6e2837e083814e77b16db43b99d985ab [term 1 LEADER]: Signalling peer b6684bfcd7404fe68d8cb196153412d5 to start an election
I20250109 08:57:00.331907 21160 consensus_queue.cc:260] T 05148030cdd443048fe5899b03fd92fe P ead0b707c1014770b25c5a0db0454830 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:57:00.333179 21160 raft_consensus.cc:3054] T 05148030cdd443048fe5899b03fd92fe P ead0b707c1014770b25c5a0db0454830 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:57:00.334405 21011 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "ec3e3f891e3c4a879077b9ee897e5d52"
dest_uuid: "b6684bfcd7404fe68d8cb196153412d5"
from {username='slave'} at 127.0.0.1:59350
I20250109 08:57:00.335187 21011 raft_consensus.cc:491] T ec3e3f891e3c4a879077b9ee897e5d52 P b6684bfcd7404fe68d8cb196153412d5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:57:00.335580 21011 raft_consensus.cc:3054] T ec3e3f891e3c4a879077b9ee897e5d52 P b6684bfcd7404fe68d8cb196153412d5 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:57:00.336862 21243 consensus_queue.cc:1035] T ec3e3f891e3c4a879077b9ee897e5d52 P 6e2837e083814e77b16db43b99d985ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:57:00.342849 21202 consensus_queue.cc:1035] T 166b00d1332d4f39b57ae9f58cb7fc5f P b6684bfcd7404fe68d8cb196153412d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:57:00.343304 21011 raft_consensus.cc:513] T ec3e3f891e3c4a879077b9ee897e5d52 P b6684bfcd7404fe68d8cb196153412d5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:57:00.344166 21160 raft_consensus.cc:2463] T 05148030cdd443048fe5899b03fd92fe P ead0b707c1014770b25c5a0db0454830 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e2837e083814e77b16db43b99d985ab in term 2.
I20250109 08:57:00.345306 21049 leader_election.cc:304] T 05148030cdd443048fe5899b03fd92fe P 6e2837e083814e77b16db43b99d985ab [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e2837e083814e77b16db43b99d985ab, ead0b707c1014770b25c5a0db0454830; no voters:
I20250109 08:57:00.346026 21210 raft_consensus.cc:2798] T 05148030cdd443048fe5899b03fd92fe P 6e2837e083814e77b16db43b99d985ab [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:57:00.346472 21210 raft_consensus.cc:695] T 05148030cdd443048fe5899b03fd92fe P 6e2837e083814e77b16db43b99d985ab [term 2 LEADER]: Becoming Leader. State: Replica: 6e2837e083814e77b16db43b99d985ab, State: Running, Role: LEADER
I20250109 08:57:00.347239 21210 consensus_queue.cc:237] T 05148030cdd443048fe5899b03fd92fe P 6e2837e083814e77b16db43b99d985ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:57:00.352653 21011 leader_election.cc:290] T ec3e3f891e3c4a879077b9ee897e5d52 P b6684bfcd7404fe68d8cb196153412d5 [CANDIDATE]: Term 2 election: Requested vote from peers 6e2837e083814e77b16db43b99d985ab (127.19.226.194:44083), ead0b707c1014770b25c5a0db0454830 (127.19.226.195:35749)
I20250109 08:57:00.345794 21009 raft_consensus.cc:2463] T 05148030cdd443048fe5899b03fd92fe P b6684bfcd7404fe68d8cb196153412d5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e2837e083814e77b16db43b99d985ab in term 2.
I20250109 08:57:00.353812 21159 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec3e3f891e3c4a879077b9ee897e5d52" candidate_uuid: "b6684bfcd7404fe68d8cb196153412d5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ead0b707c1014770b25c5a0db0454830"
I20250109 08:57:00.356449 21085 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec3e3f891e3c4a879077b9ee897e5d52" candidate_uuid: "b6684bfcd7404fe68d8cb196153412d5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6e2837e083814e77b16db43b99d985ab"
I20250109 08:57:00.357008 21085 raft_consensus.cc:3049] T ec3e3f891e3c4a879077b9ee897e5d52 P 6e2837e083814e77b16db43b99d985ab [term 1 LEADER]: Stepping down as leader of term 1
I20250109 08:57:00.357348 21085 raft_consensus.cc:738] T ec3e3f891e3c4a879077b9ee897e5d52 P 6e2837e083814e77b16db43b99d985ab [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6e2837e083814e77b16db43b99d985ab, State: Running, Role: LEADER
I20250109 08:57:00.355926 20909 catalog_manager.cc:5526] T 05148030cdd443048fe5899b03fd92fe P 6e2837e083814e77b16db43b99d985ab reported cstate change: term changed from 1 to 2, leader changed from ead0b707c1014770b25c5a0db0454830 (127.19.226.195) to 6e2837e083814e77b16db43b99d985ab (127.19.226.194). New cstate: current_term: 2 leader_uuid: "6e2837e083814e77b16db43b99d985ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } health_report { overall_health: UNKNOWN } } }
I20250109 08:57:00.358127 21085 consensus_queue.cc:260] T ec3e3f891e3c4a879077b9ee897e5d52 P 6e2837e083814e77b16db43b99d985ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:57:00.359185 21085 raft_consensus.cc:3054] T ec3e3f891e3c4a879077b9ee897e5d52 P 6e2837e083814e77b16db43b99d985ab [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:57:00.365233 21085 raft_consensus.cc:2463] T ec3e3f891e3c4a879077b9ee897e5d52 P 6e2837e083814e77b16db43b99d985ab [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b6684bfcd7404fe68d8cb196153412d5 in term 2.
I20250109 08:57:00.369416 21244 consensus_queue.cc:1035] T 166b00d1332d4f39b57ae9f58cb7fc5f P b6684bfcd7404fe68d8cb196153412d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250109 08:57:00.371351 20974 leader_election.cc:304] T ec3e3f891e3c4a879077b9ee897e5d52 P b6684bfcd7404fe68d8cb196153412d5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e2837e083814e77b16db43b99d985ab, b6684bfcd7404fe68d8cb196153412d5; no voters: ead0b707c1014770b25c5a0db0454830
I20250109 08:57:00.372473 21244 raft_consensus.cc:2798] T ec3e3f891e3c4a879077b9ee897e5d52 P b6684bfcd7404fe68d8cb196153412d5 [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:57:00.372872 21244 raft_consensus.cc:695] T ec3e3f891e3c4a879077b9ee897e5d52 P b6684bfcd7404fe68d8cb196153412d5 [term 2 LEADER]: Becoming Leader. State: Replica: b6684bfcd7404fe68d8cb196153412d5, State: Running, Role: LEADER
I20250109 08:57:00.373623 21244 consensus_queue.cc:237] T ec3e3f891e3c4a879077b9ee897e5d52 P b6684bfcd7404fe68d8cb196153412d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:57:00.380515 21245 raft_consensus.cc:988] T 166b00d1332d4f39b57ae9f58cb7fc5f P b6684bfcd7404fe68d8cb196153412d5: : Instructing follower 6e2837e083814e77b16db43b99d985ab to start an election
I20250109 08:57:00.380249 20909 catalog_manager.cc:5526] T ec3e3f891e3c4a879077b9ee897e5d52 P b6684bfcd7404fe68d8cb196153412d5 reported cstate change: term changed from 1 to 2, leader changed from 6e2837e083814e77b16db43b99d985ab (127.19.226.194) to b6684bfcd7404fe68d8cb196153412d5 (127.19.226.193). New cstate: current_term: 2 leader_uuid: "b6684bfcd7404fe68d8cb196153412d5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } health_report { overall_health: UNKNOWN } } }
I20250109 08:57:00.380949 21244 raft_consensus.cc:1076] T 166b00d1332d4f39b57ae9f58cb7fc5f P b6684bfcd7404fe68d8cb196153412d5 [term 1 LEADER]: Signalling peer 6e2837e083814e77b16db43b99d985ab to start an election
I20250109 08:57:00.382534 21085 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "166b00d1332d4f39b57ae9f58cb7fc5f"
dest_uuid: "6e2837e083814e77b16db43b99d985ab"
from {username='slave'} at 127.0.0.1:44078
I20250109 08:57:00.383036 21085 raft_consensus.cc:491] T 166b00d1332d4f39b57ae9f58cb7fc5f P 6e2837e083814e77b16db43b99d985ab [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:57:00.383380 21085 raft_consensus.cc:3054] T 166b00d1332d4f39b57ae9f58cb7fc5f P 6e2837e083814e77b16db43b99d985ab [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:57:00.388505 21085 raft_consensus.cc:513] T 166b00d1332d4f39b57ae9f58cb7fc5f P 6e2837e083814e77b16db43b99d985ab [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:57:00.390712 21009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "166b00d1332d4f39b57ae9f58cb7fc5f" candidate_uuid: "6e2837e083814e77b16db43b99d985ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b6684bfcd7404fe68d8cb196153412d5"
I20250109 08:57:00.391017 21160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "166b00d1332d4f39b57ae9f58cb7fc5f" candidate_uuid: "6e2837e083814e77b16db43b99d985ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ead0b707c1014770b25c5a0db0454830"
I20250109 08:57:00.391216 21009 raft_consensus.cc:3049] T 166b00d1332d4f39b57ae9f58cb7fc5f P b6684bfcd7404fe68d8cb196153412d5 [term 1 LEADER]: Stepping down as leader of term 1
I20250109 08:57:00.391530 21009 raft_consensus.cc:738] T 166b00d1332d4f39b57ae9f58cb7fc5f P b6684bfcd7404fe68d8cb196153412d5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b6684bfcd7404fe68d8cb196153412d5, State: Running, Role: LEADER
I20250109 08:57:00.391547 21160 raft_consensus.cc:3054] T 166b00d1332d4f39b57ae9f58cb7fc5f P ead0b707c1014770b25c5a0db0454830 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:57:00.390462 21085 leader_election.cc:290] T 166b00d1332d4f39b57ae9f58cb7fc5f P 6e2837e083814e77b16db43b99d985ab [CANDIDATE]: Term 2 election: Requested vote from peers b6684bfcd7404fe68d8cb196153412d5 (127.19.226.193:34209), ead0b707c1014770b25c5a0db0454830 (127.19.226.195:35749)
I20250109 08:57:00.392290 21009 consensus_queue.cc:260] T 166b00d1332d4f39b57ae9f58cb7fc5f P b6684bfcd7404fe68d8cb196153412d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:57:00.393412 21009 raft_consensus.cc:3054] T 166b00d1332d4f39b57ae9f58cb7fc5f P b6684bfcd7404fe68d8cb196153412d5 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:57:00.396732 21256 consensus_queue.cc:1035] T af13c7179cc24b258e516eceb2b0cac0 P ead0b707c1014770b25c5a0db0454830 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:57:00.397059 21160 raft_consensus.cc:2463] T 166b00d1332d4f39b57ae9f58cb7fc5f P ead0b707c1014770b25c5a0db0454830 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e2837e083814e77b16db43b99d985ab in term 2.
I20250109 08:57:00.397917 21049 leader_election.cc:304] T 166b00d1332d4f39b57ae9f58cb7fc5f P 6e2837e083814e77b16db43b99d985ab [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e2837e083814e77b16db43b99d985ab, ead0b707c1014770b25c5a0db0454830; no voters:
I20250109 08:57:00.398600 21210 raft_consensus.cc:2798] T 166b00d1332d4f39b57ae9f58cb7fc5f P 6e2837e083814e77b16db43b99d985ab [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:57:00.398916 21210 raft_consensus.cc:695] T 166b00d1332d4f39b57ae9f58cb7fc5f P 6e2837e083814e77b16db43b99d985ab [term 2 LEADER]: Becoming Leader. State: Replica: 6e2837e083814e77b16db43b99d985ab, State: Running, Role: LEADER
I20250109 08:57:00.399642 21210 consensus_queue.cc:237] T 166b00d1332d4f39b57ae9f58cb7fc5f P 6e2837e083814e77b16db43b99d985ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:57:00.400175 21009 raft_consensus.cc:2463] T 166b00d1332d4f39b57ae9f58cb7fc5f P b6684bfcd7404fe68d8cb196153412d5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e2837e083814e77b16db43b99d985ab in term 2.
I20250109 08:57:00.406248 20907 catalog_manager.cc:5526] T 166b00d1332d4f39b57ae9f58cb7fc5f P 6e2837e083814e77b16db43b99d985ab reported cstate change: term changed from 1 to 2, leader changed from b6684bfcd7404fe68d8cb196153412d5 (127.19.226.193) to 6e2837e083814e77b16db43b99d985ab (127.19.226.194). New cstate: current_term: 2 leader_uuid: "6e2837e083814e77b16db43b99d985ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } health_report { overall_health: UNKNOWN } } }
I20250109 08:57:00.409174 21238 consensus_queue.cc:1035] T af13c7179cc24b258e516eceb2b0cac0 P ead0b707c1014770b25c5a0db0454830 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:57:00.413221 21245 consensus_queue.cc:1035] T 42af5f76a37645b5a3865739d54a6cb3 P b6684bfcd7404fe68d8cb196153412d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:57:00.429898 21245 consensus_queue.cc:1035] T 42af5f76a37645b5a3865739d54a6cb3 P b6684bfcd7404fe68d8cb196153412d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:57:00.430435 21206 raft_consensus.cc:988] T af13c7179cc24b258e516eceb2b0cac0 P ead0b707c1014770b25c5a0db0454830: : Instructing follower 6e2837e083814e77b16db43b99d985ab to start an election
I20250109 08:57:00.430841 21238 raft_consensus.cc:1076] T af13c7179cc24b258e516eceb2b0cac0 P ead0b707c1014770b25c5a0db0454830 [term 1 LEADER]: Signalling peer 6e2837e083814e77b16db43b99d985ab to start an election
I20250109 08:57:00.432278 21086 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "af13c7179cc24b258e516eceb2b0cac0"
dest_uuid: "6e2837e083814e77b16db43b99d985ab"
from {username='slave'} at 127.0.0.1:44082
I20250109 08:57:00.432847 21086 raft_consensus.cc:491] T af13c7179cc24b258e516eceb2b0cac0 P 6e2837e083814e77b16db43b99d985ab [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:57:00.433233 21086 raft_consensus.cc:3054] T af13c7179cc24b258e516eceb2b0cac0 P 6e2837e083814e77b16db43b99d985ab [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:57:00.440145 21086 raft_consensus.cc:513] T af13c7179cc24b258e516eceb2b0cac0 P 6e2837e083814e77b16db43b99d985ab [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:57:00.441859 21086 leader_election.cc:290] T af13c7179cc24b258e516eceb2b0cac0 P 6e2837e083814e77b16db43b99d985ab [CANDIDATE]: Term 2 election: Requested vote from peers b6684bfcd7404fe68d8cb196153412d5 (127.19.226.193:34209), ead0b707c1014770b25c5a0db0454830 (127.19.226.195:35749)
I20250109 08:57:00.442479 21160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af13c7179cc24b258e516eceb2b0cac0" candidate_uuid: "6e2837e083814e77b16db43b99d985ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ead0b707c1014770b25c5a0db0454830"
I20250109 08:57:00.442529 21011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af13c7179cc24b258e516eceb2b0cac0" candidate_uuid: "6e2837e083814e77b16db43b99d985ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b6684bfcd7404fe68d8cb196153412d5"
I20250109 08:57:00.443189 21160 raft_consensus.cc:3049] T af13c7179cc24b258e516eceb2b0cac0 P ead0b707c1014770b25c5a0db0454830 [term 1 LEADER]: Stepping down as leader of term 1
I20250109 08:57:00.443244 21011 raft_consensus.cc:3054] T af13c7179cc24b258e516eceb2b0cac0 P b6684bfcd7404fe68d8cb196153412d5 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:57:00.443625 21160 raft_consensus.cc:738] T af13c7179cc24b258e516eceb2b0cac0 P ead0b707c1014770b25c5a0db0454830 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ead0b707c1014770b25c5a0db0454830, State: Running, Role: LEADER
I20250109 08:57:00.444387 21160 consensus_queue.cc:260] T af13c7179cc24b258e516eceb2b0cac0 P ead0b707c1014770b25c5a0db0454830 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:57:00.445578 21160 raft_consensus.cc:3054] T af13c7179cc24b258e516eceb2b0cac0 P ead0b707c1014770b25c5a0db0454830 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:57:00.447862 21011 raft_consensus.cc:2463] T af13c7179cc24b258e516eceb2b0cac0 P b6684bfcd7404fe68d8cb196153412d5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e2837e083814e77b16db43b99d985ab in term 2.
I20250109 08:57:00.448750 21051 leader_election.cc:304] T af13c7179cc24b258e516eceb2b0cac0 P 6e2837e083814e77b16db43b99d985ab [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e2837e083814e77b16db43b99d985ab, b6684bfcd7404fe68d8cb196153412d5; no voters:
I20250109 08:57:00.449486 21210 raft_consensus.cc:2798] T af13c7179cc24b258e516eceb2b0cac0 P 6e2837e083814e77b16db43b99d985ab [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:57:00.449826 21210 raft_consensus.cc:695] T af13c7179cc24b258e516eceb2b0cac0 P 6e2837e083814e77b16db43b99d985ab [term 2 LEADER]: Becoming Leader. State: Replica: 6e2837e083814e77b16db43b99d985ab, State: Running, Role: LEADER
I20250109 08:57:00.450491 21210 consensus_queue.cc:237] T af13c7179cc24b258e516eceb2b0cac0 P 6e2837e083814e77b16db43b99d985ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:57:00.450863 21160 raft_consensus.cc:2463] T af13c7179cc24b258e516eceb2b0cac0 P ead0b707c1014770b25c5a0db0454830 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e2837e083814e77b16db43b99d985ab in term 2.
I20250109 08:57:00.456126 20907 catalog_manager.cc:5526] T af13c7179cc24b258e516eceb2b0cac0 P 6e2837e083814e77b16db43b99d985ab reported cstate change: term changed from 1 to 2, leader changed from ead0b707c1014770b25c5a0db0454830 (127.19.226.195) to 6e2837e083814e77b16db43b99d985ab (127.19.226.194). New cstate: current_term: 2 leader_uuid: "6e2837e083814e77b16db43b99d985ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } health_report { overall_health: UNKNOWN } } }
I20250109 08:57:00.601354 21160 raft_consensus.cc:1270] T 53b47173fb6a4a358c8ad9c2df0f6f18 P ead0b707c1014770b25c5a0db0454830 [term 2 FOLLOWER]: Refusing update from remote peer 6e2837e083814e77b16db43b99d985ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:57:00.602604 21210 consensus_queue.cc:1035] T 53b47173fb6a4a358c8ad9c2df0f6f18 P 6e2837e083814e77b16db43b99d985ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:57:00.609413 21011 raft_consensus.cc:1270] T 53b47173fb6a4a358c8ad9c2df0f6f18 P b6684bfcd7404fe68d8cb196153412d5 [term 2 FOLLOWER]: Refusing update from remote peer 6e2837e083814e77b16db43b99d985ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:57:00.610787 21243 consensus_queue.cc:1035] T 53b47173fb6a4a358c8ad9c2df0f6f18 P 6e2837e083814e77b16db43b99d985ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:57:00.713207 21086 raft_consensus.cc:1270] T 502ebc25110a4c04a71b8b2e8eea07d4 P 6e2837e083814e77b16db43b99d985ab [term 2 FOLLOWER]: Refusing update from remote peer b6684bfcd7404fe68d8cb196153412d5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:57:00.714360 21245 consensus_queue.cc:1035] T 502ebc25110a4c04a71b8b2e8eea07d4 P b6684bfcd7404fe68d8cb196153412d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:57:00.723639 21160 raft_consensus.cc:1270] T 502ebc25110a4c04a71b8b2e8eea07d4 P ead0b707c1014770b25c5a0db0454830 [term 2 FOLLOWER]: Refusing update from remote peer b6684bfcd7404fe68d8cb196153412d5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:57:00.724881 21244 consensus_queue.cc:1035] T 502ebc25110a4c04a71b8b2e8eea07d4 P b6684bfcd7404fe68d8cb196153412d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:57:00.770668 21160 raft_consensus.cc:1270] T 05148030cdd443048fe5899b03fd92fe P ead0b707c1014770b25c5a0db0454830 [term 2 FOLLOWER]: Refusing update from remote peer 6e2837e083814e77b16db43b99d985ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:57:00.771903 21243 consensus_queue.cc:1035] T 05148030cdd443048fe5899b03fd92fe P 6e2837e083814e77b16db43b99d985ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:57:00.778910 21011 raft_consensus.cc:1270] T 05148030cdd443048fe5899b03fd92fe P b6684bfcd7404fe68d8cb196153412d5 [term 2 FOLLOWER]: Refusing update from remote peer 6e2837e083814e77b16db43b99d985ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:57:00.780556 21210 consensus_queue.cc:1035] T 05148030cdd443048fe5899b03fd92fe P 6e2837e083814e77b16db43b99d985ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:57:00.788125 21160 raft_consensus.cc:1270] T 166b00d1332d4f39b57ae9f58cb7fc5f P ead0b707c1014770b25c5a0db0454830 [term 2 FOLLOWER]: Refusing update from remote peer 6e2837e083814e77b16db43b99d985ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:57:00.789319 21210 consensus_queue.cc:1035] T 166b00d1332d4f39b57ae9f58cb7fc5f P 6e2837e083814e77b16db43b99d985ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:57:00.796558 21011 raft_consensus.cc:1270] T 166b00d1332d4f39b57ae9f58cb7fc5f P b6684bfcd7404fe68d8cb196153412d5 [term 2 FOLLOWER]: Refusing update from remote peer 6e2837e083814e77b16db43b99d985ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:57:00.797948 21210 consensus_queue.cc:1035] T 166b00d1332d4f39b57ae9f58cb7fc5f P 6e2837e083814e77b16db43b99d985ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:57:00.833154 21086 raft_consensus.cc:1270] T 948e27b3034d44338e2b7d74f96374f7 P 6e2837e083814e77b16db43b99d985ab [term 2 FOLLOWER]: Refusing update from remote peer b6684bfcd7404fe68d8cb196153412d5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:57:00.834378 21244 consensus_queue.cc:1035] T 948e27b3034d44338e2b7d74f96374f7 P b6684bfcd7404fe68d8cb196153412d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:57:00.841821 21160 raft_consensus.cc:1270] T 948e27b3034d44338e2b7d74f96374f7 P ead0b707c1014770b25c5a0db0454830 [term 2 FOLLOWER]: Refusing update from remote peer b6684bfcd7404fe68d8cb196153412d5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:57:00.843439 21290 consensus_queue.cc:1035] T 948e27b3034d44338e2b7d74f96374f7 P b6684bfcd7404fe68d8cb196153412d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:57:00.844810 21085 raft_consensus.cc:1270] T ec3e3f891e3c4a879077b9ee897e5d52 P 6e2837e083814e77b16db43b99d985ab [term 2 FOLLOWER]: Refusing update from remote peer b6684bfcd7404fe68d8cb196153412d5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:57:00.846344 21290 consensus_queue.cc:1035] T ec3e3f891e3c4a879077b9ee897e5d52 P b6684bfcd7404fe68d8cb196153412d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:57:00.857501 21160 raft_consensus.cc:3054] T ec3e3f891e3c4a879077b9ee897e5d52 P ead0b707c1014770b25c5a0db0454830 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:57:00.864061 21160 raft_consensus.cc:1270] T ec3e3f891e3c4a879077b9ee897e5d52 P ead0b707c1014770b25c5a0db0454830 [term 2 FOLLOWER]: Refusing update from remote peer b6684bfcd7404fe68d8cb196153412d5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:57:00.865278 21244 consensus_queue.cc:1035] T ec3e3f891e3c4a879077b9ee897e5d52 P b6684bfcd7404fe68d8cb196153412d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:57:00.920867 21011 raft_consensus.cc:1270] T af13c7179cc24b258e516eceb2b0cac0 P b6684bfcd7404fe68d8cb196153412d5 [term 2 FOLLOWER]: Refusing update from remote peer 6e2837e083814e77b16db43b99d985ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:57:00.922061 21210 consensus_queue.cc:1035] T af13c7179cc24b258e516eceb2b0cac0 P 6e2837e083814e77b16db43b99d985ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:57:00.928887 21160 raft_consensus.cc:1270] T af13c7179cc24b258e516eceb2b0cac0 P ead0b707c1014770b25c5a0db0454830 [term 2 FOLLOWER]: Refusing update from remote peer 6e2837e083814e77b16db43b99d985ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:57:00.930523 21242 consensus_queue.cc:1035] T af13c7179cc24b258e516eceb2b0cac0 P 6e2837e083814e77b16db43b99d985ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:57:03.120128 20363 auto_leader_rebalancer-test.cc:339] The leader distribution is
I20250109 08:57:03.120865 20363 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250109 08:57:03.146662 21011 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "163310a12b44475ca3727662dad194a6"
dest_uuid: "b6684bfcd7404fe68d8cb196153412d5"
mode: GRACEFUL
new_leader_uuid: "ead0b707c1014770b25c5a0db0454830"
from {username='slave'} at 127.0.0.1:59356
I20250109 08:57:03.147176 21011 raft_consensus.cc:604] T 163310a12b44475ca3727662dad194a6 P b6684bfcd7404fe68d8cb196153412d5 [term 1 LEADER]: Received request to transfer leadership to TS ead0b707c1014770b25c5a0db0454830
I20250109 08:57:03.149263 21011 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "ec3e3f891e3c4a879077b9ee897e5d52"
dest_uuid: "b6684bfcd7404fe68d8cb196153412d5"
mode: GRACEFUL
new_leader_uuid: "ead0b707c1014770b25c5a0db0454830"
from {username='slave'} at 127.0.0.1:59356
I20250109 08:57:03.149719 21011 raft_consensus.cc:604] T ec3e3f891e3c4a879077b9ee897e5d52 P b6684bfcd7404fe68d8cb196153412d5 [term 2 LEADER]: Received request to transfer leadership to TS ead0b707c1014770b25c5a0db0454830
I20250109 08:57:03.150641 20363 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 2
I20250109 08:57:03.151031 20363 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250109 08:57:03.410045 21244 raft_consensus.cc:988] T ec3e3f891e3c4a879077b9ee897e5d52 P b6684bfcd7404fe68d8cb196153412d5: : Instructing follower ead0b707c1014770b25c5a0db0454830 to start an election
I20250109 08:57:03.410482 21293 raft_consensus.cc:1076] T ec3e3f891e3c4a879077b9ee897e5d52 P b6684bfcd7404fe68d8cb196153412d5 [term 2 LEADER]: Signalling peer ead0b707c1014770b25c5a0db0454830 to start an election
I20250109 08:57:03.411769 21160 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "ec3e3f891e3c4a879077b9ee897e5d52"
dest_uuid: "ead0b707c1014770b25c5a0db0454830"
from {username='slave'} at 127.0.0.1:35956
I20250109 08:57:03.412181 21160 raft_consensus.cc:491] T ec3e3f891e3c4a879077b9ee897e5d52 P ead0b707c1014770b25c5a0db0454830 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:57:03.412441 21160 raft_consensus.cc:3054] T ec3e3f891e3c4a879077b9ee897e5d52 P ead0b707c1014770b25c5a0db0454830 [term 2 FOLLOWER]: Advancing to term 3
I20250109 08:57:03.416230 21160 raft_consensus.cc:513] T ec3e3f891e3c4a879077b9ee897e5d52 P ead0b707c1014770b25c5a0db0454830 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:57:03.417627 21160 leader_election.cc:290] T ec3e3f891e3c4a879077b9ee897e5d52 P ead0b707c1014770b25c5a0db0454830 [CANDIDATE]: Term 3 election: Requested vote from peers b6684bfcd7404fe68d8cb196153412d5 (127.19.226.193:34209), 6e2837e083814e77b16db43b99d985ab (127.19.226.194:44083)
I20250109 08:57:03.418661 21086 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec3e3f891e3c4a879077b9ee897e5d52" candidate_uuid: "ead0b707c1014770b25c5a0db0454830" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6e2837e083814e77b16db43b99d985ab"
I20250109 08:57:03.418676 21011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec3e3f891e3c4a879077b9ee897e5d52" candidate_uuid: "ead0b707c1014770b25c5a0db0454830" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b6684bfcd7404fe68d8cb196153412d5"
I20250109 08:57:03.419175 21086 raft_consensus.cc:3054] T ec3e3f891e3c4a879077b9ee897e5d52 P 6e2837e083814e77b16db43b99d985ab [term 2 FOLLOWER]: Advancing to term 3
I20250109 08:57:03.419225 21011 raft_consensus.cc:3049] T ec3e3f891e3c4a879077b9ee897e5d52 P b6684bfcd7404fe68d8cb196153412d5 [term 2 LEADER]: Stepping down as leader of term 2
I20250109 08:57:03.419559 21011 raft_consensus.cc:738] T ec3e3f891e3c4a879077b9ee897e5d52 P b6684bfcd7404fe68d8cb196153412d5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b6684bfcd7404fe68d8cb196153412d5, State: Running, Role: LEADER
I20250109 08:57:03.420121 21011 consensus_queue.cc:260] T ec3e3f891e3c4a879077b9ee897e5d52 P b6684bfcd7404fe68d8cb196153412d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:57:03.421027 21011 raft_consensus.cc:3054] T ec3e3f891e3c4a879077b9ee897e5d52 P b6684bfcd7404fe68d8cb196153412d5 [term 2 FOLLOWER]: Advancing to term 3
I20250109 08:57:03.423400 21086 raft_consensus.cc:2463] T ec3e3f891e3c4a879077b9ee897e5d52 P 6e2837e083814e77b16db43b99d985ab [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ead0b707c1014770b25c5a0db0454830 in term 3.
I20250109 08:57:03.424335 21123 leader_election.cc:304] T ec3e3f891e3c4a879077b9ee897e5d52 P ead0b707c1014770b25c5a0db0454830 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e2837e083814e77b16db43b99d985ab, ead0b707c1014770b25c5a0db0454830; no voters:
I20250109 08:57:03.425171 21292 raft_consensus.cc:2798] T ec3e3f891e3c4a879077b9ee897e5d52 P ead0b707c1014770b25c5a0db0454830 [term 3 FOLLOWER]: Leader election won for term 3
I20250109 08:57:03.425475 21011 raft_consensus.cc:2463] T ec3e3f891e3c4a879077b9ee897e5d52 P b6684bfcd7404fe68d8cb196153412d5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ead0b707c1014770b25c5a0db0454830 in term 3.
I20250109 08:57:03.426110 21292 raft_consensus.cc:695] T ec3e3f891e3c4a879077b9ee897e5d52 P ead0b707c1014770b25c5a0db0454830 [term 3 LEADER]: Becoming Leader. State: Replica: ead0b707c1014770b25c5a0db0454830, State: Running, Role: LEADER
I20250109 08:57:03.426846 21292 consensus_queue.cc:237] T ec3e3f891e3c4a879077b9ee897e5d52 P ead0b707c1014770b25c5a0db0454830 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:57:03.432497 20907 catalog_manager.cc:5526] T ec3e3f891e3c4a879077b9ee897e5d52 P ead0b707c1014770b25c5a0db0454830 reported cstate change: term changed from 2 to 3, leader changed from b6684bfcd7404fe68d8cb196153412d5 (127.19.226.193) to ead0b707c1014770b25c5a0db0454830 (127.19.226.195). New cstate: current_term: 3 leader_uuid: "ead0b707c1014770b25c5a0db0454830" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } health_report { overall_health: HEALTHY } } }
I20250109 08:57:03.484864 21244 raft_consensus.cc:988] T 163310a12b44475ca3727662dad194a6 P b6684bfcd7404fe68d8cb196153412d5: : Instructing follower ead0b707c1014770b25c5a0db0454830 to start an election
I20250109 08:57:03.485312 21293 raft_consensus.cc:1076] T 163310a12b44475ca3727662dad194a6 P b6684bfcd7404fe68d8cb196153412d5 [term 1 LEADER]: Signalling peer ead0b707c1014770b25c5a0db0454830 to start an election
I20250109 08:57:03.486596 21160 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "163310a12b44475ca3727662dad194a6"
dest_uuid: "ead0b707c1014770b25c5a0db0454830"
from {username='slave'} at 127.0.0.1:35956
I20250109 08:57:03.487010 21160 raft_consensus.cc:491] T 163310a12b44475ca3727662dad194a6 P ead0b707c1014770b25c5a0db0454830 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:57:03.487282 21160 raft_consensus.cc:3054] T 163310a12b44475ca3727662dad194a6 P ead0b707c1014770b25c5a0db0454830 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:57:03.491008 21160 raft_consensus.cc:513] T 163310a12b44475ca3727662dad194a6 P ead0b707c1014770b25c5a0db0454830 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:57:03.492300 21160 leader_election.cc:290] T 163310a12b44475ca3727662dad194a6 P ead0b707c1014770b25c5a0db0454830 [CANDIDATE]: Term 2 election: Requested vote from peers b6684bfcd7404fe68d8cb196153412d5 (127.19.226.193:34209), 6e2837e083814e77b16db43b99d985ab (127.19.226.194:44083)
I20250109 08:57:03.493017 21011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "163310a12b44475ca3727662dad194a6" candidate_uuid: "ead0b707c1014770b25c5a0db0454830" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b6684bfcd7404fe68d8cb196153412d5"
I20250109 08:57:03.493260 21086 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "163310a12b44475ca3727662dad194a6" candidate_uuid: "ead0b707c1014770b25c5a0db0454830" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6e2837e083814e77b16db43b99d985ab"
I20250109 08:57:03.493618 21011 raft_consensus.cc:3049] T 163310a12b44475ca3727662dad194a6 P b6684bfcd7404fe68d8cb196153412d5 [term 1 LEADER]: Stepping down as leader of term 1
I20250109 08:57:03.493793 21086 raft_consensus.cc:3054] T 163310a12b44475ca3727662dad194a6 P 6e2837e083814e77b16db43b99d985ab [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:57:03.493963 21011 raft_consensus.cc:738] T 163310a12b44475ca3727662dad194a6 P b6684bfcd7404fe68d8cb196153412d5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b6684bfcd7404fe68d8cb196153412d5, State: Running, Role: LEADER
I20250109 08:57:03.494777 21011 consensus_queue.cc:260] T 163310a12b44475ca3727662dad194a6 P b6684bfcd7404fe68d8cb196153412d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:57:03.495940 21011 raft_consensus.cc:3054] T 163310a12b44475ca3727662dad194a6 P b6684bfcd7404fe68d8cb196153412d5 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:57:03.499949 21086 raft_consensus.cc:2463] T 163310a12b44475ca3727662dad194a6 P 6e2837e083814e77b16db43b99d985ab [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ead0b707c1014770b25c5a0db0454830 in term 2.
I20250109 08:57:03.500942 21123 leader_election.cc:304] T 163310a12b44475ca3727662dad194a6 P ead0b707c1014770b25c5a0db0454830 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e2837e083814e77b16db43b99d985ab, ead0b707c1014770b25c5a0db0454830; no voters:
I20250109 08:57:03.501541 21292 raft_consensus.cc:2798] T 163310a12b44475ca3727662dad194a6 P ead0b707c1014770b25c5a0db0454830 [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:57:03.501693 21011 raft_consensus.cc:2463] T 163310a12b44475ca3727662dad194a6 P b6684bfcd7404fe68d8cb196153412d5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ead0b707c1014770b25c5a0db0454830 in term 2.
I20250109 08:57:03.501902 21292 raft_consensus.cc:695] T 163310a12b44475ca3727662dad194a6 P ead0b707c1014770b25c5a0db0454830 [term 2 LEADER]: Becoming Leader. State: Replica: ead0b707c1014770b25c5a0db0454830, State: Running, Role: LEADER
I20250109 08:57:03.502605 21292 consensus_queue.cc:237] T 163310a12b44475ca3727662dad194a6 P ead0b707c1014770b25c5a0db0454830 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:57:03.509820 20907 catalog_manager.cc:5526] T 163310a12b44475ca3727662dad194a6 P ead0b707c1014770b25c5a0db0454830 reported cstate change: term changed from 1 to 2, leader changed from b6684bfcd7404fe68d8cb196153412d5 (127.19.226.193) to ead0b707c1014770b25c5a0db0454830 (127.19.226.195). New cstate: current_term: 2 leader_uuid: "ead0b707c1014770b25c5a0db0454830" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } health_report { overall_health: HEALTHY } } }
I20250109 08:57:03.882521 21086 raft_consensus.cc:1270] T 163310a12b44475ca3727662dad194a6 P 6e2837e083814e77b16db43b99d985ab [term 2 FOLLOWER]: Refusing update from remote peer ead0b707c1014770b25c5a0db0454830: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:57:03.883602 21292 consensus_queue.cc:1035] T 163310a12b44475ca3727662dad194a6 P ead0b707c1014770b25c5a0db0454830 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:57:03.886797 21011 raft_consensus.cc:1270] T 163310a12b44475ca3727662dad194a6 P b6684bfcd7404fe68d8cb196153412d5 [term 2 FOLLOWER]: Refusing update from remote peer ead0b707c1014770b25c5a0db0454830: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:57:03.888101 21292 consensus_queue.cc:1035] T 163310a12b44475ca3727662dad194a6 P ead0b707c1014770b25c5a0db0454830 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:57:03.957340 21086 raft_consensus.cc:1270] T ec3e3f891e3c4a879077b9ee897e5d52 P 6e2837e083814e77b16db43b99d985ab [term 3 FOLLOWER]: Refusing update from remote peer ead0b707c1014770b25c5a0db0454830: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20250109 08:57:03.958469 21304 consensus_queue.cc:1035] T ec3e3f891e3c4a879077b9ee897e5d52 P ead0b707c1014770b25c5a0db0454830 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:57:03.966514 21009 raft_consensus.cc:1270] T ec3e3f891e3c4a879077b9ee897e5d52 P b6684bfcd7404fe68d8cb196153412d5 [term 3 FOLLOWER]: Refusing update from remote peer ead0b707c1014770b25c5a0db0454830: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20250109 08:57:03.968091 21310 consensus_queue.cc:1035] T ec3e3f891e3c4a879077b9ee897e5d52 P ead0b707c1014770b25c5a0db0454830 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20250109 08:57:04.151757 20363 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250109 08:57:04.157109 20363 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250109 08:57:04.157497 20363 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250109 08:57:05.158221 20363 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250109 08:57:05.163583 20363 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250109 08:57:05.163990 20363 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250109 08:57:06.164762 20363 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250109 08:57:06.170459 20363 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250109 08:57:06.170852 20363 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250109 08:57:07.171645 20363 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250109 08:57:07.178076 20363 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250109 08:57:07.178612 20363 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250109 08:57:08.179551 20363 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250109 08:57:08.185369 20363 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250109 08:57:08.185760 20363 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250109 08:57:09.186508 20363 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250109 08:57:09.192592 20363 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250109 08:57:09.192976 20363 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250109 08:57:10.193756 20363 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250109 08:57:10.200016 20363 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250109 08:57:10.200446 20363 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250109 08:57:11.201246 20363 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250109 08:57:11.207160 20363 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250109 08:57:11.207661 20363 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250109 08:57:12.208467 20363 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250109 08:57:12.213394 20363 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250109 08:57:12.213743 20363 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250109 08:57:13.239905 20363 auto_leader_rebalancer-test.cc:358] The leader distribution is
I20250109 08:57:13.247800 21160 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "ec3e3f891e3c4a879077b9ee897e5d52"
dest_uuid: "ead0b707c1014770b25c5a0db0454830"
mode: GRACEFUL
new_leader_uuid: "b6684bfcd7404fe68d8cb196153412d5"
from {username='slave'} at 127.0.0.1:35984
I20250109 08:57:13.248570 21160 raft_consensus.cc:604] T ec3e3f891e3c4a879077b9ee897e5d52 P ead0b707c1014770b25c5a0db0454830 [term 3 LEADER]: Received request to transfer leadership to TS b6684bfcd7404fe68d8cb196153412d5
I20250109 08:57:13.252786 21160 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "163310a12b44475ca3727662dad194a6"
dest_uuid: "ead0b707c1014770b25c5a0db0454830"
mode: GRACEFUL
new_leader_uuid: "b6684bfcd7404fe68d8cb196153412d5"
from {username='slave'} at 127.0.0.1:35984
I20250109 08:57:13.253225 21160 raft_consensus.cc:604] T 163310a12b44475ca3727662dad194a6 P ead0b707c1014770b25c5a0db0454830 [term 2 LEADER]: Received request to transfer leadership to TS b6684bfcd7404fe68d8cb196153412d5
I20250109 08:57:13.259348 21085 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "af13c7179cc24b258e516eceb2b0cac0"
dest_uuid: "6e2837e083814e77b16db43b99d985ab"
mode: GRACEFUL
new_leader_uuid: "b6684bfcd7404fe68d8cb196153412d5"
from {username='slave'} at 127.0.0.1:44084
I20250109 08:57:13.259796 21085 raft_consensus.cc:604] T af13c7179cc24b258e516eceb2b0cac0 P 6e2837e083814e77b16db43b99d985ab [term 2 LEADER]: Received request to transfer leadership to TS b6684bfcd7404fe68d8cb196153412d5
I20250109 08:57:13.263847 21085 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "05148030cdd443048fe5899b03fd92fe"
dest_uuid: "6e2837e083814e77b16db43b99d985ab"
mode: GRACEFUL
new_leader_uuid: "b6684bfcd7404fe68d8cb196153412d5"
from {username='slave'} at 127.0.0.1:44084
I20250109 08:57:13.264400 21085 raft_consensus.cc:604] T 05148030cdd443048fe5899b03fd92fe P 6e2837e083814e77b16db43b99d985ab [term 2 LEADER]: Received request to transfer leadership to TS b6684bfcd7404fe68d8cb196153412d5
I20250109 08:57:13.264803 21321 raft_consensus.cc:988] T af13c7179cc24b258e516eceb2b0cac0 P 6e2837e083814e77b16db43b99d985ab: : Instructing follower b6684bfcd7404fe68d8cb196153412d5 to start an election
I20250109 08:57:13.265316 21315 raft_consensus.cc:1076] T af13c7179cc24b258e516eceb2b0cac0 P 6e2837e083814e77b16db43b99d985ab [term 2 LEADER]: Signalling peer b6684bfcd7404fe68d8cb196153412d5 to start an election
I20250109 08:57:13.266816 21011 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "af13c7179cc24b258e516eceb2b0cac0"
dest_uuid: "b6684bfcd7404fe68d8cb196153412d5"
from {username='slave'} at 127.0.0.1:59350
I20250109 08:57:13.267328 21011 raft_consensus.cc:491] T af13c7179cc24b258e516eceb2b0cac0 P b6684bfcd7404fe68d8cb196153412d5 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:57:13.267721 21011 raft_consensus.cc:3054] T af13c7179cc24b258e516eceb2b0cac0 P b6684bfcd7404fe68d8cb196153412d5 [term 2 FOLLOWER]: Advancing to term 3
I20250109 08:57:13.269236 21085 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "166b00d1332d4f39b57ae9f58cb7fc5f"
dest_uuid: "6e2837e083814e77b16db43b99d985ab"
mode: GRACEFUL
new_leader_uuid: "b6684bfcd7404fe68d8cb196153412d5"
from {username='slave'} at 127.0.0.1:44084
I20250109 08:57:13.269676 21085 raft_consensus.cc:604] T 166b00d1332d4f39b57ae9f58cb7fc5f P 6e2837e083814e77b16db43b99d985ab [term 2 LEADER]: Received request to transfer leadership to TS b6684bfcd7404fe68d8cb196153412d5
I20250109 08:57:13.274505 21011 raft_consensus.cc:513] T af13c7179cc24b258e516eceb2b0cac0 P b6684bfcd7404fe68d8cb196153412d5 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:57:13.276335 21011 leader_election.cc:290] T af13c7179cc24b258e516eceb2b0cac0 P b6684bfcd7404fe68d8cb196153412d5 [CANDIDATE]: Term 3 election: Requested vote from peers 6e2837e083814e77b16db43b99d985ab (127.19.226.194:44083), ead0b707c1014770b25c5a0db0454830 (127.19.226.195:35749)
I20250109 08:57:13.277037 21085 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af13c7179cc24b258e516eceb2b0cac0" candidate_uuid: "b6684bfcd7404fe68d8cb196153412d5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6e2837e083814e77b16db43b99d985ab"
I20250109 08:57:13.277276 21160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af13c7179cc24b258e516eceb2b0cac0" candidate_uuid: "b6684bfcd7404fe68d8cb196153412d5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ead0b707c1014770b25c5a0db0454830"
I20250109 08:57:13.277827 21160 raft_consensus.cc:3054] T af13c7179cc24b258e516eceb2b0cac0 P ead0b707c1014770b25c5a0db0454830 [term 2 FOLLOWER]: Advancing to term 3
I20250109 08:57:13.277776 21085 raft_consensus.cc:3049] T af13c7179cc24b258e516eceb2b0cac0 P 6e2837e083814e77b16db43b99d985ab [term 2 LEADER]: Stepping down as leader of term 2
I20250109 08:57:13.278183 21085 raft_consensus.cc:738] T af13c7179cc24b258e516eceb2b0cac0 P 6e2837e083814e77b16db43b99d985ab [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6e2837e083814e77b16db43b99d985ab, State: Running, Role: LEADER
I20250109 08:57:13.278795 21085 consensus_queue.cc:260] T af13c7179cc24b258e516eceb2b0cac0 P 6e2837e083814e77b16db43b99d985ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:57:13.279721 21085 raft_consensus.cc:3054] T af13c7179cc24b258e516eceb2b0cac0 P 6e2837e083814e77b16db43b99d985ab [term 2 FOLLOWER]: Advancing to term 3
I20250109 08:57:13.282279 21160 raft_consensus.cc:2463] T af13c7179cc24b258e516eceb2b0cac0 P ead0b707c1014770b25c5a0db0454830 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b6684bfcd7404fe68d8cb196153412d5 in term 3.
I20250109 08:57:13.283335 20974 leader_election.cc:304] T af13c7179cc24b258e516eceb2b0cac0 P b6684bfcd7404fe68d8cb196153412d5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b6684bfcd7404fe68d8cb196153412d5, ead0b707c1014770b25c5a0db0454830; no voters:
I20250109 08:57:13.284049 21317 raft_consensus.cc:2798] T af13c7179cc24b258e516eceb2b0cac0 P b6684bfcd7404fe68d8cb196153412d5 [term 3 FOLLOWER]: Leader election won for term 3
I20250109 08:57:13.284710 21085 raft_consensus.cc:2463] T af13c7179cc24b258e516eceb2b0cac0 P 6e2837e083814e77b16db43b99d985ab [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b6684bfcd7404fe68d8cb196153412d5 in term 3.
I20250109 08:57:13.284986 21317 raft_consensus.cc:695] T af13c7179cc24b258e516eceb2b0cac0 P b6684bfcd7404fe68d8cb196153412d5 [term 3 LEADER]: Becoming Leader. State: Replica: b6684bfcd7404fe68d8cb196153412d5, State: Running, Role: LEADER
I20250109 08:57:13.286007 21317 consensus_queue.cc:237] T af13c7179cc24b258e516eceb2b0cac0 P b6684bfcd7404fe68d8cb196153412d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:57:13.293678 20909 catalog_manager.cc:5526] T af13c7179cc24b258e516eceb2b0cac0 P b6684bfcd7404fe68d8cb196153412d5 reported cstate change: term changed from 2 to 3, leader changed from 6e2837e083814e77b16db43b99d985ab (127.19.226.194) to b6684bfcd7404fe68d8cb196153412d5 (127.19.226.193). New cstate: current_term: 3 leader_uuid: "b6684bfcd7404fe68d8cb196153412d5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } health_report { overall_health: UNKNOWN } } }
I20250109 08:57:13.311867 21328 raft_consensus.cc:988] T 163310a12b44475ca3727662dad194a6 P ead0b707c1014770b25c5a0db0454830: : Instructing follower b6684bfcd7404fe68d8cb196153412d5 to start an election
I20250109 08:57:13.312433 21320 raft_consensus.cc:1076] T 163310a12b44475ca3727662dad194a6 P ead0b707c1014770b25c5a0db0454830 [term 2 LEADER]: Signalling peer b6684bfcd7404fe68d8cb196153412d5 to start an election
I20250109 08:57:13.313719 21011 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "163310a12b44475ca3727662dad194a6"
dest_uuid: "b6684bfcd7404fe68d8cb196153412d5"
from {username='slave'} at 127.0.0.1:59340
I20250109 08:57:13.314296 21011 raft_consensus.cc:491] T 163310a12b44475ca3727662dad194a6 P b6684bfcd7404fe68d8cb196153412d5 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:57:13.314679 21011 raft_consensus.cc:3054] T 163310a12b44475ca3727662dad194a6 P b6684bfcd7404fe68d8cb196153412d5 [term 2 FOLLOWER]: Advancing to term 3
I20250109 08:57:13.316751 21321 raft_consensus.cc:988] T 05148030cdd443048fe5899b03fd92fe P 6e2837e083814e77b16db43b99d985ab: : Instructing follower b6684bfcd7404fe68d8cb196153412d5 to start an election
I20250109 08:57:13.317185 21321 raft_consensus.cc:1076] T 05148030cdd443048fe5899b03fd92fe P 6e2837e083814e77b16db43b99d985ab [term 2 LEADER]: Signalling peer b6684bfcd7404fe68d8cb196153412d5 to start an election
I20250109 08:57:13.318667 21009 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "05148030cdd443048fe5899b03fd92fe"
dest_uuid: "b6684bfcd7404fe68d8cb196153412d5"
from {username='slave'} at 127.0.0.1:59350
I20250109 08:57:13.319070 21009 raft_consensus.cc:491] T 05148030cdd443048fe5899b03fd92fe P b6684bfcd7404fe68d8cb196153412d5 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:57:13.319320 21009 raft_consensus.cc:3054] T 05148030cdd443048fe5899b03fd92fe P b6684bfcd7404fe68d8cb196153412d5 [term 2 FOLLOWER]: Advancing to term 3
I20250109 08:57:13.320199 21011 raft_consensus.cc:513] T 163310a12b44475ca3727662dad194a6 P b6684bfcd7404fe68d8cb196153412d5 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:57:13.321480 21011 leader_election.cc:290] T 163310a12b44475ca3727662dad194a6 P b6684bfcd7404fe68d8cb196153412d5 [CANDIDATE]: Term 3 election: Requested vote from peers 6e2837e083814e77b16db43b99d985ab (127.19.226.194:44083), ead0b707c1014770b25c5a0db0454830 (127.19.226.195:35749)
I20250109 08:57:13.322283 21085 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "163310a12b44475ca3727662dad194a6" candidate_uuid: "b6684bfcd7404fe68d8cb196153412d5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6e2837e083814e77b16db43b99d985ab"
I20250109 08:57:13.322641 21160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "163310a12b44475ca3727662dad194a6" candidate_uuid: "b6684bfcd7404fe68d8cb196153412d5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ead0b707c1014770b25c5a0db0454830"
I20250109 08:57:13.322845 21085 raft_consensus.cc:3054] T 163310a12b44475ca3727662dad194a6 P 6e2837e083814e77b16db43b99d985ab [term 2 FOLLOWER]: Advancing to term 3
I20250109 08:57:13.323163 21160 raft_consensus.cc:3049] T 163310a12b44475ca3727662dad194a6 P ead0b707c1014770b25c5a0db0454830 [term 2 LEADER]: Stepping down as leader of term 2
I20250109 08:57:13.323454 21160 raft_consensus.cc:738] T 163310a12b44475ca3727662dad194a6 P ead0b707c1014770b25c5a0db0454830 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ead0b707c1014770b25c5a0db0454830, State: Running, Role: LEADER
I20250109 08:57:13.324261 21009 raft_consensus.cc:513] T 05148030cdd443048fe5899b03fd92fe P b6684bfcd7404fe68d8cb196153412d5 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:57:13.324159 21160 consensus_queue.cc:260] T 163310a12b44475ca3727662dad194a6 P ead0b707c1014770b25c5a0db0454830 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:57:13.325455 21160 raft_consensus.cc:3054] T 163310a12b44475ca3727662dad194a6 P ead0b707c1014770b25c5a0db0454830 [term 2 FOLLOWER]: Advancing to term 3
I20250109 08:57:13.325712 21009 leader_election.cc:290] T 05148030cdd443048fe5899b03fd92fe P b6684bfcd7404fe68d8cb196153412d5 [CANDIDATE]: Term 3 election: Requested vote from peers 6e2837e083814e77b16db43b99d985ab (127.19.226.194:44083), ead0b707c1014770b25c5a0db0454830 (127.19.226.195:35749)
I20250109 08:57:13.326745 21158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05148030cdd443048fe5899b03fd92fe" candidate_uuid: "b6684bfcd7404fe68d8cb196153412d5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ead0b707c1014770b25c5a0db0454830"
I20250109 08:57:13.327215 21158 raft_consensus.cc:3054] T 05148030cdd443048fe5899b03fd92fe P ead0b707c1014770b25c5a0db0454830 [term 2 FOLLOWER]: Advancing to term 3
I20250109 08:57:13.327193 21086 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05148030cdd443048fe5899b03fd92fe" candidate_uuid: "b6684bfcd7404fe68d8cb196153412d5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6e2837e083814e77b16db43b99d985ab"
I20250109 08:57:13.327809 21086 raft_consensus.cc:3049] T 05148030cdd443048fe5899b03fd92fe P 6e2837e083814e77b16db43b99d985ab [term 2 LEADER]: Stepping down as leader of term 2
I20250109 08:57:13.328102 21086 raft_consensus.cc:738] T 05148030cdd443048fe5899b03fd92fe P 6e2837e083814e77b16db43b99d985ab [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6e2837e083814e77b16db43b99d985ab, State: Running, Role: LEADER
I20250109 08:57:13.328787 21086 consensus_queue.cc:260] T 05148030cdd443048fe5899b03fd92fe P 6e2837e083814e77b16db43b99d985ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:57:13.329916 21086 raft_consensus.cc:3054] T 05148030cdd443048fe5899b03fd92fe P 6e2837e083814e77b16db43b99d985ab [term 2 FOLLOWER]: Advancing to term 3
I20250109 08:57:13.331475 21160 raft_consensus.cc:2463] T 163310a12b44475ca3727662dad194a6 P ead0b707c1014770b25c5a0db0454830 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b6684bfcd7404fe68d8cb196153412d5 in term 3.
I20250109 08:57:13.332414 20974 leader_election.cc:304] T 163310a12b44475ca3727662dad194a6 P b6684bfcd7404fe68d8cb196153412d5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b6684bfcd7404fe68d8cb196153412d5, ead0b707c1014770b25c5a0db0454830; no voters:
I20250109 08:57:13.333004 21317 raft_consensus.cc:2798] T 163310a12b44475ca3727662dad194a6 P b6684bfcd7404fe68d8cb196153412d5 [term 3 FOLLOWER]: Leader election won for term 3
I20250109 08:57:13.333467 21317 raft_consensus.cc:695] T 163310a12b44475ca3727662dad194a6 P b6684bfcd7404fe68d8cb196153412d5 [term 3 LEADER]: Becoming Leader. State: Replica: b6684bfcd7404fe68d8cb196153412d5, State: Running, Role: LEADER
I20250109 08:57:13.333343 21158 raft_consensus.cc:2463] T 05148030cdd443048fe5899b03fd92fe P ead0b707c1014770b25c5a0db0454830 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b6684bfcd7404fe68d8cb196153412d5 in term 3.
I20250109 08:57:13.334313 21317 consensus_queue.cc:237] T 163310a12b44475ca3727662dad194a6 P b6684bfcd7404fe68d8cb196153412d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:57:13.334955 21085 raft_consensus.cc:2463] T 163310a12b44475ca3727662dad194a6 P 6e2837e083814e77b16db43b99d985ab [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b6684bfcd7404fe68d8cb196153412d5 in term 3.
I20250109 08:57:13.335992 21086 raft_consensus.cc:2463] T 05148030cdd443048fe5899b03fd92fe P 6e2837e083814e77b16db43b99d985ab [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b6684bfcd7404fe68d8cb196153412d5 in term 3.
I20250109 08:57:13.336249 20974 leader_election.cc:304] T 05148030cdd443048fe5899b03fd92fe P b6684bfcd7404fe68d8cb196153412d5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b6684bfcd7404fe68d8cb196153412d5, ead0b707c1014770b25c5a0db0454830; no voters:
I20250109 08:57:13.337147 21322 raft_consensus.cc:2798] T 05148030cdd443048fe5899b03fd92fe P b6684bfcd7404fe68d8cb196153412d5 [term 3 FOLLOWER]: Leader election won for term 3
I20250109 08:57:13.338405 21322 raft_consensus.cc:695] T 05148030cdd443048fe5899b03fd92fe P b6684bfcd7404fe68d8cb196153412d5 [term 3 LEADER]: Becoming Leader. State: Replica: b6684bfcd7404fe68d8cb196153412d5, State: Running, Role: LEADER
I20250109 08:57:13.339259 21322 consensus_queue.cc:237] T 05148030cdd443048fe5899b03fd92fe P b6684bfcd7404fe68d8cb196153412d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:57:13.345142 20909 catalog_manager.cc:5526] T 163310a12b44475ca3727662dad194a6 P b6684bfcd7404fe68d8cb196153412d5 reported cstate change: term changed from 2 to 3, leader changed from ead0b707c1014770b25c5a0db0454830 (127.19.226.195) to b6684bfcd7404fe68d8cb196153412d5 (127.19.226.193). New cstate: current_term: 3 leader_uuid: "b6684bfcd7404fe68d8cb196153412d5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } health_report { overall_health: UNKNOWN } } }
I20250109 08:57:13.355628 20909 catalog_manager.cc:5526] T 05148030cdd443048fe5899b03fd92fe P b6684bfcd7404fe68d8cb196153412d5 reported cstate change: term changed from 2 to 3, leader changed from 6e2837e083814e77b16db43b99d985ab (127.19.226.194) to b6684bfcd7404fe68d8cb196153412d5 (127.19.226.193). New cstate: current_term: 3 leader_uuid: "b6684bfcd7404fe68d8cb196153412d5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } health_report { overall_health: UNKNOWN } } }
I20250109 08:57:13.436828 21328 raft_consensus.cc:988] T ec3e3f891e3c4a879077b9ee897e5d52 P ead0b707c1014770b25c5a0db0454830: : Instructing follower b6684bfcd7404fe68d8cb196153412d5 to start an election
I20250109 08:57:13.437229 21320 raft_consensus.cc:1076] T ec3e3f891e3c4a879077b9ee897e5d52 P ead0b707c1014770b25c5a0db0454830 [term 3 LEADER]: Signalling peer b6684bfcd7404fe68d8cb196153412d5 to start an election
I20250109 08:57:13.438407 21009 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "ec3e3f891e3c4a879077b9ee897e5d52"
dest_uuid: "b6684bfcd7404fe68d8cb196153412d5"
from {username='slave'} at 127.0.0.1:59340
I20250109 08:57:13.438796 21009 raft_consensus.cc:491] T ec3e3f891e3c4a879077b9ee897e5d52 P b6684bfcd7404fe68d8cb196153412d5 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:57:13.439062 21009 raft_consensus.cc:3054] T ec3e3f891e3c4a879077b9ee897e5d52 P b6684bfcd7404fe68d8cb196153412d5 [term 3 FOLLOWER]: Advancing to term 4
I20250109 08:57:13.442912 21009 raft_consensus.cc:513] T ec3e3f891e3c4a879077b9ee897e5d52 P b6684bfcd7404fe68d8cb196153412d5 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:57:13.444183 21009 leader_election.cc:290] T ec3e3f891e3c4a879077b9ee897e5d52 P b6684bfcd7404fe68d8cb196153412d5 [CANDIDATE]: Term 4 election: Requested vote from peers 6e2837e083814e77b16db43b99d985ab (127.19.226.194:44083), ead0b707c1014770b25c5a0db0454830 (127.19.226.195:35749)
I20250109 08:57:13.444849 21086 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec3e3f891e3c4a879077b9ee897e5d52" candidate_uuid: "b6684bfcd7404fe68d8cb196153412d5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "6e2837e083814e77b16db43b99d985ab"
I20250109 08:57:13.445223 21158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec3e3f891e3c4a879077b9ee897e5d52" candidate_uuid: "b6684bfcd7404fe68d8cb196153412d5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "ead0b707c1014770b25c5a0db0454830"
I20250109 08:57:13.445374 21086 raft_consensus.cc:3054] T ec3e3f891e3c4a879077b9ee897e5d52 P 6e2837e083814e77b16db43b99d985ab [term 3 FOLLOWER]: Advancing to term 4
I20250109 08:57:13.445760 21158 raft_consensus.cc:3049] T ec3e3f891e3c4a879077b9ee897e5d52 P ead0b707c1014770b25c5a0db0454830 [term 3 LEADER]: Stepping down as leader of term 3
I20250109 08:57:13.446072 21158 raft_consensus.cc:738] T ec3e3f891e3c4a879077b9ee897e5d52 P ead0b707c1014770b25c5a0db0454830 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ead0b707c1014770b25c5a0db0454830, State: Running, Role: LEADER
I20250109 08:57:13.446796 21158 consensus_queue.cc:260] T ec3e3f891e3c4a879077b9ee897e5d52 P ead0b707c1014770b25c5a0db0454830 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:57:13.448073 21158 raft_consensus.cc:3054] T ec3e3f891e3c4a879077b9ee897e5d52 P ead0b707c1014770b25c5a0db0454830 [term 3 FOLLOWER]: Advancing to term 4
I20250109 08:57:13.451360 21086 raft_consensus.cc:2463] T ec3e3f891e3c4a879077b9ee897e5d52 P 6e2837e083814e77b16db43b99d985ab [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b6684bfcd7404fe68d8cb196153412d5 in term 4.
I20250109 08:57:13.452286 20974 leader_election.cc:304] T ec3e3f891e3c4a879077b9ee897e5d52 P b6684bfcd7404fe68d8cb196153412d5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e2837e083814e77b16db43b99d985ab, b6684bfcd7404fe68d8cb196153412d5; no voters:
I20250109 08:57:13.452980 21322 raft_consensus.cc:2798] T ec3e3f891e3c4a879077b9ee897e5d52 P b6684bfcd7404fe68d8cb196153412d5 [term 4 FOLLOWER]: Leader election won for term 4
I20250109 08:57:13.453550 21322 raft_consensus.cc:695] T ec3e3f891e3c4a879077b9ee897e5d52 P b6684bfcd7404fe68d8cb196153412d5 [term 4 LEADER]: Becoming Leader. State: Replica: b6684bfcd7404fe68d8cb196153412d5, State: Running, Role: LEADER
I20250109 08:57:13.453835 21158 raft_consensus.cc:2463] T ec3e3f891e3c4a879077b9ee897e5d52 P ead0b707c1014770b25c5a0db0454830 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b6684bfcd7404fe68d8cb196153412d5 in term 4.
I20250109 08:57:13.454334 21322 consensus_queue.cc:237] T ec3e3f891e3c4a879077b9ee897e5d52 P b6684bfcd7404fe68d8cb196153412d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:57:13.460189 20909 catalog_manager.cc:5526] T ec3e3f891e3c4a879077b9ee897e5d52 P b6684bfcd7404fe68d8cb196153412d5 reported cstate change: term changed from 3 to 4, leader changed from ead0b707c1014770b25c5a0db0454830 (127.19.226.195) to b6684bfcd7404fe68d8cb196153412d5 (127.19.226.193). New cstate: current_term: 4 leader_uuid: "b6684bfcd7404fe68d8cb196153412d5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } health_report { overall_health: UNKNOWN } } }
I20250109 08:57:13.669297 21321 raft_consensus.cc:988] T 166b00d1332d4f39b57ae9f58cb7fc5f P 6e2837e083814e77b16db43b99d985ab: : Instructing follower b6684bfcd7404fe68d8cb196153412d5 to start an election
I20250109 08:57:13.669723 21315 raft_consensus.cc:1076] T 166b00d1332d4f39b57ae9f58cb7fc5f P 6e2837e083814e77b16db43b99d985ab [term 2 LEADER]: Signalling peer b6684bfcd7404fe68d8cb196153412d5 to start an election
I20250109 08:57:13.671089 21009 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "166b00d1332d4f39b57ae9f58cb7fc5f"
dest_uuid: "b6684bfcd7404fe68d8cb196153412d5"
from {username='slave'} at 127.0.0.1:59350
I20250109 08:57:13.671532 21009 raft_consensus.cc:491] T 166b00d1332d4f39b57ae9f58cb7fc5f P b6684bfcd7404fe68d8cb196153412d5 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:57:13.671762 21009 raft_consensus.cc:3054] T 166b00d1332d4f39b57ae9f58cb7fc5f P b6684bfcd7404fe68d8cb196153412d5 [term 2 FOLLOWER]: Advancing to term 3
I20250109 08:57:13.675673 21009 raft_consensus.cc:513] T 166b00d1332d4f39b57ae9f58cb7fc5f P b6684bfcd7404fe68d8cb196153412d5 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:57:13.677068 21009 leader_election.cc:290] T 166b00d1332d4f39b57ae9f58cb7fc5f P b6684bfcd7404fe68d8cb196153412d5 [CANDIDATE]: Term 3 election: Requested vote from peers 6e2837e083814e77b16db43b99d985ab (127.19.226.194:44083), ead0b707c1014770b25c5a0db0454830 (127.19.226.195:35749)
I20250109 08:57:13.677852 21086 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "166b00d1332d4f39b57ae9f58cb7fc5f" candidate_uuid: "b6684bfcd7404fe68d8cb196153412d5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6e2837e083814e77b16db43b99d985ab"
I20250109 08:57:13.678099 21158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "166b00d1332d4f39b57ae9f58cb7fc5f" candidate_uuid: "b6684bfcd7404fe68d8cb196153412d5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ead0b707c1014770b25c5a0db0454830"
I20250109 08:57:13.678359 21086 raft_consensus.cc:3049] T 166b00d1332d4f39b57ae9f58cb7fc5f P 6e2837e083814e77b16db43b99d985ab [term 2 LEADER]: Stepping down as leader of term 2
I20250109 08:57:13.678565 21158 raft_consensus.cc:3054] T 166b00d1332d4f39b57ae9f58cb7fc5f P ead0b707c1014770b25c5a0db0454830 [term 2 FOLLOWER]: Advancing to term 3
I20250109 08:57:13.678617 21086 raft_consensus.cc:738] T 166b00d1332d4f39b57ae9f58cb7fc5f P 6e2837e083814e77b16db43b99d985ab [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6e2837e083814e77b16db43b99d985ab, State: Running, Role: LEADER
I20250109 08:57:13.679255 21086 consensus_queue.cc:260] T 166b00d1332d4f39b57ae9f58cb7fc5f P 6e2837e083814e77b16db43b99d985ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:57:13.680075 21086 raft_consensus.cc:3054] T 166b00d1332d4f39b57ae9f58cb7fc5f P 6e2837e083814e77b16db43b99d985ab [term 2 FOLLOWER]: Advancing to term 3
I20250109 08:57:13.683056 21158 raft_consensus.cc:2463] T 166b00d1332d4f39b57ae9f58cb7fc5f P ead0b707c1014770b25c5a0db0454830 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b6684bfcd7404fe68d8cb196153412d5 in term 3.
I20250109 08:57:13.684028 20974 leader_election.cc:304] T 166b00d1332d4f39b57ae9f58cb7fc5f P b6684bfcd7404fe68d8cb196153412d5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b6684bfcd7404fe68d8cb196153412d5, ead0b707c1014770b25c5a0db0454830; no voters:
I20250109 08:57:13.684113 21086 raft_consensus.cc:2463] T 166b00d1332d4f39b57ae9f58cb7fc5f P 6e2837e083814e77b16db43b99d985ab [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b6684bfcd7404fe68d8cb196153412d5 in term 3.
I20250109 08:57:13.684670 21322 raft_consensus.cc:2798] T 166b00d1332d4f39b57ae9f58cb7fc5f P b6684bfcd7404fe68d8cb196153412d5 [term 3 FOLLOWER]: Leader election won for term 3
I20250109 08:57:13.685112 21322 raft_consensus.cc:695] T 166b00d1332d4f39b57ae9f58cb7fc5f P b6684bfcd7404fe68d8cb196153412d5 [term 3 LEADER]: Becoming Leader. State: Replica: b6684bfcd7404fe68d8cb196153412d5, State: Running, Role: LEADER
I20250109 08:57:13.685786 21322 consensus_queue.cc:237] T 166b00d1332d4f39b57ae9f58cb7fc5f P b6684bfcd7404fe68d8cb196153412d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:57:13.692584 20909 catalog_manager.cc:5526] T 166b00d1332d4f39b57ae9f58cb7fc5f P b6684bfcd7404fe68d8cb196153412d5 reported cstate change: term changed from 2 to 3, leader changed from 6e2837e083814e77b16db43b99d985ab (127.19.226.194) to b6684bfcd7404fe68d8cb196153412d5 (127.19.226.193). New cstate: current_term: 3 leader_uuid: "b6684bfcd7404fe68d8cb196153412d5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } health_report { overall_health: UNKNOWN } } }
I20250109 08:57:13.715139 21086 raft_consensus.cc:1270] T af13c7179cc24b258e516eceb2b0cac0 P 6e2837e083814e77b16db43b99d985ab [term 3 FOLLOWER]: Refusing update from remote peer b6684bfcd7404fe68d8cb196153412d5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20250109 08:57:13.716427 21322 consensus_queue.cc:1035] T af13c7179cc24b258e516eceb2b0cac0 P b6684bfcd7404fe68d8cb196153412d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:57:13.726871 21158 raft_consensus.cc:1270] T af13c7179cc24b258e516eceb2b0cac0 P ead0b707c1014770b25c5a0db0454830 [term 3 FOLLOWER]: Refusing update from remote peer b6684bfcd7404fe68d8cb196153412d5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20250109 08:57:13.728569 21086 raft_consensus.cc:1270] T 05148030cdd443048fe5899b03fd92fe P 6e2837e083814e77b16db43b99d985ab [term 3 FOLLOWER]: Refusing update from remote peer b6684bfcd7404fe68d8cb196153412d5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20250109 08:57:13.728659 21330 consensus_queue.cc:1035] T af13c7179cc24b258e516eceb2b0cac0 P b6684bfcd7404fe68d8cb196153412d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20250109 08:57:13.729916 21330 consensus_queue.cc:1035] T 05148030cdd443048fe5899b03fd92fe P b6684bfcd7404fe68d8cb196153412d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:57:13.741357 21160 raft_consensus.cc:1270] T 05148030cdd443048fe5899b03fd92fe P ead0b707c1014770b25c5a0db0454830 [term 3 FOLLOWER]: Refusing update from remote peer b6684bfcd7404fe68d8cb196153412d5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20250109 08:57:13.742731 21322 consensus_queue.cc:1035] T 05148030cdd443048fe5899b03fd92fe P b6684bfcd7404fe68d8cb196153412d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:57:13.791018 21160 raft_consensus.cc:1270] T 163310a12b44475ca3727662dad194a6 P ead0b707c1014770b25c5a0db0454830 [term 3 FOLLOWER]: Refusing update from remote peer b6684bfcd7404fe68d8cb196153412d5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20250109 08:57:13.792049 21322 consensus_queue.cc:1035] T 163310a12b44475ca3727662dad194a6 P b6684bfcd7404fe68d8cb196153412d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:57:13.800448 21086 raft_consensus.cc:1270] T 163310a12b44475ca3727662dad194a6 P 6e2837e083814e77b16db43b99d985ab [term 3 FOLLOWER]: Refusing update from remote peer b6684bfcd7404fe68d8cb196153412d5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20250109 08:57:13.802038 21317 consensus_queue.cc:1035] T 163310a12b44475ca3727662dad194a6 P b6684bfcd7404fe68d8cb196153412d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:57:13.930899 21086 raft_consensus.cc:1270] T ec3e3f891e3c4a879077b9ee897e5d52 P 6e2837e083814e77b16db43b99d985ab [term 4 FOLLOWER]: Refusing update from remote peer b6684bfcd7404fe68d8cb196153412d5: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20250109 08:57:13.931960 21317 consensus_queue.cc:1035] T ec3e3f891e3c4a879077b9ee897e5d52 P b6684bfcd7404fe68d8cb196153412d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250109 08:57:13.941157 21160 raft_consensus.cc:1270] T ec3e3f891e3c4a879077b9ee897e5d52 P ead0b707c1014770b25c5a0db0454830 [term 4 FOLLOWER]: Refusing update from remote peer b6684bfcd7404fe68d8cb196153412d5: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20250109 08:57:13.942521 21317 consensus_queue.cc:1035] T ec3e3f891e3c4a879077b9ee897e5d52 P b6684bfcd7404fe68d8cb196153412d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250109 08:57:14.234619 21086 raft_consensus.cc:1270] T 166b00d1332d4f39b57ae9f58cb7fc5f P 6e2837e083814e77b16db43b99d985ab [term 3 FOLLOWER]: Refusing update from remote peer b6684bfcd7404fe68d8cb196153412d5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20250109 08:57:14.235829 21344 consensus_queue.cc:1035] T 166b00d1332d4f39b57ae9f58cb7fc5f P b6684bfcd7404fe68d8cb196153412d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:57:14.245805 21160 raft_consensus.cc:1270] T 166b00d1332d4f39b57ae9f58cb7fc5f P ead0b707c1014770b25c5a0db0454830 [term 3 FOLLOWER]: Refusing update from remote peer b6684bfcd7404fe68d8cb196153412d5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20250109 08:57:14.247157 21344 consensus_queue.cc:1035] T 166b00d1332d4f39b57ae9f58cb7fc5f P b6684bfcd7404fe68d8cb196153412d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:57:16.302585 20363 auto_leader_rebalancer-test.cc:373] The leader distribution is
I20250109 08:57:16.303260 20363 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250109 08:57:16.309824 21009 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "166b00d1332d4f39b57ae9f58cb7fc5f"
dest_uuid: "b6684bfcd7404fe68d8cb196153412d5"
mode: GRACEFUL
new_leader_uuid: "ead0b707c1014770b25c5a0db0454830"
from {username='slave'} at 127.0.0.1:59356
I20250109 08:57:16.310300 21009 raft_consensus.cc:604] T 166b00d1332d4f39b57ae9f58cb7fc5f P b6684bfcd7404fe68d8cb196153412d5 [term 3 LEADER]: Received request to transfer leadership to TS ead0b707c1014770b25c5a0db0454830
I20250109 08:57:16.312238 21009 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "42af5f76a37645b5a3865739d54a6cb3"
dest_uuid: "b6684bfcd7404fe68d8cb196153412d5"
mode: GRACEFUL
new_leader_uuid: "6e2837e083814e77b16db43b99d985ab"
from {username='slave'} at 127.0.0.1:59356
I20250109 08:57:16.312659 21009 raft_consensus.cc:604] T 42af5f76a37645b5a3865739d54a6cb3 P b6684bfcd7404fe68d8cb196153412d5 [term 1 LEADER]: Received request to transfer leadership to TS 6e2837e083814e77b16db43b99d985ab
I20250109 08:57:16.314378 21009 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "502ebc25110a4c04a71b8b2e8eea07d4"
dest_uuid: "b6684bfcd7404fe68d8cb196153412d5"
mode: GRACEFUL
new_leader_uuid: "ead0b707c1014770b25c5a0db0454830"
from {username='slave'} at 127.0.0.1:59356
I20250109 08:57:16.314754 21009 raft_consensus.cc:604] T 502ebc25110a4c04a71b8b2e8eea07d4 P b6684bfcd7404fe68d8cb196153412d5 [term 2 LEADER]: Received request to transfer leadership to TS ead0b707c1014770b25c5a0db0454830
I20250109 08:57:16.316447 21009 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "ec3e3f891e3c4a879077b9ee897e5d52"
dest_uuid: "b6684bfcd7404fe68d8cb196153412d5"
mode: GRACEFUL
new_leader_uuid: "6e2837e083814e77b16db43b99d985ab"
from {username='slave'} at 127.0.0.1:59356
I20250109 08:57:16.316852 21009 raft_consensus.cc:604] T ec3e3f891e3c4a879077b9ee897e5d52 P b6684bfcd7404fe68d8cb196153412d5 [term 4 LEADER]: Received request to transfer leadership to TS 6e2837e083814e77b16db43b99d985ab
I20250109 08:57:16.317602 20363 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 4
I20250109 08:57:16.318132 20363 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250109 08:57:16.611873 21348 raft_consensus.cc:988] T 502ebc25110a4c04a71b8b2e8eea07d4 P b6684bfcd7404fe68d8cb196153412d5: : Instructing follower ead0b707c1014770b25c5a0db0454830 to start an election
I20250109 08:57:16.612412 21349 raft_consensus.cc:1076] T 502ebc25110a4c04a71b8b2e8eea07d4 P b6684bfcd7404fe68d8cb196153412d5 [term 2 LEADER]: Signalling peer ead0b707c1014770b25c5a0db0454830 to start an election
I20250109 08:57:16.613596 21158 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "502ebc25110a4c04a71b8b2e8eea07d4"
dest_uuid: "ead0b707c1014770b25c5a0db0454830"
from {username='slave'} at 127.0.0.1:35956
I20250109 08:57:16.613922 21349 raft_consensus.cc:988] T ec3e3f891e3c4a879077b9ee897e5d52 P b6684bfcd7404fe68d8cb196153412d5: : Instructing follower 6e2837e083814e77b16db43b99d985ab to start an election
I20250109 08:57:16.614094 21158 raft_consensus.cc:491] T 502ebc25110a4c04a71b8b2e8eea07d4 P ead0b707c1014770b25c5a0db0454830 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:57:16.614368 21348 raft_consensus.cc:1076] T ec3e3f891e3c4a879077b9ee897e5d52 P b6684bfcd7404fe68d8cb196153412d5 [term 4 LEADER]: Signalling peer 6e2837e083814e77b16db43b99d985ab to start an election
I20250109 08:57:16.614503 21158 raft_consensus.cc:3054] T 502ebc25110a4c04a71b8b2e8eea07d4 P ead0b707c1014770b25c5a0db0454830 [term 2 FOLLOWER]: Advancing to term 3
I20250109 08:57:16.615630 21086 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "ec3e3f891e3c4a879077b9ee897e5d52"
dest_uuid: "6e2837e083814e77b16db43b99d985ab"
from {username='slave'} at 127.0.0.1:44078
I20250109 08:57:16.616065 21086 raft_consensus.cc:491] T ec3e3f891e3c4a879077b9ee897e5d52 P 6e2837e083814e77b16db43b99d985ab [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:57:16.616398 21086 raft_consensus.cc:3054] T ec3e3f891e3c4a879077b9ee897e5d52 P 6e2837e083814e77b16db43b99d985ab [term 4 FOLLOWER]: Advancing to term 5
I20250109 08:57:16.619246 21158 raft_consensus.cc:513] T 502ebc25110a4c04a71b8b2e8eea07d4 P ead0b707c1014770b25c5a0db0454830 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } }
I20250109 08:57:16.621011 21086 raft_consensus.cc:513] T ec3e3f891e3c4a879077b9ee897e5d52 P 6e2837e083814e77b16db43b99d985ab [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:57:16.621537 21158 leader_election.cc:290] T 502ebc25110a4c04a71b8b2e8eea07d4 P ead0b707c1014770b25c5a0db0454830 [CANDIDATE]: Term 3 election: Requested vote from peers b6684bfcd7404fe68d8cb196153412d5 (127.19.226.193:34209), 6e2837e083814e77b16db43b99d985ab (127.19.226.194:44083)
I20250109 08:57:16.621886 21009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "502ebc25110a4c04a71b8b2e8eea07d4" candidate_uuid: "ead0b707c1014770b25c5a0db0454830" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b6684bfcd7404fe68d8cb196153412d5"
I20250109 08:57:16.622169 21085 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "502ebc25110a4c04a71b8b2e8eea07d4" candidate_uuid: "ead0b707c1014770b25c5a0db0454830" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6e2837e083814e77b16db43b99d985ab"
I20250109 08:57:16.622363 21009 raft_consensus.cc:3049] T 502ebc25110a4c04a71b8b2e8eea07d4 P b6684bfcd7404fe68d8cb196153412d5 [term 2 LEADER]: Stepping down as leader of term 2
I20250109 08:57:16.622746 21085 raft_consensus.cc:3054] T 502ebc25110a4c04a71b8b2e8eea07d4 P 6e2837e083814e77b16db43b99d985ab [term 2 FOLLOWER]: Advancing to term 3
I20250109 08:57:16.622764 21009 raft_consensus.cc:738] T 502ebc25110a4c04a71b8b2e8eea07d4 P b6684bfcd7404fe68d8cb196153412d5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b6684bfcd7404fe68d8cb196153412d5, State: Running, Role: LEADER
I20250109 08:57:16.623638 21009 consensus_queue.cc:260] T 502ebc25110a4c04a71b8b2e8eea07d4 P b6684bfcd7404fe68d8cb196153412d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } }
I20250109 08:57:16.624068 21086 leader_election.cc:290] T ec3e3f891e3c4a879077b9ee897e5d52 P 6e2837e083814e77b16db43b99d985ab [CANDIDATE]: Term 5 election: Requested vote from peers b6684bfcd7404fe68d8cb196153412d5 (127.19.226.193:34209), ead0b707c1014770b25c5a0db0454830 (127.19.226.195:35749)
I20250109 08:57:16.624482 21011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec3e3f891e3c4a879077b9ee897e5d52" candidate_uuid: "6e2837e083814e77b16db43b99d985ab" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "b6684bfcd7404fe68d8cb196153412d5"
I20250109 08:57:16.624774 21009 raft_consensus.cc:3054] T 502ebc25110a4c04a71b8b2e8eea07d4 P b6684bfcd7404fe68d8cb196153412d5 [term 2 FOLLOWER]: Advancing to term 3
I20250109 08:57:16.625006 21011 raft_consensus.cc:3049] T ec3e3f891e3c4a879077b9ee897e5d52 P b6684bfcd7404fe68d8cb196153412d5 [term 4 LEADER]: Stepping down as leader of term 4
I20250109 08:57:16.625340 21011 raft_consensus.cc:738] T ec3e3f891e3c4a879077b9ee897e5d52 P b6684bfcd7404fe68d8cb196153412d5 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: b6684bfcd7404fe68d8cb196153412d5, State: Running, Role: LEADER
I20250109 08:57:16.625284 21158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec3e3f891e3c4a879077b9ee897e5d52" candidate_uuid: "6e2837e083814e77b16db43b99d985ab" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "ead0b707c1014770b25c5a0db0454830"
I20250109 08:57:16.625844 21158 raft_consensus.cc:3054] T ec3e3f891e3c4a879077b9ee897e5d52 P ead0b707c1014770b25c5a0db0454830 [term 4 FOLLOWER]: Advancing to term 5
I20250109 08:57:16.626048 21011 consensus_queue.cc:260] T ec3e3f891e3c4a879077b9ee897e5d52 P b6684bfcd7404fe68d8cb196153412d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:57:16.627038 21011 raft_consensus.cc:3054] T ec3e3f891e3c4a879077b9ee897e5d52 P b6684bfcd7404fe68d8cb196153412d5 [term 4 FOLLOWER]: Advancing to term 5
I20250109 08:57:16.630774 21085 raft_consensus.cc:2463] T 502ebc25110a4c04a71b8b2e8eea07d4 P 6e2837e083814e77b16db43b99d985ab [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ead0b707c1014770b25c5a0db0454830 in term 3.
I20250109 08:57:16.631902 21009 raft_consensus.cc:2463] T 502ebc25110a4c04a71b8b2e8eea07d4 P b6684bfcd7404fe68d8cb196153412d5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ead0b707c1014770b25c5a0db0454830 in term 3.
I20250109 08:57:16.631995 21123 leader_election.cc:304] T 502ebc25110a4c04a71b8b2e8eea07d4 P ead0b707c1014770b25c5a0db0454830 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e2837e083814e77b16db43b99d985ab, ead0b707c1014770b25c5a0db0454830; no voters:
I20250109 08:57:16.632195 21158 raft_consensus.cc:2463] T ec3e3f891e3c4a879077b9ee897e5d52 P ead0b707c1014770b25c5a0db0454830 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e2837e083814e77b16db43b99d985ab in term 5.
I20250109 08:57:16.632925 21328 raft_consensus.cc:2798] T 502ebc25110a4c04a71b8b2e8eea07d4 P ead0b707c1014770b25c5a0db0454830 [term 3 FOLLOWER]: Leader election won for term 3
I20250109 08:57:16.633567 21049 leader_election.cc:304] T ec3e3f891e3c4a879077b9ee897e5d52 P 6e2837e083814e77b16db43b99d985ab [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e2837e083814e77b16db43b99d985ab, ead0b707c1014770b25c5a0db0454830; no voters:
I20250109 08:57:16.634012 21328 raft_consensus.cc:695] T 502ebc25110a4c04a71b8b2e8eea07d4 P ead0b707c1014770b25c5a0db0454830 [term 3 LEADER]: Becoming Leader. State: Replica: ead0b707c1014770b25c5a0db0454830, State: Running, Role: LEADER
I20250109 08:57:16.634317 21315 raft_consensus.cc:2798] T ec3e3f891e3c4a879077b9ee897e5d52 P 6e2837e083814e77b16db43b99d985ab [term 5 FOLLOWER]: Leader election won for term 5
I20250109 08:57:16.634815 21328 consensus_queue.cc:237] T 502ebc25110a4c04a71b8b2e8eea07d4 P ead0b707c1014770b25c5a0db0454830 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } }
I20250109 08:57:16.635316 21315 raft_consensus.cc:695] T ec3e3f891e3c4a879077b9ee897e5d52 P 6e2837e083814e77b16db43b99d985ab [term 5 LEADER]: Becoming Leader. State: Replica: 6e2837e083814e77b16db43b99d985ab, State: Running, Role: LEADER
I20250109 08:57:16.636046 21315 consensus_queue.cc:237] T ec3e3f891e3c4a879077b9ee897e5d52 P 6e2837e083814e77b16db43b99d985ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:57:16.639722 21011 raft_consensus.cc:2463] T ec3e3f891e3c4a879077b9ee897e5d52 P b6684bfcd7404fe68d8cb196153412d5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e2837e083814e77b16db43b99d985ab in term 5.
I20250109 08:57:16.643955 20909 catalog_manager.cc:5526] T ec3e3f891e3c4a879077b9ee897e5d52 P 6e2837e083814e77b16db43b99d985ab reported cstate change: term changed from 4 to 5, leader changed from b6684bfcd7404fe68d8cb196153412d5 (127.19.226.193) to 6e2837e083814e77b16db43b99d985ab (127.19.226.194). New cstate: current_term: 5 leader_uuid: "6e2837e083814e77b16db43b99d985ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } health_report { overall_health: UNKNOWN } } }
I20250109 08:57:16.646026 20907 catalog_manager.cc:5526] T 502ebc25110a4c04a71b8b2e8eea07d4 P ead0b707c1014770b25c5a0db0454830 reported cstate change: term changed from 2 to 3, leader changed from b6684bfcd7404fe68d8cb196153412d5 (127.19.226.193) to ead0b707c1014770b25c5a0db0454830 (127.19.226.195). New cstate: current_term: 3 leader_uuid: "ead0b707c1014770b25c5a0db0454830" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } health_report { overall_health: UNKNOWN } } }
I20250109 08:57:16.730403 21349 raft_consensus.cc:988] T 166b00d1332d4f39b57ae9f58cb7fc5f P b6684bfcd7404fe68d8cb196153412d5: : Instructing follower ead0b707c1014770b25c5a0db0454830 to start an election
I20250109 08:57:16.730803 21348 raft_consensus.cc:1076] T 166b00d1332d4f39b57ae9f58cb7fc5f P b6684bfcd7404fe68d8cb196153412d5 [term 3 LEADER]: Signalling peer ead0b707c1014770b25c5a0db0454830 to start an election
I20250109 08:57:16.732062 21158 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "166b00d1332d4f39b57ae9f58cb7fc5f"
dest_uuid: "ead0b707c1014770b25c5a0db0454830"
from {username='slave'} at 127.0.0.1:35956
I20250109 08:57:16.732539 21158 raft_consensus.cc:491] T 166b00d1332d4f39b57ae9f58cb7fc5f P ead0b707c1014770b25c5a0db0454830 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:57:16.732805 21158 raft_consensus.cc:3054] T 166b00d1332d4f39b57ae9f58cb7fc5f P ead0b707c1014770b25c5a0db0454830 [term 3 FOLLOWER]: Advancing to term 4
I20250109 08:57:16.737144 21158 raft_consensus.cc:513] T 166b00d1332d4f39b57ae9f58cb7fc5f P ead0b707c1014770b25c5a0db0454830 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:57:16.738575 21158 leader_election.cc:290] T 166b00d1332d4f39b57ae9f58cb7fc5f P ead0b707c1014770b25c5a0db0454830 [CANDIDATE]: Term 4 election: Requested vote from peers b6684bfcd7404fe68d8cb196153412d5 (127.19.226.193:34209), 6e2837e083814e77b16db43b99d985ab (127.19.226.194:44083)
I20250109 08:57:16.739404 21011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "166b00d1332d4f39b57ae9f58cb7fc5f" candidate_uuid: "ead0b707c1014770b25c5a0db0454830" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "b6684bfcd7404fe68d8cb196153412d5"
I20250109 08:57:16.739461 21085 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "166b00d1332d4f39b57ae9f58cb7fc5f" candidate_uuid: "ead0b707c1014770b25c5a0db0454830" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "6e2837e083814e77b16db43b99d985ab"
I20250109 08:57:16.739943 21011 raft_consensus.cc:3049] T 166b00d1332d4f39b57ae9f58cb7fc5f P b6684bfcd7404fe68d8cb196153412d5 [term 3 LEADER]: Stepping down as leader of term 3
I20250109 08:57:16.740087 21085 raft_consensus.cc:3054] T 166b00d1332d4f39b57ae9f58cb7fc5f P 6e2837e083814e77b16db43b99d985ab [term 3 FOLLOWER]: Advancing to term 4
I20250109 08:57:16.740229 21011 raft_consensus.cc:738] T 166b00d1332d4f39b57ae9f58cb7fc5f P b6684bfcd7404fe68d8cb196153412d5 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b6684bfcd7404fe68d8cb196153412d5, State: Running, Role: LEADER
I20250109 08:57:16.740919 21011 consensus_queue.cc:260] T 166b00d1332d4f39b57ae9f58cb7fc5f P b6684bfcd7404fe68d8cb196153412d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:57:16.741724 21011 raft_consensus.cc:3054] T 166b00d1332d4f39b57ae9f58cb7fc5f P b6684bfcd7404fe68d8cb196153412d5 [term 3 FOLLOWER]: Advancing to term 4
I20250109 08:57:16.744493 21085 raft_consensus.cc:2463] T 166b00d1332d4f39b57ae9f58cb7fc5f P 6e2837e083814e77b16db43b99d985ab [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ead0b707c1014770b25c5a0db0454830 in term 4.
I20250109 08:57:16.745373 21123 leader_election.cc:304] T 166b00d1332d4f39b57ae9f58cb7fc5f P ead0b707c1014770b25c5a0db0454830 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e2837e083814e77b16db43b99d985ab, ead0b707c1014770b25c5a0db0454830; no voters:
I20250109 08:57:16.745834 21011 raft_consensus.cc:2463] T 166b00d1332d4f39b57ae9f58cb7fc5f P b6684bfcd7404fe68d8cb196153412d5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ead0b707c1014770b25c5a0db0454830 in term 4.
I20250109 08:57:16.746052 21328 raft_consensus.cc:2798] T 166b00d1332d4f39b57ae9f58cb7fc5f P ead0b707c1014770b25c5a0db0454830 [term 4 FOLLOWER]: Leader election won for term 4
I20250109 08:57:16.746515 21328 raft_consensus.cc:695] T 166b00d1332d4f39b57ae9f58cb7fc5f P ead0b707c1014770b25c5a0db0454830 [term 4 LEADER]: Becoming Leader. State: Replica: ead0b707c1014770b25c5a0db0454830, State: Running, Role: LEADER
I20250109 08:57:16.747318 21328 consensus_queue.cc:237] T 166b00d1332d4f39b57ae9f58cb7fc5f P ead0b707c1014770b25c5a0db0454830 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:57:16.751232 21349 raft_consensus.cc:988] T 42af5f76a37645b5a3865739d54a6cb3 P b6684bfcd7404fe68d8cb196153412d5: : Instructing follower 6e2837e083814e77b16db43b99d985ab to start an election
I20250109 08:57:16.751727 21348 raft_consensus.cc:1076] T 42af5f76a37645b5a3865739d54a6cb3 P b6684bfcd7404fe68d8cb196153412d5 [term 1 LEADER]: Signalling peer 6e2837e083814e77b16db43b99d985ab to start an election
I20250109 08:57:16.753023 21085 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "42af5f76a37645b5a3865739d54a6cb3"
dest_uuid: "6e2837e083814e77b16db43b99d985ab"
from {username='slave'} at 127.0.0.1:44078
I20250109 08:57:16.753538 21085 raft_consensus.cc:491] T 42af5f76a37645b5a3865739d54a6cb3 P 6e2837e083814e77b16db43b99d985ab [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:57:16.753932 21085 raft_consensus.cc:3054] T 42af5f76a37645b5a3865739d54a6cb3 P 6e2837e083814e77b16db43b99d985ab [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:57:16.754681 20907 catalog_manager.cc:5526] T 166b00d1332d4f39b57ae9f58cb7fc5f P ead0b707c1014770b25c5a0db0454830 reported cstate change: term changed from 3 to 4, leader changed from b6684bfcd7404fe68d8cb196153412d5 (127.19.226.193) to ead0b707c1014770b25c5a0db0454830 (127.19.226.195). New cstate: current_term: 4 leader_uuid: "ead0b707c1014770b25c5a0db0454830" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } health_report { overall_health: HEALTHY } } }
I20250109 08:57:16.758983 21085 raft_consensus.cc:513] T 42af5f76a37645b5a3865739d54a6cb3 P 6e2837e083814e77b16db43b99d985ab [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:57:16.761044 21085 leader_election.cc:290] T 42af5f76a37645b5a3865739d54a6cb3 P 6e2837e083814e77b16db43b99d985ab [CANDIDATE]: Term 2 election: Requested vote from peers b6684bfcd7404fe68d8cb196153412d5 (127.19.226.193:34209), ead0b707c1014770b25c5a0db0454830 (127.19.226.195:35749)
I20250109 08:57:16.761200 21011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42af5f76a37645b5a3865739d54a6cb3" candidate_uuid: "6e2837e083814e77b16db43b99d985ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b6684bfcd7404fe68d8cb196153412d5"
I20250109 08:57:16.761363 21158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42af5f76a37645b5a3865739d54a6cb3" candidate_uuid: "6e2837e083814e77b16db43b99d985ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ead0b707c1014770b25c5a0db0454830"
I20250109 08:57:16.761749 21011 raft_consensus.cc:3049] T 42af5f76a37645b5a3865739d54a6cb3 P b6684bfcd7404fe68d8cb196153412d5 [term 1 LEADER]: Stepping down as leader of term 1
I20250109 08:57:16.762151 21011 raft_consensus.cc:738] T 42af5f76a37645b5a3865739d54a6cb3 P b6684bfcd7404fe68d8cb196153412d5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b6684bfcd7404fe68d8cb196153412d5, State: Running, Role: LEADER
I20250109 08:57:16.762729 21158 raft_consensus.cc:3054] T 42af5f76a37645b5a3865739d54a6cb3 P ead0b707c1014770b25c5a0db0454830 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:57:16.762799 21011 consensus_queue.cc:260] T 42af5f76a37645b5a3865739d54a6cb3 P b6684bfcd7404fe68d8cb196153412d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:57:16.763721 21011 raft_consensus.cc:3054] T 42af5f76a37645b5a3865739d54a6cb3 P b6684bfcd7404fe68d8cb196153412d5 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:57:16.767374 21158 raft_consensus.cc:2463] T 42af5f76a37645b5a3865739d54a6cb3 P ead0b707c1014770b25c5a0db0454830 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e2837e083814e77b16db43b99d985ab in term 2.
I20250109 08:57:16.767640 21011 raft_consensus.cc:2463] T 42af5f76a37645b5a3865739d54a6cb3 P b6684bfcd7404fe68d8cb196153412d5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e2837e083814e77b16db43b99d985ab in term 2.
I20250109 08:57:16.768180 21049 leader_election.cc:304] T 42af5f76a37645b5a3865739d54a6cb3 P 6e2837e083814e77b16db43b99d985ab [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e2837e083814e77b16db43b99d985ab, ead0b707c1014770b25c5a0db0454830; no voters:
I20250109 08:57:16.768774 21315 raft_consensus.cc:2798] T 42af5f76a37645b5a3865739d54a6cb3 P 6e2837e083814e77b16db43b99d985ab [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:57:16.769138 21315 raft_consensus.cc:695] T 42af5f76a37645b5a3865739d54a6cb3 P 6e2837e083814e77b16db43b99d985ab [term 2 LEADER]: Becoming Leader. State: Replica: 6e2837e083814e77b16db43b99d985ab, State: Running, Role: LEADER
I20250109 08:57:16.769768 21315 consensus_queue.cc:237] T 42af5f76a37645b5a3865739d54a6cb3 P 6e2837e083814e77b16db43b99d985ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } }
I20250109 08:57:16.775985 20907 catalog_manager.cc:5526] T 42af5f76a37645b5a3865739d54a6cb3 P 6e2837e083814e77b16db43b99d985ab reported cstate change: term changed from 1 to 2, leader changed from b6684bfcd7404fe68d8cb196153412d5 (127.19.226.193) to 6e2837e083814e77b16db43b99d985ab (127.19.226.194). New cstate: current_term: 2 leader_uuid: "6e2837e083814e77b16db43b99d985ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 } health_report { overall_health: UNKNOWN } } }
I20250109 08:57:17.033298 21011 raft_consensus.cc:1270] T ec3e3f891e3c4a879077b9ee897e5d52 P b6684bfcd7404fe68d8cb196153412d5 [term 5 FOLLOWER]: Refusing update from remote peer 6e2837e083814e77b16db43b99d985ab: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20250109 08:57:17.034410 21315 consensus_queue.cc:1035] T ec3e3f891e3c4a879077b9ee897e5d52 P 6e2837e083814e77b16db43b99d985ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250109 08:57:17.043637 21158 raft_consensus.cc:1270] T ec3e3f891e3c4a879077b9ee897e5d52 P ead0b707c1014770b25c5a0db0454830 [term 5 FOLLOWER]: Refusing update from remote peer 6e2837e083814e77b16db43b99d985ab: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20250109 08:57:17.044961 21315 consensus_queue.cc:1035] T ec3e3f891e3c4a879077b9ee897e5d52 P 6e2837e083814e77b16db43b99d985ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250109 08:57:17.045818 21011 raft_consensus.cc:1270] T 502ebc25110a4c04a71b8b2e8eea07d4 P b6684bfcd7404fe68d8cb196153412d5 [term 3 FOLLOWER]: Refusing update from remote peer ead0b707c1014770b25c5a0db0454830: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20250109 08:57:17.047102 21328 consensus_queue.cc:1035] T 502ebc25110a4c04a71b8b2e8eea07d4 P ead0b707c1014770b25c5a0db0454830 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:57:17.056948 21085 raft_consensus.cc:1270] T 502ebc25110a4c04a71b8b2e8eea07d4 P 6e2837e083814e77b16db43b99d985ab [term 3 FOLLOWER]: Refusing update from remote peer ead0b707c1014770b25c5a0db0454830: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20250109 08:57:17.058589 21351 consensus_queue.cc:1035] T 502ebc25110a4c04a71b8b2e8eea07d4 P ead0b707c1014770b25c5a0db0454830 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:57:17.158694 21085 raft_consensus.cc:1270] T 166b00d1332d4f39b57ae9f58cb7fc5f P 6e2837e083814e77b16db43b99d985ab [term 4 FOLLOWER]: Refusing update from remote peer ead0b707c1014770b25c5a0db0454830: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20250109 08:57:17.159723 21351 consensus_queue.cc:1035] T 166b00d1332d4f39b57ae9f58cb7fc5f P ead0b707c1014770b25c5a0db0454830 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e2837e083814e77b16db43b99d985ab" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 44083 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250109 08:57:17.167287 21011 raft_consensus.cc:1270] T 166b00d1332d4f39b57ae9f58cb7fc5f P b6684bfcd7404fe68d8cb196153412d5 [term 4 FOLLOWER]: Refusing update from remote peer ead0b707c1014770b25c5a0db0454830: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20250109 08:57:17.169238 21351 consensus_queue.cc:1035] T 166b00d1332d4f39b57ae9f58cb7fc5f P ead0b707c1014770b25c5a0db0454830 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250109 08:57:17.318946 20363 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250109 08:57:17.324810 20363 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250109 08:57:17.325176 20363 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250109 08:57:17.329428 21158 raft_consensus.cc:1270] T 42af5f76a37645b5a3865739d54a6cb3 P ead0b707c1014770b25c5a0db0454830 [term 2 FOLLOWER]: Refusing update from remote peer 6e2837e083814e77b16db43b99d985ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:57:17.330471 21315 consensus_queue.cc:1035] T 42af5f76a37645b5a3865739d54a6cb3 P 6e2837e083814e77b16db43b99d985ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "ead0b707c1014770b25c5a0db0454830" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:57:17.339394 21011 raft_consensus.cc:1270] T 42af5f76a37645b5a3865739d54a6cb3 P b6684bfcd7404fe68d8cb196153412d5 [term 2 FOLLOWER]: Refusing update from remote peer 6e2837e083814e77b16db43b99d985ab: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:57:17.340662 21315 consensus_queue.cc:1035] T 42af5f76a37645b5a3865739d54a6cb3 P 6e2837e083814e77b16db43b99d985ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6684bfcd7404fe68d8cb196153412d5" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:57:18.326084 20363 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250109 08:57:18.332070 20363 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250109 08:57:18.332495 20363 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250109 08:57:19.333251 20363 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250109 08:57:19.338464 20363 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250109 08:57:19.338816 20363 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250109 08:57:20.339468 20363 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250109 08:57:20.344460 20363 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250109 08:57:20.344815 20363 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250109 08:57:21.345587 20363 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250109 08:57:21.351838 20363 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250109 08:57:21.352181 20363 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250109 08:57:22.352934 20363 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250109 08:57:22.358047 20363 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250109 08:57:22.358430 20363 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250109 08:57:23.359100 20363 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250109 08:57:23.364149 20363 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250109 08:57:23.364557 20363 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250109 08:57:24.365226 20363 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250109 08:57:24.370406 20363 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250109 08:57:24.370774 20363 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250109 08:57:25.371431 20363 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250109 08:57:25.376560 20363 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250109 08:57:25.376924 20363 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250109 08:57:26.403694 20363 auto_leader_rebalancer-test.cc:384] The leader distribution is
I20250109 08:57:26.404196 20363 tablet_server.cc:178] TabletServer@127.19.226.193:0 shutting down...
W20250109 08:57:26.420194 21125 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 5 similar messages]
W20250109 08:57:26.423651 21125 consensus_peers.cc:487] T 166b00d1332d4f39b57ae9f58cb7fc5f P ead0b707c1014770b25c5a0db0454830 -> Peer b6684bfcd7404fe68d8cb196153412d5 (127.19.226.193:34209): Couldn't send request to peer b6684bfcd7404fe68d8cb196153412d5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:57:26.429188 20363 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250109 08:57:26.429822 20363 tablet_replica.cc:331] T af13c7179cc24b258e516eceb2b0cac0 P b6684bfcd7404fe68d8cb196153412d5: stopping tablet replica
I20250109 08:57:26.430490 20363 raft_consensus.cc:2238] T af13c7179cc24b258e516eceb2b0cac0 P b6684bfcd7404fe68d8cb196153412d5 [term 3 LEADER]: Raft consensus shutting down.
I20250109 08:57:26.431361 20363 raft_consensus.cc:2267] T af13c7179cc24b258e516eceb2b0cac0 P b6684bfcd7404fe68d8cb196153412d5 [term 3 FOLLOWER]: Raft consensus is shut down!
I20250109 08:57:26.433221 20363 tablet_replica.cc:331] T 163310a12b44475ca3727662dad194a6 P b6684bfcd7404fe68d8cb196153412d5: stopping tablet replica
I20250109 08:57:26.433719 20363 raft_consensus.cc:2238] T 163310a12b44475ca3727662dad194a6 P b6684bfcd7404fe68d8cb196153412d5 [term 3 LEADER]: Raft consensus shutting down.
I20250109 08:57:26.434536 20363 raft_consensus.cc:2267] T 163310a12b44475ca3727662dad194a6 P b6684bfcd7404fe68d8cb196153412d5 [term 3 FOLLOWER]: Raft consensus is shut down!
I20250109 08:57:26.436154 20363 tablet_replica.cc:331] T 42af5f76a37645b5a3865739d54a6cb3 P b6684bfcd7404fe68d8cb196153412d5: stopping tablet replica
I20250109 08:57:26.436650 20363 raft_consensus.cc:2238] T 42af5f76a37645b5a3865739d54a6cb3 P b6684bfcd7404fe68d8cb196153412d5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:57:26.437040 20363 raft_consensus.cc:2267] T 42af5f76a37645b5a3865739d54a6cb3 P b6684bfcd7404fe68d8cb196153412d5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:57:26.438660 20363 tablet_replica.cc:331] T ec3e3f891e3c4a879077b9ee897e5d52 P b6684bfcd7404fe68d8cb196153412d5: stopping tablet replica
I20250109 08:57:26.439136 20363 raft_consensus.cc:2238] T ec3e3f891e3c4a879077b9ee897e5d52 P b6684bfcd7404fe68d8cb196153412d5 [term 5 FOLLOWER]: Raft consensus shutting down.
I20250109 08:57:26.439502 20363 raft_consensus.cc:2267] T ec3e3f891e3c4a879077b9ee897e5d52 P b6684bfcd7404fe68d8cb196153412d5 [term 5 FOLLOWER]: Raft consensus is shut down!
I20250109 08:57:26.441098 20363 tablet_replica.cc:331] T 53b47173fb6a4a358c8ad9c2df0f6f18 P b6684bfcd7404fe68d8cb196153412d5: stopping tablet replica
I20250109 08:57:26.441576 20363 raft_consensus.cc:2238] T 53b47173fb6a4a358c8ad9c2df0f6f18 P b6684bfcd7404fe68d8cb196153412d5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:57:26.441949 20363 raft_consensus.cc:2267] T 53b47173fb6a4a358c8ad9c2df0f6f18 P b6684bfcd7404fe68d8cb196153412d5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:57:26.443499 20363 tablet_replica.cc:331] T 166b00d1332d4f39b57ae9f58cb7fc5f P b6684bfcd7404fe68d8cb196153412d5: stopping tablet replica
I20250109 08:57:26.443934 20363 raft_consensus.cc:2238] T 166b00d1332d4f39b57ae9f58cb7fc5f P b6684bfcd7404fe68d8cb196153412d5 [term 4 FOLLOWER]: Raft consensus shutting down.
I20250109 08:57:26.444293 20363 raft_consensus.cc:2267] T 166b00d1332d4f39b57ae9f58cb7fc5f P b6684bfcd7404fe68d8cb196153412d5 [term 4 FOLLOWER]: Raft consensus is shut down!
I20250109 08:57:26.445875 20363 tablet_replica.cc:331] T cb177a6fcf4144e2ae1a560eb627e276 P b6684bfcd7404fe68d8cb196153412d5: stopping tablet replica
I20250109 08:57:26.446317 20363 raft_consensus.cc:2238] T cb177a6fcf4144e2ae1a560eb627e276 P b6684bfcd7404fe68d8cb196153412d5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:57:26.446656 20363 raft_consensus.cc:2267] T cb177a6fcf4144e2ae1a560eb627e276 P b6684bfcd7404fe68d8cb196153412d5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:57:26.448180 20363 tablet_replica.cc:331] T 05148030cdd443048fe5899b03fd92fe P b6684bfcd7404fe68d8cb196153412d5: stopping tablet replica
I20250109 08:57:26.448628 20363 raft_consensus.cc:2238] T 05148030cdd443048fe5899b03fd92fe P b6684bfcd7404fe68d8cb196153412d5 [term 3 LEADER]: Raft consensus shutting down.
I20250109 08:57:26.449358 20363 raft_consensus.cc:2267] T 05148030cdd443048fe5899b03fd92fe P b6684bfcd7404fe68d8cb196153412d5 [term 3 FOLLOWER]: Raft consensus is shut down!
I20250109 08:57:26.450964 20363 tablet_replica.cc:331] T 948e27b3034d44338e2b7d74f96374f7 P b6684bfcd7404fe68d8cb196153412d5: stopping tablet replica
I20250109 08:57:26.451386 20363 raft_consensus.cc:2238] T 948e27b3034d44338e2b7d74f96374f7 P b6684bfcd7404fe68d8cb196153412d5 [term 2 LEADER]: Raft consensus shutting down.
I20250109 08:57:26.452044 20363 raft_consensus.cc:2267] T 948e27b3034d44338e2b7d74f96374f7 P b6684bfcd7404fe68d8cb196153412d5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:57:26.453673 20363 tablet_replica.cc:331] T 502ebc25110a4c04a71b8b2e8eea07d4 P b6684bfcd7404fe68d8cb196153412d5: stopping tablet replica
I20250109 08:57:26.454080 20363 raft_consensus.cc:2238] T 502ebc25110a4c04a71b8b2e8eea07d4 P b6684bfcd7404fe68d8cb196153412d5 [term 3 FOLLOWER]: Raft consensus shutting down.
I20250109 08:57:26.454458 20363 raft_consensus.cc:2267] T 502ebc25110a4c04a71b8b2e8eea07d4 P b6684bfcd7404fe68d8cb196153412d5 [term 3 FOLLOWER]: Raft consensus is shut down!
I20250109 08:57:26.482247 20363 tablet_server.cc:195] TabletServer@127.19.226.193:0 shutdown complete.
W20250109 08:57:26.486716 21125 consensus_peers.cc:487] T cb177a6fcf4144e2ae1a560eb627e276 P ead0b707c1014770b25c5a0db0454830 -> Peer b6684bfcd7404fe68d8cb196153412d5 (127.19.226.193:34209): Couldn't send request to peer b6684bfcd7404fe68d8cb196153412d5. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.193:34209: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250109 08:57:26.502589 20363 tablet_server.cc:178] TabletServer@127.19.226.194:0 shutting down...
W20250109 08:57:26.505640 21051 consensus_peers.cc:487] T 42af5f76a37645b5a3865739d54a6cb3 P 6e2837e083814e77b16db43b99d985ab -> Peer b6684bfcd7404fe68d8cb196153412d5 (127.19.226.193:34209): Couldn't send request to peer b6684bfcd7404fe68d8cb196153412d5. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.193:34209: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250109 08:57:26.525435 20363 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250109 08:57:26.526178 20363 tablet_replica.cc:331] T 42af5f76a37645b5a3865739d54a6cb3 P 6e2837e083814e77b16db43b99d985ab: stopping tablet replica
I20250109 08:57:26.526973 20363 raft_consensus.cc:2238] T 42af5f76a37645b5a3865739d54a6cb3 P 6e2837e083814e77b16db43b99d985ab [term 2 LEADER]: Raft consensus shutting down.
I20250109 08:57:26.527716 20363 raft_consensus.cc:2267] T 42af5f76a37645b5a3865739d54a6cb3 P 6e2837e083814e77b16db43b99d985ab [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:57:26.529572 20363 tablet_replica.cc:331] T 163310a12b44475ca3727662dad194a6 P 6e2837e083814e77b16db43b99d985ab: stopping tablet replica
I20250109 08:57:26.530114 20363 raft_consensus.cc:2238] T 163310a12b44475ca3727662dad194a6 P 6e2837e083814e77b16db43b99d985ab [term 3 FOLLOWER]: Raft consensus shutting down.
I20250109 08:57:26.530618 20363 raft_consensus.cc:2267] T 163310a12b44475ca3727662dad194a6 P 6e2837e083814e77b16db43b99d985ab [term 3 FOLLOWER]: Raft consensus is shut down!
I20250109 08:57:26.532671 20363 tablet_replica.cc:331] T af13c7179cc24b258e516eceb2b0cac0 P 6e2837e083814e77b16db43b99d985ab: stopping tablet replica
I20250109 08:57:26.533408 20363 raft_consensus.cc:2238] T af13c7179cc24b258e516eceb2b0cac0 P 6e2837e083814e77b16db43b99d985ab [term 3 FOLLOWER]: Raft consensus shutting down.
I20250109 08:57:26.533941 20363 raft_consensus.cc:2267] T af13c7179cc24b258e516eceb2b0cac0 P 6e2837e083814e77b16db43b99d985ab [term 3 FOLLOWER]: Raft consensus is shut down!
I20250109 08:57:26.535696 20363 tablet_replica.cc:331] T 05148030cdd443048fe5899b03fd92fe P 6e2837e083814e77b16db43b99d985ab: stopping tablet replica
I20250109 08:57:26.536134 20363 raft_consensus.cc:2238] T 05148030cdd443048fe5899b03fd92fe P 6e2837e083814e77b16db43b99d985ab [term 3 FOLLOWER]: Raft consensus shutting down.
I20250109 08:57:26.536572 20363 raft_consensus.cc:2267] T 05148030cdd443048fe5899b03fd92fe P 6e2837e083814e77b16db43b99d985ab [term 3 FOLLOWER]: Raft consensus is shut down!
I20250109 08:57:26.538045 20363 tablet_replica.cc:331] T cb177a6fcf4144e2ae1a560eb627e276 P 6e2837e083814e77b16db43b99d985ab: stopping tablet replica
I20250109 08:57:26.538470 20363 raft_consensus.cc:2238] T cb177a6fcf4144e2ae1a560eb627e276 P 6e2837e083814e77b16db43b99d985ab [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:57:26.538782 20363 raft_consensus.cc:2267] T cb177a6fcf4144e2ae1a560eb627e276 P 6e2837e083814e77b16db43b99d985ab [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:57:26.540325 20363 tablet_replica.cc:331] T 948e27b3034d44338e2b7d74f96374f7 P 6e2837e083814e77b16db43b99d985ab: stopping tablet replica
I20250109 08:57:26.540761 20363 raft_consensus.cc:2238] T 948e27b3034d44338e2b7d74f96374f7 P 6e2837e083814e77b16db43b99d985ab [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:57:26.541118 20363 raft_consensus.cc:2267] T 948e27b3034d44338e2b7d74f96374f7 P 6e2837e083814e77b16db43b99d985ab [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:57:26.542583 20363 tablet_replica.cc:331] T 502ebc25110a4c04a71b8b2e8eea07d4 P 6e2837e083814e77b16db43b99d985ab: stopping tablet replica
I20250109 08:57:26.542989 20363 raft_consensus.cc:2238] T 502ebc25110a4c04a71b8b2e8eea07d4 P 6e2837e083814e77b16db43b99d985ab [term 3 FOLLOWER]: Raft consensus shutting down.
I20250109 08:57:26.543355 20363 raft_consensus.cc:2267] T 502ebc25110a4c04a71b8b2e8eea07d4 P 6e2837e083814e77b16db43b99d985ab [term 3 FOLLOWER]: Raft consensus is shut down!
I20250109 08:57:26.545027 20363 tablet_replica.cc:331] T 166b00d1332d4f39b57ae9f58cb7fc5f P 6e2837e083814e77b16db43b99d985ab: stopping tablet replica
I20250109 08:57:26.545449 20363 raft_consensus.cc:2238] T 166b00d1332d4f39b57ae9f58cb7fc5f P 6e2837e083814e77b16db43b99d985ab [term 4 FOLLOWER]: Raft consensus shutting down.
I20250109 08:57:26.545800 20363 raft_consensus.cc:2267] T 166b00d1332d4f39b57ae9f58cb7fc5f P 6e2837e083814e77b16db43b99d985ab [term 4 FOLLOWER]: Raft consensus is shut down!
I20250109 08:57:26.547272 20363 tablet_replica.cc:331] T 53b47173fb6a4a358c8ad9c2df0f6f18 P 6e2837e083814e77b16db43b99d985ab: stopping tablet replica
I20250109 08:57:26.547639 20363 raft_consensus.cc:2238] T 53b47173fb6a4a358c8ad9c2df0f6f18 P 6e2837e083814e77b16db43b99d985ab [term 2 LEADER]: Raft consensus shutting down.
I20250109 08:57:26.548306 20363 raft_consensus.cc:2267] T 53b47173fb6a4a358c8ad9c2df0f6f18 P 6e2837e083814e77b16db43b99d985ab [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:57:26.549810 20363 tablet_replica.cc:331] T ec3e3f891e3c4a879077b9ee897e5d52 P 6e2837e083814e77b16db43b99d985ab: stopping tablet replica
I20250109 08:57:26.550181 20363 raft_consensus.cc:2238] T ec3e3f891e3c4a879077b9ee897e5d52 P 6e2837e083814e77b16db43b99d985ab [term 5 LEADER]: Raft consensus shutting down.
I20250109 08:57:26.550874 20363 raft_consensus.cc:2267] T ec3e3f891e3c4a879077b9ee897e5d52 P 6e2837e083814e77b16db43b99d985ab [term 5 FOLLOWER]: Raft consensus is shut down!
W20250109 08:57:26.573125 21123 consensus_peers.cc:487] T 502ebc25110a4c04a71b8b2e8eea07d4 P ead0b707c1014770b25c5a0db0454830 -> Peer 6e2837e083814e77b16db43b99d985ab (127.19.226.194:44083): Couldn't send request to peer 6e2837e083814e77b16db43b99d985ab. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.194:44083: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250109 08:57:26.578213 20363 tablet_server.cc:195] TabletServer@127.19.226.194:0 shutdown complete.
I20250109 08:57:26.597111 20363 tablet_server.cc:178] TabletServer@127.19.226.195:0 shutting down...
I20250109 08:57:26.616545 20363 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250109 08:57:26.617180 20363 tablet_replica.cc:331] T 502ebc25110a4c04a71b8b2e8eea07d4 P ead0b707c1014770b25c5a0db0454830: stopping tablet replica
I20250109 08:57:26.617951 20363 raft_consensus.cc:2238] T 502ebc25110a4c04a71b8b2e8eea07d4 P ead0b707c1014770b25c5a0db0454830 [term 3 LEADER]: Raft consensus shutting down.
I20250109 08:57:26.618914 20363 raft_consensus.cc:2267] T 502ebc25110a4c04a71b8b2e8eea07d4 P ead0b707c1014770b25c5a0db0454830 [term 3 FOLLOWER]: Raft consensus is shut down!
I20250109 08:57:26.620584 20363 tablet_replica.cc:331] T 163310a12b44475ca3727662dad194a6 P ead0b707c1014770b25c5a0db0454830: stopping tablet replica
I20250109 08:57:26.621055 20363 raft_consensus.cc:2238] T 163310a12b44475ca3727662dad194a6 P ead0b707c1014770b25c5a0db0454830 [term 3 FOLLOWER]: Raft consensus shutting down.
I20250109 08:57:26.621407 20363 raft_consensus.cc:2267] T 163310a12b44475ca3727662dad194a6 P ead0b707c1014770b25c5a0db0454830 [term 3 FOLLOWER]: Raft consensus is shut down!
I20250109 08:57:26.622999 20363 tablet_replica.cc:331] T ec3e3f891e3c4a879077b9ee897e5d52 P ead0b707c1014770b25c5a0db0454830: stopping tablet replica
I20250109 08:57:26.623426 20363 raft_consensus.cc:2238] T ec3e3f891e3c4a879077b9ee897e5d52 P ead0b707c1014770b25c5a0db0454830 [term 5 FOLLOWER]: Raft consensus shutting down.
I20250109 08:57:26.623770 20363 raft_consensus.cc:2267] T ec3e3f891e3c4a879077b9ee897e5d52 P ead0b707c1014770b25c5a0db0454830 [term 5 FOLLOWER]: Raft consensus is shut down!
I20250109 08:57:26.625345 20363 tablet_replica.cc:331] T 948e27b3034d44338e2b7d74f96374f7 P ead0b707c1014770b25c5a0db0454830: stopping tablet replica
I20250109 08:57:26.625808 20363 raft_consensus.cc:2238] T 948e27b3034d44338e2b7d74f96374f7 P ead0b707c1014770b25c5a0db0454830 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:57:26.626168 20363 raft_consensus.cc:2267] T 948e27b3034d44338e2b7d74f96374f7 P ead0b707c1014770b25c5a0db0454830 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:57:26.627640 20363 tablet_replica.cc:331] T cb177a6fcf4144e2ae1a560eb627e276 P ead0b707c1014770b25c5a0db0454830: stopping tablet replica
I20250109 08:57:26.628047 20363 raft_consensus.cc:2238] T cb177a6fcf4144e2ae1a560eb627e276 P ead0b707c1014770b25c5a0db0454830 [term 1 LEADER]: Raft consensus shutting down.
I20250109 08:57:26.628769 20363 raft_consensus.cc:2267] T cb177a6fcf4144e2ae1a560eb627e276 P ead0b707c1014770b25c5a0db0454830 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:57:26.630324 20363 tablet_replica.cc:331] T 05148030cdd443048fe5899b03fd92fe P ead0b707c1014770b25c5a0db0454830: stopping tablet replica
I20250109 08:57:26.630733 20363 raft_consensus.cc:2238] T 05148030cdd443048fe5899b03fd92fe P ead0b707c1014770b25c5a0db0454830 [term 3 FOLLOWER]: Raft consensus shutting down.
I20250109 08:57:26.631107 20363 raft_consensus.cc:2267] T 05148030cdd443048fe5899b03fd92fe P ead0b707c1014770b25c5a0db0454830 [term 3 FOLLOWER]: Raft consensus is shut down!
I20250109 08:57:26.632551 20363 tablet_replica.cc:331] T 166b00d1332d4f39b57ae9f58cb7fc5f P ead0b707c1014770b25c5a0db0454830: stopping tablet replica
I20250109 08:57:26.632956 20363 raft_consensus.cc:2238] T 166b00d1332d4f39b57ae9f58cb7fc5f P ead0b707c1014770b25c5a0db0454830 [term 4 LEADER]: Raft consensus shutting down.
I20250109 08:57:26.633675 20363 raft_consensus.cc:2267] T 166b00d1332d4f39b57ae9f58cb7fc5f P ead0b707c1014770b25c5a0db0454830 [term 4 FOLLOWER]: Raft consensus is shut down!
I20250109 08:57:26.635183 20363 tablet_replica.cc:331] T 53b47173fb6a4a358c8ad9c2df0f6f18 P ead0b707c1014770b25c5a0db0454830: stopping tablet replica
I20250109 08:57:26.635565 20363 raft_consensus.cc:2238] T 53b47173fb6a4a358c8ad9c2df0f6f18 P ead0b707c1014770b25c5a0db0454830 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:57:26.635933 20363 raft_consensus.cc:2267] T 53b47173fb6a4a358c8ad9c2df0f6f18 P ead0b707c1014770b25c5a0db0454830 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:57:26.637470 20363 tablet_replica.cc:331] T af13c7179cc24b258e516eceb2b0cac0 P ead0b707c1014770b25c5a0db0454830: stopping tablet replica
I20250109 08:57:26.637861 20363 raft_consensus.cc:2238] T af13c7179cc24b258e516eceb2b0cac0 P ead0b707c1014770b25c5a0db0454830 [term 3 FOLLOWER]: Raft consensus shutting down.
I20250109 08:57:26.638190 20363 raft_consensus.cc:2267] T af13c7179cc24b258e516eceb2b0cac0 P ead0b707c1014770b25c5a0db0454830 [term 3 FOLLOWER]: Raft consensus is shut down!
I20250109 08:57:26.639588 20363 tablet_replica.cc:331] T 42af5f76a37645b5a3865739d54a6cb3 P ead0b707c1014770b25c5a0db0454830: stopping tablet replica
I20250109 08:57:26.639956 20363 raft_consensus.cc:2238] T 42af5f76a37645b5a3865739d54a6cb3 P ead0b707c1014770b25c5a0db0454830 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:57:26.640280 20363 raft_consensus.cc:2267] T 42af5f76a37645b5a3865739d54a6cb3 P ead0b707c1014770b25c5a0db0454830 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:57:26.666054 20363 tablet_server.cc:195] TabletServer@127.19.226.195:0 shutdown complete.
I20250109 08:57:26.684191 20363 master.cc:537] Master@127.19.226.254:37551 shutting down...
I20250109 08:57:26.703050 20363 raft_consensus.cc:2238] T 00000000000000000000000000000000 P 8c3697782dab47b383365dc945f94c68 [term 1 LEADER]: Raft consensus shutting down.
I20250109 08:57:26.703569 20363 raft_consensus.cc:2267] T 00000000000000000000000000000000 P 8c3697782dab47b383365dc945f94c68 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:57:26.703850 20363 tablet_replica.cc:331] T 00000000000000000000000000000000 P 8c3697782dab47b383365dc945f94c68: stopping tablet replica
I20250109 08:57:26.721894 20363 master.cc:559] Master@127.19.226.254:37551 shutdown complete.
6e2837e083814e77b16db43b99d985ab 4
b6684bfcd7404fe68d8cb196153412d5 5
ead0b707c1014770b25c5a0db0454830 1
6e2837e083814e77b16db43b99d985ab 4
b6684bfcd7404fe68d8cb196153412d5 3
ead0b707c1014770b25c5a0db0454830 3
6e2837e083814e77b16db43b99d985ab 1
b6684bfcd7404fe68d8cb196153412d5 8
ead0b707c1014770b25c5a0db0454830 1
6e2837e083814e77b16db43b99d985ab 3
b6684bfcd7404fe68d8cb196153412d5 4
ead0b707c1014770b25c5a0db0454830 3
[ OK ] LeaderRebalancerTest.FunctionalTestForNotDivided (28430 ms)
[ RUN ] LeaderRebalancerTest.AddTserver
I20250109 08:57:26.758309 20363 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.19.226.254:38499
I20250109 08:57:26.759362 20363 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250109 08:57:26.764899 21387 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250109 08:57:26.765446 21388 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250109 08:57:26.766022 21390 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250109 08:57:26.766393 20363 server_base.cc:1034] running on GCE node
I20250109 08:57:26.767232 20363 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250109 08:57:26.767422 20363 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250109 08:57:26.767551 20363 hybrid_clock.cc:648] HybridClock initialized: now 1736413046767535 us; error 0 us; skew 500 ppm
I20250109 08:57:26.768065 20363 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250109 08:57:26.774390 20363 webserver.cc:458] Webserver started at http://127.19.226.254:32997/ using document root <none> and password file <none>
I20250109 08:57:26.774880 20363 fs_manager.cc:362] Metadata directory not provided
I20250109 08:57:26.775044 20363 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250109 08:57:26.775297 20363 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250109 08:57:26.776721 20363 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736412987538469-20363-0/minicluster-data/master-0-root/instance:
uuid: "1f1a09d9b3e04cfcb8db842978c0cf7b"
format_stamp: "Formatted at 2025-01-09 08:57:26 on dist-test-slave-g8ds"
I20250109 08:57:26.781020 20363 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20250109 08:57:26.784112 21395 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250109 08:57:26.784824 20363 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20250109 08:57:26.785084 20363 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736412987538469-20363-0/minicluster-data/master-0-root
uuid: "1f1a09d9b3e04cfcb8db842978c0cf7b"
format_stamp: "Formatted at 2025-01-09 08:57:26 on dist-test-slave-g8ds"
I20250109 08:57:26.785337 20363 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736412987538469-20363-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736412987538469-20363-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736412987538469-20363-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250109 08:57:26.796725 20363 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250109 08:57:26.797838 20363 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250109 08:57:26.833577 20363 rpc_server.cc:307] RPC server started. Bound to: 127.19.226.254:38499
I20250109 08:57:26.833693 21446 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.226.254:38499 every 8 connection(s)
I20250109 08:57:26.837285 21447 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:26.846760 21447 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f1a09d9b3e04cfcb8db842978c0cf7b: Bootstrap starting.
I20250109 08:57:26.851192 21447 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1f1a09d9b3e04cfcb8db842978c0cf7b: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:26.855223 21447 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f1a09d9b3e04cfcb8db842978c0cf7b: No bootstrap required, opened a new log
I20250109 08:57:26.857247 21447 raft_consensus.cc:357] T 00000000000000000000000000000000 P 1f1a09d9b3e04cfcb8db842978c0cf7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f1a09d9b3e04cfcb8db842978c0cf7b" member_type: VOTER }
I20250109 08:57:26.857697 21447 raft_consensus.cc:383] T 00000000000000000000000000000000 P 1f1a09d9b3e04cfcb8db842978c0cf7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:26.857934 21447 raft_consensus.cc:738] T 00000000000000000000000000000000 P 1f1a09d9b3e04cfcb8db842978c0cf7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f1a09d9b3e04cfcb8db842978c0cf7b, State: Initialized, Role: FOLLOWER
I20250109 08:57:26.858465 21447 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1f1a09d9b3e04cfcb8db842978c0cf7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f1a09d9b3e04cfcb8db842978c0cf7b" member_type: VOTER }
I20250109 08:57:26.858894 21447 raft_consensus.cc:397] T 00000000000000000000000000000000 P 1f1a09d9b3e04cfcb8db842978c0cf7b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250109 08:57:26.859123 21447 raft_consensus.cc:491] T 00000000000000000000000000000000 P 1f1a09d9b3e04cfcb8db842978c0cf7b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250109 08:57:26.859369 21447 raft_consensus.cc:3054] T 00000000000000000000000000000000 P 1f1a09d9b3e04cfcb8db842978c0cf7b [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:26.863301 21447 raft_consensus.cc:513] T 00000000000000000000000000000000 P 1f1a09d9b3e04cfcb8db842978c0cf7b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f1a09d9b3e04cfcb8db842978c0cf7b" member_type: VOTER }
I20250109 08:57:26.863790 21447 leader_election.cc:304] T 00000000000000000000000000000000 P 1f1a09d9b3e04cfcb8db842978c0cf7b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1f1a09d9b3e04cfcb8db842978c0cf7b; no voters:
I20250109 08:57:26.864883 21447 leader_election.cc:290] T 00000000000000000000000000000000 P 1f1a09d9b3e04cfcb8db842978c0cf7b [CANDIDATE]: Term 1 election: Requested vote from peers
I20250109 08:57:26.865244 21450 raft_consensus.cc:2798] T 00000000000000000000000000000000 P 1f1a09d9b3e04cfcb8db842978c0cf7b [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:57:26.866459 21450 raft_consensus.cc:695] T 00000000000000000000000000000000 P 1f1a09d9b3e04cfcb8db842978c0cf7b [term 1 LEADER]: Becoming Leader. State: Replica: 1f1a09d9b3e04cfcb8db842978c0cf7b, State: Running, Role: LEADER
I20250109 08:57:26.867069 21450 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1f1a09d9b3e04cfcb8db842978c0cf7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f1a09d9b3e04cfcb8db842978c0cf7b" member_type: VOTER }
I20250109 08:57:26.867728 21447 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1f1a09d9b3e04cfcb8db842978c0cf7b [sys.catalog]: configured and running, proceeding with master startup.
I20250109 08:57:26.871276 21451 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f1a09d9b3e04cfcb8db842978c0cf7b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1f1a09d9b3e04cfcb8db842978c0cf7b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f1a09d9b3e04cfcb8db842978c0cf7b" member_type: VOTER } }
I20250109 08:57:26.871464 21452 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f1a09d9b3e04cfcb8db842978c0cf7b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1f1a09d9b3e04cfcb8db842978c0cf7b. Latest consensus state: current_term: 1 leader_uuid: "1f1a09d9b3e04cfcb8db842978c0cf7b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f1a09d9b3e04cfcb8db842978c0cf7b" member_type: VOTER } }
I20250109 08:57:26.871851 21451 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f1a09d9b3e04cfcb8db842978c0cf7b [sys.catalog]: This master's current role is: LEADER
I20250109 08:57:26.872046 21452 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f1a09d9b3e04cfcb8db842978c0cf7b [sys.catalog]: This master's current role is: LEADER
I20250109 08:57:26.874883 21456 catalog_manager.cc:1476] Loading table and tablet metadata into memory...
I20250109 08:57:26.881805 21456 catalog_manager.cc:1485] Initializing Kudu cluster ID...
I20250109 08:57:26.882784 20363 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20250109 08:57:26.889909 21456 catalog_manager.cc:1348] Generated new cluster ID: cabb9f6053b34578b643ca5e46fd4dcf
I20250109 08:57:26.890184 21456 catalog_manager.cc:1496] Initializing Kudu internal certificate authority...
I20250109 08:57:26.906543 21456 catalog_manager.cc:1371] Generated new certificate authority record
I20250109 08:57:26.907801 21456 catalog_manager.cc:1505] Loading token signing keys...
I20250109 08:57:26.927066 21456 catalog_manager.cc:5899] T 00000000000000000000000000000000 P 1f1a09d9b3e04cfcb8db842978c0cf7b: Generated new TSK 0
I20250109 08:57:26.927675 21456 catalog_manager.cc:1515] Initializing in-progress tserver states...
I20250109 08:57:26.949430 20363 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250109 08:57:26.955077 21468 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250109 08:57:26.956166 21469 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250109 08:57:26.957281 21471 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250109 08:57:26.958199 20363 server_base.cc:1034] running on GCE node
I20250109 08:57:26.958988 20363 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250109 08:57:26.959160 20363 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250109 08:57:26.959275 20363 hybrid_clock.cc:648] HybridClock initialized: now 1736413046959265 us; error 0 us; skew 500 ppm
I20250109 08:57:26.959702 20363 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250109 08:57:26.961880 20363 webserver.cc:458] Webserver started at http://127.19.226.193:33749/ using document root <none> and password file <none>
I20250109 08:57:26.962301 20363 fs_manager.cc:362] Metadata directory not provided
I20250109 08:57:26.962443 20363 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250109 08:57:26.962643 20363 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250109 08:57:26.963613 20363 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736412987538469-20363-0/minicluster-data/ts-0-root/instance:
uuid: "8bf6e26e98b14566899578a990b27d47"
format_stamp: "Formatted at 2025-01-09 08:57:26 on dist-test-slave-g8ds"
I20250109 08:57:26.967905 20363 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20250109 08:57:26.971028 21476 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250109 08:57:26.971663 20363 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20250109 08:57:26.971925 20363 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736412987538469-20363-0/minicluster-data/ts-0-root
uuid: "8bf6e26e98b14566899578a990b27d47"
format_stamp: "Formatted at 2025-01-09 08:57:26 on dist-test-slave-g8ds"
I20250109 08:57:26.972172 20363 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736412987538469-20363-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736412987538469-20363-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736412987538469-20363-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250109 08:57:26.982663 20363 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250109 08:57:26.983794 20363 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250109 08:57:26.985167 20363 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250109 08:57:26.987351 20363 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250109 08:57:26.987537 20363 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250109 08:57:26.987761 20363 ts_tablet_manager.cc:610] Registered 0 tablets
I20250109 08:57:26.987915 20363 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250109 08:57:27.024757 20363 rpc_server.cc:307] RPC server started. Bound to: 127.19.226.193:42725
I20250109 08:57:27.024842 21538 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.226.193:42725 every 8 connection(s)
I20250109 08:57:27.029072 20363 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250109 08:57:27.036011 21543 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250109 08:57:27.038630 21544 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250109 08:57:27.040264 20363 server_base.cc:1034] running on GCE node
W20250109 08:57:27.040714 21546 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250109 08:57:27.041496 21539 heartbeater.cc:346] Connected to a master server at 127.19.226.254:38499
I20250109 08:57:27.041761 20363 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20250109 08:57:27.041873 21539 heartbeater.cc:463] Registering TS with master...
W20250109 08:57:27.042037 20363 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250109 08:57:27.042239 20363 hybrid_clock.cc:648] HybridClock initialized: now 1736413047042222 us; error 0 us; skew 500 ppm
I20250109 08:57:27.042735 21539 heartbeater.cc:510] Master 127.19.226.254:38499 requested a full tablet report, sending...
I20250109 08:57:27.042944 20363 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250109 08:57:27.044934 21412 ts_manager.cc:194] Registered new tserver with Master: 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725)
I20250109 08:57:27.045512 20363 webserver.cc:458] Webserver started at http://127.19.226.194:44449/ using document root <none> and password file <none>
I20250109 08:57:27.045980 20363 fs_manager.cc:362] Metadata directory not provided
I20250109 08:57:27.046164 20363 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250109 08:57:27.046388 20363 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250109 08:57:27.046759 21412 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:33018
I20250109 08:57:27.047538 20363 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736412987538469-20363-0/minicluster-data/ts-1-root/instance:
uuid: "a2b1a692435241ee988e0bd1661cd4c3"
format_stamp: "Formatted at 2025-01-09 08:57:27 on dist-test-slave-g8ds"
I20250109 08:57:27.052062 20363 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.001s sys 0.005s
I20250109 08:57:27.055205 21551 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250109 08:57:27.055996 20363 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20250109 08:57:27.056268 20363 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736412987538469-20363-0/minicluster-data/ts-1-root
uuid: "a2b1a692435241ee988e0bd1661cd4c3"
format_stamp: "Formatted at 2025-01-09 08:57:27 on dist-test-slave-g8ds"
I20250109 08:57:27.056578 20363 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736412987538469-20363-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736412987538469-20363-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736412987538469-20363-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250109 08:57:27.066393 20363 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250109 08:57:27.067529 20363 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250109 08:57:27.068998 20363 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250109 08:57:27.071069 20363 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250109 08:57:27.071255 20363 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250109 08:57:27.071465 20363 ts_tablet_manager.cc:610] Registered 0 tablets
I20250109 08:57:27.071607 20363 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250109 08:57:27.108683 20363 rpc_server.cc:307] RPC server started. Bound to: 127.19.226.194:45527
I20250109 08:57:27.108786 21613 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.226.194:45527 every 8 connection(s)
I20250109 08:57:27.112985 20363 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250109 08:57:27.120813 21617 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250109 08:57:27.122812 21618 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250109 08:57:27.124323 21620 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250109 08:57:27.125425 21614 heartbeater.cc:346] Connected to a master server at 127.19.226.254:38499
I20250109 08:57:27.125813 21614 heartbeater.cc:463] Registering TS with master...
I20250109 08:57:27.126473 20363 server_base.cc:1034] running on GCE node
I20250109 08:57:27.126639 21614 heartbeater.cc:510] Master 127.19.226.254:38499 requested a full tablet report, sending...
I20250109 08:57:27.127249 20363 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250109 08:57:27.127486 20363 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250109 08:57:27.127682 20363 hybrid_clock.cc:648] HybridClock initialized: now 1736413047127663 us; error 0 us; skew 500 ppm
I20250109 08:57:27.128235 20363 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250109 08:57:27.128551 21412 ts_manager.cc:194] Registered new tserver with Master: a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
I20250109 08:57:27.130046 21412 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:33024
I20250109 08:57:27.130712 20363 webserver.cc:458] Webserver started at http://127.19.226.195:45159/ using document root <none> and password file <none>
I20250109 08:57:27.131268 20363 fs_manager.cc:362] Metadata directory not provided
I20250109 08:57:27.131443 20363 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250109 08:57:27.131666 20363 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250109 08:57:27.132771 20363 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736412987538469-20363-0/minicluster-data/ts-2-root/instance:
uuid: "787ddcc824c2490a808fc5a53452dc5e"
format_stamp: "Formatted at 2025-01-09 08:57:27 on dist-test-slave-g8ds"
I20250109 08:57:27.136744 20363 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20250109 08:57:27.139564 21625 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250109 08:57:27.140223 20363 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20250109 08:57:27.140498 20363 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736412987538469-20363-0/minicluster-data/ts-2-root
uuid: "787ddcc824c2490a808fc5a53452dc5e"
format_stamp: "Formatted at 2025-01-09 08:57:27 on dist-test-slave-g8ds"
I20250109 08:57:27.140769 20363 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736412987538469-20363-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736412987538469-20363-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736412987538469-20363-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250109 08:57:27.159222 20363 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250109 08:57:27.160297 20363 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250109 08:57:27.161630 20363 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250109 08:57:27.163698 20363 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250109 08:57:27.163897 20363 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250109 08:57:27.164111 20363 ts_tablet_manager.cc:610] Registered 0 tablets
I20250109 08:57:27.164258 20363 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250109 08:57:27.201858 20363 rpc_server.cc:307] RPC server started. Bound to: 127.19.226.195:35731
I20250109 08:57:27.201946 21687 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.226.195:35731 every 8 connection(s)
I20250109 08:57:27.213902 21688 heartbeater.cc:346] Connected to a master server at 127.19.226.254:38499
I20250109 08:57:27.214217 21688 heartbeater.cc:463] Registering TS with master...
I20250109 08:57:27.214874 21688 heartbeater.cc:510] Master 127.19.226.254:38499 requested a full tablet report, sending...
I20250109 08:57:27.216756 21412 ts_manager.cc:194] Registered new tserver with Master: 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:27.216912 20363 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.012218687s
I20250109 08:57:27.218457 21412 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:33026
I20250109 08:57:27.219941 20363 test_util.cc:274] Using random seed: 719120076
I20250109 08:57:27.246683 21412 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33042:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250109 08:57:27.253803 21412 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250109 08:57:27.440656 21576 tablet_service.cc:1467] Processing CreateTablet for tablet 37280454805d47f98a415a8f0c16d869 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20250109 08:57:27.441304 21574 tablet_service.cc:1467] Processing CreateTablet for tablet 7a7211b8d4374ad1afd82e8cf85bad81 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20250109 08:57:27.442150 21576 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37280454805d47f98a415a8f0c16d869. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:27.442147 21579 tablet_service.cc:1467] Processing CreateTablet for tablet 206785f29c824cddbe9bbc00d365ffb3 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION VALUES < 36398027
I20250109 08:57:27.443235 21578 tablet_service.cc:1467] Processing CreateTablet for tablet 20ea30dba4314290ae45d9b4c13bebdd (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20250109 08:57:27.443464 21573 tablet_service.cc:1467] Processing CreateTablet for tablet e601a034e10e418bac8683443d86a1c1 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20250109 08:57:27.444571 21578 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20ea30dba4314290ae45d9b4c13bebdd. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:27.445140 21572 tablet_service.cc:1467] Processing CreateTablet for tablet 80123146836349938344efc61333695f (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20250109 08:57:27.447149 21570 tablet_service.cc:1467] Processing CreateTablet for tablet 610441ef4b1d40c284eff3a714e4214e (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20250109 08:57:27.448168 21575 tablet_service.cc:1467] Processing CreateTablet for tablet 97304b06c7fa40ef8345a72c796c47f8 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20250109 08:57:27.448827 21572 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80123146836349938344efc61333695f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:27.449440 21579 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 206785f29c824cddbe9bbc00d365ffb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:27.450309 21570 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 610441ef4b1d40c284eff3a714e4214e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:27.447001 21577 tablet_service.cc:1467] Processing CreateTablet for tablet 79c1ed8ab551478291f1f24cc940cb27 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20250109 08:57:27.451589 21577 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 79c1ed8ab551478291f1f24cc940cb27. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:27.452979 21573 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e601a034e10e418bac8683443d86a1c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:27.447039 21571 tablet_service.cc:1467] Processing CreateTablet for tablet db34a00c02b84ee988edc7d6e344c1f5 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20250109 08:57:27.454857 21571 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db34a00c02b84ee988edc7d6e344c1f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:27.456698 21575 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97304b06c7fa40ef8345a72c796c47f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:27.460690 21574 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a7211b8d4374ad1afd82e8cf85bad81. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:27.479346 21579 tablet_service.cc:1467] Processing CreateTablet for tablet 363fa693a5114f2bb22f4ab67722a1c6 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20250109 08:57:27.480631 21579 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 363fa693a5114f2bb22f4ab67722a1c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:27.486507 21571 tablet_service.cc:1467] Processing CreateTablet for tablet c761cb691bd241ae825917a3fbc3ad24 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20250109 08:57:27.488061 21571 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c761cb691bd241ae825917a3fbc3ad24. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:27.500355 21653 tablet_service.cc:1467] Processing CreateTablet for tablet 206785f29c824cddbe9bbc00d365ffb3 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION VALUES < 36398027
I20250109 08:57:27.501669 21653 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 206785f29c824cddbe9bbc00d365ffb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:27.504998 21652 tablet_service.cc:1467] Processing CreateTablet for tablet 20ea30dba4314290ae45d9b4c13bebdd (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20250109 08:57:27.506276 21652 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20ea30dba4314290ae45d9b4c13bebdd. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:27.510015 21651 tablet_service.cc:1467] Processing CreateTablet for tablet 79c1ed8ab551478291f1f24cc940cb27 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20250109 08:57:27.511313 21651 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 79c1ed8ab551478291f1f24cc940cb27. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:27.513897 21650 tablet_service.cc:1467] Processing CreateTablet for tablet 37280454805d47f98a415a8f0c16d869 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20250109 08:57:27.515204 21650 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37280454805d47f98a415a8f0c16d869. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:27.515029 21648 tablet_service.cc:1467] Processing CreateTablet for tablet 7a7211b8d4374ad1afd82e8cf85bad81 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20250109 08:57:27.516412 21648 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a7211b8d4374ad1afd82e8cf85bad81. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:27.520980 21649 tablet_service.cc:1467] Processing CreateTablet for tablet 97304b06c7fa40ef8345a72c796c47f8 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20250109 08:57:27.522403 21649 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97304b06c7fa40ef8345a72c796c47f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:27.525477 21647 tablet_service.cc:1467] Processing CreateTablet for tablet e601a034e10e418bac8683443d86a1c1 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20250109 08:57:27.525838 21646 tablet_service.cc:1467] Processing CreateTablet for tablet 80123146836349938344efc61333695f (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20250109 08:57:27.526896 21647 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e601a034e10e418bac8683443d86a1c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:27.529477 21578 tablet_service.cc:1467] Processing CreateTablet for tablet 306c4892bbff4dbfbe5a0645294aa627 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20250109 08:57:27.530125 21646 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80123146836349938344efc61333695f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:27.537871 21645 tablet_service.cc:1467] Processing CreateTablet for tablet db34a00c02b84ee988edc7d6e344c1f5 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20250109 08:57:27.539291 21578 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 306c4892bbff4dbfbe5a0645294aa627. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:27.541637 21644 tablet_service.cc:1467] Processing CreateTablet for tablet 610441ef4b1d40c284eff3a714e4214e (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20250109 08:57:27.540441 21574 tablet_service.cc:1467] Processing CreateTablet for tablet 71ac0243d72845659291184835652dde (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20250109 08:57:27.543089 21574 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 71ac0243d72845659291184835652dde. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:27.539244 21645 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db34a00c02b84ee988edc7d6e344c1f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:27.554842 21570 tablet_service.cc:1467] Processing CreateTablet for tablet 0e06021cd6624e50af2c914399ebd6c4 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20250109 08:57:27.557979 21644 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 610441ef4b1d40c284eff3a714e4214e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:27.557029 21570 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e06021cd6624e50af2c914399ebd6c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:27.565358 21576 tablet_service.cc:1467] Processing CreateTablet for tablet d8493108cba04514bd2236192cef1079 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20250109 08:57:27.575445 21504 tablet_service.cc:1467] Processing CreateTablet for tablet 206785f29c824cddbe9bbc00d365ffb3 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION VALUES < 36398027
I20250109 08:57:27.576805 21504 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 206785f29c824cddbe9bbc00d365ffb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:27.576624 21574 tablet_service.cc:1467] Processing CreateTablet for tablet 5ac5ffe3cd65463fb448e819e3978841 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20250109 08:57:27.577461 21503 tablet_service.cc:1467] Processing CreateTablet for tablet 20ea30dba4314290ae45d9b4c13bebdd (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20250109 08:57:27.578709 21503 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20ea30dba4314290ae45d9b4c13bebdd. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:27.581550 21502 tablet_service.cc:1467] Processing CreateTablet for tablet 79c1ed8ab551478291f1f24cc940cb27 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20250109 08:57:27.582180 21575 tablet_service.cc:1467] Processing CreateTablet for tablet 180b4d539a144a38905bf340ecb24ef4 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20250109 08:57:27.582809 21502 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 79c1ed8ab551478291f1f24cc940cb27. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:27.585469 21500 tablet_service.cc:1467] Processing CreateTablet for tablet 97304b06c7fa40ef8345a72c796c47f8 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20250109 08:57:27.586756 21500 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97304b06c7fa40ef8345a72c796c47f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:27.587325 21702 tablet_bootstrap.cc:492] T 37280454805d47f98a415a8f0c16d869 P a2b1a692435241ee988e0bd1661cd4c3: Bootstrap starting.
I20250109 08:57:27.589489 21499 tablet_service.cc:1467] Processing CreateTablet for tablet 7a7211b8d4374ad1afd82e8cf85bad81 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20250109 08:57:27.590736 21499 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a7211b8d4374ad1afd82e8cf85bad81. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:27.593644 21576 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8493108cba04514bd2236192cef1079. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:27.593469 21501 tablet_service.cc:1467] Processing CreateTablet for tablet 37280454805d47f98a415a8f0c16d869 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20250109 08:57:27.594725 21501 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37280454805d47f98a415a8f0c16d869. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:27.596829 21498 tablet_service.cc:1467] Processing CreateTablet for tablet e601a034e10e418bac8683443d86a1c1 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20250109 08:57:27.598207 21498 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e601a034e10e418bac8683443d86a1c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:27.601899 21497 tablet_service.cc:1467] Processing CreateTablet for tablet 80123146836349938344efc61333695f (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20250109 08:57:27.603193 21497 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80123146836349938344efc61333695f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:27.605913 21496 tablet_service.cc:1467] Processing CreateTablet for tablet db34a00c02b84ee988edc7d6e344c1f5 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20250109 08:57:27.606168 21577 tablet_service.cc:1467] Processing CreateTablet for tablet 3b5049c00b0b4dd89c49fa5d13db169e (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20250109 08:57:27.607223 21496 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db34a00c02b84ee988edc7d6e344c1f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:27.607359 21577 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b5049c00b0b4dd89c49fa5d13db169e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:27.608552 21495 tablet_service.cc:1467] Processing CreateTablet for tablet 610441ef4b1d40c284eff3a714e4214e (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20250109 08:57:27.610812 21572 tablet_service.cc:1467] Processing CreateTablet for tablet 89e1eeb24fea483eb63bd98af66e0b6b (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20250109 08:57:27.611759 21495 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 610441ef4b1d40c284eff3a714e4214e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:27.612015 21572 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 89e1eeb24fea483eb63bd98af66e0b6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:27.617410 21579 tablet_service.cc:1467] Processing CreateTablet for tablet d6f988a231a74f6fa112da9a527d4a0b (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20250109 08:57:27.624419 21652 tablet_service.cc:1467] Processing CreateTablet for tablet 363fa693a5114f2bb22f4ab67722a1c6 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20250109 08:57:27.625898 21652 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 363fa693a5114f2bb22f4ab67722a1c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:27.628506 21571 tablet_service.cc:1467] Processing CreateTablet for tablet 81d74e7d4472497aac5463a4b47e4fb9 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20250109 08:57:27.629711 21571 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81d74e7d4472497aac5463a4b47e4fb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:27.635097 21574 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ac5ffe3cd65463fb448e819e3978841. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:27.637603 21646 tablet_service.cc:1467] Processing CreateTablet for tablet c761cb691bd241ae825917a3fbc3ad24 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20250109 08:57:27.639151 21646 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c761cb691bd241ae825917a3fbc3ad24. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:27.646538 21495 tablet_service.cc:1467] Processing CreateTablet for tablet 363fa693a5114f2bb22f4ab67722a1c6 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20250109 08:57:27.640990 21579 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6f988a231a74f6fa112da9a527d4a0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:27.657125 21495 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 363fa693a5114f2bb22f4ab67722a1c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:27.649477 21644 tablet_service.cc:1467] Processing CreateTablet for tablet 81d74e7d4472497aac5463a4b47e4fb9 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20250109 08:57:27.659461 21644 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81d74e7d4472497aac5463a4b47e4fb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:27.660650 21575 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 180b4d539a144a38905bf340ecb24ef4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:27.663004 21504 tablet_service.cc:1467] Processing CreateTablet for tablet 81d74e7d4472497aac5463a4b47e4fb9 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20250109 08:57:27.664247 21504 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81d74e7d4472497aac5463a4b47e4fb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:27.667811 21573 tablet_service.cc:1467] Processing CreateTablet for tablet e38ec0ef84af4b798073eafd4d210c35 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20250109 08:57:27.693045 21651 tablet_service.cc:1467] Processing CreateTablet for tablet d8493108cba04514bd2236192cef1079 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20250109 08:57:27.655222 21496 tablet_service.cc:1467] Processing CreateTablet for tablet c761cb691bd241ae825917a3fbc3ad24 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20250109 08:57:27.753504 21496 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c761cb691bd241ae825917a3fbc3ad24. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:27.753178 21504 tablet_service.cc:1467] Processing CreateTablet for tablet d6f988a231a74f6fa112da9a527d4a0b (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20250109 08:57:27.754954 21578 tablet_service.cc:1467] Processing CreateTablet for tablet af273bab41cc4a9da780147eebfd52ec (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20250109 08:57:27.755553 21504 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6f988a231a74f6fa112da9a527d4a0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:27.755468 21574 tablet_service.cc:1467] Processing CreateTablet for tablet e8ff570914d74df8b87296010e9b864f (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20250109 08:57:27.651582 21648 tablet_service.cc:1467] Processing CreateTablet for tablet 71ac0243d72845659291184835652dde (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20250109 08:57:27.687592 21576 tablet_service.cc:1467] Processing CreateTablet for tablet d355d166de534fb4a50c9f48f2df7469 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20250109 08:57:27.757120 21648 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 71ac0243d72845659291184835652dde. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:27.688347 21503 tablet_service.cc:1467] Processing CreateTablet for tablet 71ac0243d72845659291184835652dde (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20250109 08:57:27.758020 21503 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 71ac0243d72845659291184835652dde. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:27.711887 21650 tablet_service.cc:1467] Processing CreateTablet for tablet 180b4d539a144a38905bf340ecb24ef4 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20250109 08:57:27.759240 21650 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 180b4d539a144a38905bf340ecb24ef4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:27.689853 21646 tablet_service.cc:1467] Processing CreateTablet for tablet 5ac5ffe3cd65463fb448e819e3978841 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20250109 08:57:27.691679 21579 tablet_service.cc:1467] Processing CreateTablet for tablet 90a4a2381cbe40f0b204be325ee9fea1 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20250109 08:57:27.763909 21646 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ac5ffe3cd65463fb448e819e3978841. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:27.701627 21499 tablet_service.cc:1467] Processing CreateTablet for tablet 0e06021cd6624e50af2c914399ebd6c4 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20250109 08:57:27.764958 21499 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e06021cd6624e50af2c914399ebd6c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:27.765470 21651 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8493108cba04514bd2236192cef1079. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:27.653664 21703 tablet_bootstrap.cc:492] T 206785f29c824cddbe9bbc00d365ffb3 P 8bf6e26e98b14566899578a990b27d47: Bootstrap starting.
I20250109 08:57:27.708503 21500 tablet_service.cc:1467] Processing CreateTablet for tablet 5ac5ffe3cd65463fb448e819e3978841 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20250109 08:57:27.773116 21500 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ac5ffe3cd65463fb448e819e3978841. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:27.706858 21502 tablet_service.cc:1467] Processing CreateTablet for tablet d8493108cba04514bd2236192cef1079 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20250109 08:57:27.776086 21644 tablet_service.cc:1467] Processing CreateTablet for tablet e38ec0ef84af4b798073eafd4d210c35 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20250109 08:57:27.777073 21502 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8493108cba04514bd2236192cef1079. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:27.777365 21644 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e38ec0ef84af4b798073eafd4d210c35. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:27.694882 21702 tablet_bootstrap.cc:654] T 37280454805d47f98a415a8f0c16d869 P a2b1a692435241ee988e0bd1661cd4c3: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:27.725175 21649 tablet_service.cc:1467] Processing CreateTablet for tablet 0e06021cd6624e50af2c914399ebd6c4 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20250109 08:57:27.726199 21573 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e38ec0ef84af4b798073eafd4d210c35. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:27.729168 21495 tablet_service.cc:1467] Processing CreateTablet for tablet 180b4d539a144a38905bf340ecb24ef4 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20250109 08:57:27.733458 21572 tablet_service.cc:1467] Processing CreateTablet for tablet b54d6381a2cc41dfb59b1f6b99c0c5d4 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20250109 08:57:27.735177 21497 tablet_service.cc:1467] Processing CreateTablet for tablet 3b5049c00b0b4dd89c49fa5d13db169e (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20250109 08:57:27.737717 21498 tablet_service.cc:1467] Processing CreateTablet for tablet 89e1eeb24fea483eb63bd98af66e0b6b (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20250109 08:57:27.727907 21645 tablet_service.cc:1467] Processing CreateTablet for tablet 3b5049c00b0b4dd89c49fa5d13db169e (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20250109 08:57:27.698976 21501 tablet_service.cc:1467] Processing CreateTablet for tablet 306c4892bbff4dbfbe5a0645294aa627 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20250109 08:57:27.739344 21652 tablet_service.cc:1467] Processing CreateTablet for tablet 89e1eeb24fea483eb63bd98af66e0b6b (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20250109 08:57:27.745596 21653 tablet_service.cc:1467] Processing CreateTablet for tablet d6f988a231a74f6fa112da9a527d4a0b (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20250109 08:57:27.750000 21571 tablet_service.cc:1467] Processing CreateTablet for tablet 75edb5d04fa14201ad711c1d95c81f64 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20250109 08:57:27.750648 21577 tablet_service.cc:1467] Processing CreateTablet for tablet 0cc4fcda876242c396b804edebebb01f (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20250109 08:57:27.697242 21570 tablet_service.cc:1467] Processing CreateTablet for tablet ddd14cdbefd64936960e9ffb5b38b5f2 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20250109 08:57:27.683182 21704 tablet_bootstrap.cc:492] T e601a034e10e418bac8683443d86a1c1 P 787ddcc824c2490a808fc5a53452dc5e: Bootstrap starting.
I20250109 08:57:27.680070 21647 tablet_service.cc:1467] Processing CreateTablet for tablet 306c4892bbff4dbfbe5a0645294aa627 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20250109 08:57:27.784422 21495 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 180b4d539a144a38905bf340ecb24ef4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:27.792424 21498 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 89e1eeb24fea483eb63bd98af66e0b6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:27.793010 21501 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 306c4892bbff4dbfbe5a0645294aa627. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:27.793927 21497 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b5049c00b0b4dd89c49fa5d13db169e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:27.783478 21576 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d355d166de534fb4a50c9f48f2df7469. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:27.801087 21579 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90a4a2381cbe40f0b204be325ee9fea1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:27.804162 21578 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af273bab41cc4a9da780147eebfd52ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:27.812669 21572 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b54d6381a2cc41dfb59b1f6b99c0c5d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:27.813534 21570 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ddd14cdbefd64936960e9ffb5b38b5f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:27.815325 21703 tablet_bootstrap.cc:654] T 206785f29c824cddbe9bbc00d365ffb3 P 8bf6e26e98b14566899578a990b27d47: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:27.780426 21496 tablet_service.cc:1467] Processing CreateTablet for tablet e38ec0ef84af4b798073eafd4d210c35 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20250109 08:57:27.817791 21496 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e38ec0ef84af4b798073eafd4d210c35. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:27.783577 21649 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e06021cd6624e50af2c914399ebd6c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:27.818675 21571 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75edb5d04fa14201ad711c1d95c81f64. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:27.821329 21645 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b5049c00b0b4dd89c49fa5d13db169e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:27.823599 21652 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 89e1eeb24fea483eb63bd98af66e0b6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:27.825742 21574 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8ff570914d74df8b87296010e9b864f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:27.829103 21577 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0cc4fcda876242c396b804edebebb01f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:27.832702 21647 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 306c4892bbff4dbfbe5a0645294aa627. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:27.836709 21653 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6f988a231a74f6fa112da9a527d4a0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:27.844321 21704 tablet_bootstrap.cc:654] T e601a034e10e418bac8683443d86a1c1 P 787ddcc824c2490a808fc5a53452dc5e: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:27.854283 21650 tablet_service.cc:1467] Processing CreateTablet for tablet ddd14cdbefd64936960e9ffb5b38b5f2 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20250109 08:57:27.855580 21650 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ddd14cdbefd64936960e9ffb5b38b5f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:27.860196 21497 tablet_service.cc:1467] Processing CreateTablet for tablet d355d166de534fb4a50c9f48f2df7469 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20250109 08:57:27.861536 21497 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d355d166de534fb4a50c9f48f2df7469. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:27.867508 21646 tablet_service.cc:1467] Processing CreateTablet for tablet d355d166de534fb4a50c9f48f2df7469 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20250109 08:57:27.868849 21646 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d355d166de534fb4a50c9f48f2df7469. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:27.873237 21573 tablet_service.cc:1467] Processing CreateTablet for tablet df9715570a6844d9a2686307db16eff1 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20250109 08:57:27.874615 21573 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df9715570a6844d9a2686307db16eff1. 1 dirs total, 0 dirs full, 0 dirs failed
W20250109 08:57:27.878746 21460 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20250109 08:57:27.881002 21495 tablet_service.cc:1467] Processing CreateTablet for tablet b54d6381a2cc41dfb59b1f6b99c0c5d4 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20250109 08:57:27.907876 21495 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b54d6381a2cc41dfb59b1f6b99c0c5d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:27.917326 21645 tablet_service.cc:1467] Processing CreateTablet for tablet b54d6381a2cc41dfb59b1f6b99c0c5d4 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20250109 08:57:27.917829 21651 tablet_service.cc:1467] Processing CreateTablet for tablet 75edb5d04fa14201ad711c1d95c81f64 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20250109 08:57:27.922058 21651 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75edb5d04fa14201ad711c1d95c81f64. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:27.923599 21504 tablet_service.cc:1467] Processing CreateTablet for tablet af273bab41cc4a9da780147eebfd52ec (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20250109 08:57:27.924064 21652 tablet_service.cc:1467] Processing CreateTablet for tablet 0cc4fcda876242c396b804edebebb01f (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20250109 08:57:27.924927 21504 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af273bab41cc4a9da780147eebfd52ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:27.925278 21652 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0cc4fcda876242c396b804edebebb01f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:27.926625 21704 tablet_bootstrap.cc:492] T e601a034e10e418bac8683443d86a1c1 P 787ddcc824c2490a808fc5a53452dc5e: No bootstrap required, opened a new log
I20250109 08:57:27.927105 21704 ts_tablet_manager.cc:1397] T e601a034e10e418bac8683443d86a1c1 P 787ddcc824c2490a808fc5a53452dc5e: Time spent bootstrapping tablet: real 0.244s user 0.005s sys 0.011s
I20250109 08:57:27.937975 21574 tablet_service.cc:1467] Processing CreateTablet for tablet b00671c87aac405a81c6543f4defd722 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20250109 08:57:27.939312 21574 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b00671c87aac405a81c6543f4defd722. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:27.940842 21496 tablet_service.cc:1467] Processing CreateTablet for tablet e8ff570914d74df8b87296010e9b864f (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20250109 08:57:27.951519 21496 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8ff570914d74df8b87296010e9b864f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:27.902092 21500 tablet_service.cc:1467] Processing CreateTablet for tablet 75edb5d04fa14201ad711c1d95c81f64 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20250109 08:57:27.955734 21499 tablet_service.cc:1467] Processing CreateTablet for tablet 0cc4fcda876242c396b804edebebb01f (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20250109 08:57:27.789611 21575 tablet_service.cc:1467] Processing CreateTablet for tablet 0f3bde1a1ccd4c359a2e80feca968453 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20250109 08:57:27.905665 21648 tablet_service.cc:1467] Processing CreateTablet for tablet 90a4a2381cbe40f0b204be325ee9fea1 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20250109 08:57:27.906616 21501 tablet_service.cc:1467] Processing CreateTablet for tablet ddd14cdbefd64936960e9ffb5b38b5f2 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20250109 08:57:27.890465 21702 tablet_bootstrap.cc:492] T 37280454805d47f98a415a8f0c16d869 P a2b1a692435241ee988e0bd1661cd4c3: No bootstrap required, opened a new log
I20250109 08:57:27.958185 21702 ts_tablet_manager.cc:1397] T 37280454805d47f98a415a8f0c16d869 P a2b1a692435241ee988e0bd1661cd4c3: Time spent bootstrapping tablet: real 0.371s user 0.013s sys 0.015s
I20250109 08:57:27.912801 21579 tablet_service.cc:1467] Processing CreateTablet for tablet e03846e461974d0d8944c344e715b86a (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20250109 08:57:27.961071 21579 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e03846e461974d0d8944c344e715b86a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:27.961257 21644 tablet_service.cc:1467] Processing CreateTablet for tablet af273bab41cc4a9da780147eebfd52ec (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20250109 08:57:27.965288 21702 raft_consensus.cc:357] T 37280454805d47f98a415a8f0c16d869 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:27.966593 21702 raft_consensus.cc:383] T 37280454805d47f98a415a8f0c16d869 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:27.967706 21702 raft_consensus.cc:738] T 37280454805d47f98a415a8f0c16d869 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2b1a692435241ee988e0bd1661cd4c3, State: Initialized, Role: FOLLOWER
I20250109 08:57:27.969492 21702 consensus_queue.cc:260] T 37280454805d47f98a415a8f0c16d869 P a2b1a692435241ee988e0bd1661cd4c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:27.940898 21576 tablet_service.cc:1467] Processing CreateTablet for tablet af330dcd4767423ea4c179be7f88d0c4 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20250109 08:57:27.891180 21571 tablet_service.cc:1467] Processing CreateTablet for tablet 239ca74a4c0a455a9db2652d7debb989 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20250109 08:57:27.942185 21645 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b54d6381a2cc41dfb59b1f6b99c0c5d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:27.943708 21570 tablet_service.cc:1467] Processing CreateTablet for tablet 90978f185a374feba3bc19b54f9d97f5 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20250109 08:57:27.944279 21578 tablet_service.cc:1467] Processing CreateTablet for tablet c18c1d46a6ca47598d7b161ed70a4e7a (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20250109 08:57:27.974110 21648 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90a4a2381cbe40f0b204be325ee9fea1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:27.974917 21575 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f3bde1a1ccd4c359a2e80feca968453. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:27.948263 21502 tablet_service.cc:1467] Processing CreateTablet for tablet 0f3bde1a1ccd4c359a2e80feca968453 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20250109 08:57:27.950179 21577 tablet_service.cc:1467] Processing CreateTablet for tablet c99aacce09d44f889be5480854a22760 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20250109 08:57:27.879865 21503 tablet_service.cc:1467] Processing CreateTablet for tablet 90a4a2381cbe40f0b204be325ee9fea1 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20250109 08:57:27.976790 21570 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90978f185a374feba3bc19b54f9d97f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:27.977355 21577 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c99aacce09d44f889be5480854a22760. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:27.978071 21571 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 239ca74a4c0a455a9db2652d7debb989. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:27.979913 21578 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c18c1d46a6ca47598d7b161ed70a4e7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:27.984886 21576 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af330dcd4767423ea4c179be7f88d0c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:27.989008 21647 tablet_service.cc:1467] Processing CreateTablet for tablet 0f3bde1a1ccd4c359a2e80feca968453 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20250109 08:57:27.990307 21647 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f3bde1a1ccd4c359a2e80feca968453. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:27.994589 21649 tablet_service.cc:1467] Processing CreateTablet for tablet 239ca74a4c0a455a9db2652d7debb989 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20250109 08:57:27.995786 21649 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 239ca74a4c0a455a9db2652d7debb989. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:27.953832 21704 raft_consensus.cc:357] T e601a034e10e418bac8683443d86a1c1 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:27.954975 21703 tablet_bootstrap.cc:492] T 206785f29c824cddbe9bbc00d365ffb3 P 8bf6e26e98b14566899578a990b27d47: No bootstrap required, opened a new log
I20250109 08:57:27.970785 21650 tablet_service.cc:1467] Processing CreateTablet for tablet df9715570a6844d9a2686307db16eff1 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20250109 08:57:27.973306 21501 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ddd14cdbefd64936960e9ffb5b38b5f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:27.997321 21703 ts_tablet_manager.cc:1397] T 206785f29c824cddbe9bbc00d365ffb3 P 8bf6e26e98b14566899578a990b27d47: Time spent bootstrapping tablet: real 0.344s user 0.005s sys 0.009s
I20250109 08:57:27.999675 21646 tablet_service.cc:1467] Processing CreateTablet for tablet e03846e461974d0d8944c344e715b86a (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20250109 08:57:28.000483 21500 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75edb5d04fa14201ad711c1d95c81f64. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:27.997144 21704 raft_consensus.cc:383] T e601a034e10e418bac8683443d86a1c1 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:28.001130 21704 raft_consensus.cc:738] T e601a034e10e418bac8683443d86a1c1 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Initialized, Role: FOLLOWER
I20250109 08:57:28.001974 21704 consensus_queue.cc:260] T e601a034e10e418bac8683443d86a1c1 P 787ddcc824c2490a808fc5a53452dc5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:28.003432 21503 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90a4a2381cbe40f0b204be325ee9fea1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:28.004633 21502 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f3bde1a1ccd4c359a2e80feca968453. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:28.008657 21499 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0cc4fcda876242c396b804edebebb01f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:28.000088 21703 raft_consensus.cc:357] T 206785f29c824cddbe9bbc00d365ffb3 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } }
I20250109 08:57:28.014089 21703 raft_consensus.cc:383] T 206785f29c824cddbe9bbc00d365ffb3 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:28.014395 21703 raft_consensus.cc:738] T 206785f29c824cddbe9bbc00d365ffb3 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bf6e26e98b14566899578a990b27d47, State: Initialized, Role: FOLLOWER
I20250109 08:57:28.015089 21703 consensus_queue.cc:260] T 206785f29c824cddbe9bbc00d365ffb3 P 8bf6e26e98b14566899578a990b27d47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } }
I20250109 08:57:27.996351 21652 tablet_service.cc:1467] Processing CreateTablet for tablet af330dcd4767423ea4c179be7f88d0c4 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20250109 08:57:28.019153 21570 tablet_service.cc:1467] Processing CreateTablet for tablet 8c0459d2eee64f0a83bd05ad5974f506 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20250109 08:57:28.020439 21570 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8c0459d2eee64f0a83bd05ad5974f506. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:27.969666 21572 tablet_service.cc:1467] Processing CreateTablet for tablet 126a163a89c448be89ccb9490e9f0168 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20250109 08:57:28.003826 21573 tablet_service.cc:1467] Processing CreateTablet for tablet eb4516b9a16041c2adf34be520768f3f (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20250109 08:57:27.965536 21653 tablet_service.cc:1467] Processing CreateTablet for tablet e8ff570914d74df8b87296010e9b864f (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20250109 08:57:28.027969 21703 ts_tablet_manager.cc:1428] T 206785f29c824cddbe9bbc00d365ffb3 P 8bf6e26e98b14566899578a990b27d47: Time spent starting tablet: real 0.030s user 0.003s sys 0.004s
I20250109 08:57:28.028791 21539 heartbeater.cc:502] Master 127.19.226.254:38499 was elected leader, sending a full tablet report...
I20250109 08:57:28.029094 21703 tablet_bootstrap.cc:492] T 610441ef4b1d40c284eff3a714e4214e P 8bf6e26e98b14566899578a990b27d47: Bootstrap starting.
I20250109 08:57:28.036936 21614 heartbeater.cc:502] Master 127.19.226.254:38499 was elected leader, sending a full tablet report...
I20250109 08:57:28.034412 21644 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af273bab41cc4a9da780147eebfd52ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:28.040921 21650 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df9715570a6844d9a2686307db16eff1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:28.042879 21646 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e03846e461974d0d8944c344e715b86a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:28.044535 21652 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af330dcd4767423ea4c179be7f88d0c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:28.046824 21688 heartbeater.cc:502] Master 127.19.226.254:38499 was elected leader, sending a full tablet report...
I20250109 08:57:28.047544 21702 ts_tablet_manager.cc:1428] T 37280454805d47f98a415a8f0c16d869 P a2b1a692435241ee988e0bd1661cd4c3: Time spent starting tablet: real 0.089s user 0.006s sys 0.007s
I20250109 08:57:28.049865 21704 ts_tablet_manager.cc:1428] T e601a034e10e418bac8683443d86a1c1 P 787ddcc824c2490a808fc5a53452dc5e: Time spent starting tablet: real 0.122s user 0.008s sys 0.007s
I20250109 08:57:28.051955 21653 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8ff570914d74df8b87296010e9b864f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:28.056339 21703 tablet_bootstrap.cc:654] T 610441ef4b1d40c284eff3a714e4214e P 8bf6e26e98b14566899578a990b27d47: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:28.060397 21651 tablet_service.cc:1467] Processing CreateTablet for tablet b00671c87aac405a81c6543f4defd722 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20250109 08:57:27.991851 21498 tablet_service.cc:1467] Processing CreateTablet for tablet 239ca74a4c0a455a9db2652d7debb989 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20250109 08:57:28.083899 21497 tablet_service.cc:1467] Processing CreateTablet for tablet df9715570a6844d9a2686307db16eff1 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20250109 08:57:28.087944 21498 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 239ca74a4c0a455a9db2652d7debb989. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:28.088980 21497 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df9715570a6844d9a2686307db16eff1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:28.095330 21495 tablet_service.cc:1467] Processing CreateTablet for tablet e03846e461974d0d8944c344e715b86a (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20250109 08:57:28.096971 21495 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e03846e461974d0d8944c344e715b86a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:28.022013 21501 tablet_service.cc:1467] Processing CreateTablet for tablet af330dcd4767423ea4c179be7f88d0c4 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20250109 08:57:28.100260 21501 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af330dcd4767423ea4c179be7f88d0c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:28.102895 21499 tablet_service.cc:1467] Processing CreateTablet for tablet c18c1d46a6ca47598d7b161ed70a4e7a (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20250109 08:57:28.104303 21499 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c18c1d46a6ca47598d7b161ed70a4e7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:28.111397 21497 tablet_service.cc:1467] Processing CreateTablet for tablet 126a163a89c448be89ccb9490e9f0168 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20250109 08:57:28.113126 21497 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 126a163a89c448be89ccb9490e9f0168. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:28.121971 21499 tablet_service.cc:1467] Processing CreateTablet for tablet 90978f185a374feba3bc19b54f9d97f5 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20250109 08:57:28.123458 21499 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90978f185a374feba3bc19b54f9d97f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:28.129945 21497 tablet_service.cc:1467] Processing CreateTablet for tablet c99aacce09d44f889be5480854a22760 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20250109 08:57:28.131220 21497 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c99aacce09d44f889be5480854a22760. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:28.139912 21499 tablet_service.cc:1467] Processing CreateTablet for tablet eb4516b9a16041c2adf34be520768f3f (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20250109 08:57:28.141705 21499 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb4516b9a16041c2adf34be520768f3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:28.149950 21504 tablet_service.cc:1467] Processing CreateTablet for tablet 8c0459d2eee64f0a83bd05ad5974f506 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20250109 08:57:28.151212 21504 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8c0459d2eee64f0a83bd05ad5974f506. 1 dirs total, 0 dirs full, 0 dirs failed
W20250109 08:57:28.164824 21540 tablet.cc:2367] T 206785f29c824cddbe9bbc00d365ffb3 P 8bf6e26e98b14566899578a990b27d47: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20250109 08:57:28.164952 21500 tablet_service.cc:1467] Processing CreateTablet for tablet b00671c87aac405a81c6543f4defd722 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20250109 08:57:28.166755 21500 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b00671c87aac405a81c6543f4defd722. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:28.172055 21497 tablet_service.cc:1467] Processing CreateTablet for tablet f961d3f357944118a69f905621731552 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20250109 08:57:28.173476 21502 tablet_service.cc:1467] Processing CreateTablet for tablet ab6c3fcdf226433c9e1303004f6c96b4 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20250109 08:57:28.177285 21572 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 126a163a89c448be89ccb9490e9f0168. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:28.182402 21495 tablet_service.cc:1467] Processing CreateTablet for tablet a3aa271f2f0144eea9261e0e07ab833b (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20250109 08:57:28.178972 21496 tablet_service.cc:1467] Processing CreateTablet for tablet d015a10f95eb4f19b6d7c53cb201fa80 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20250109 08:57:28.183646 21573 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb4516b9a16041c2adf34be520768f3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:28.180584 21497 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f961d3f357944118a69f905621731552. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:28.186674 21496 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d015a10f95eb4f19b6d7c53cb201fa80. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:28.190485 21504 tablet_service.cc:1467] Processing CreateTablet for tablet f5f85ad7c7c94acfaf9d550d55fce48c (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20250109 08:57:28.191727 21504 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5f85ad7c7c94acfaf9d550d55fce48c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:28.193163 21502 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab6c3fcdf226433c9e1303004f6c96b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:28.197576 21495 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3aa271f2f0144eea9261e0e07ab833b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:28.170276 21501 tablet_service.cc:1467] Processing CreateTablet for tablet 7781c00473024307987096b596da12ac (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20250109 08:57:28.201436 21501 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7781c00473024307987096b596da12ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:28.182195 21498 tablet_service.cc:1467] Processing CreateTablet for tablet 1b582ab8d8f044d9b9038a28e958feeb (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20250109 08:57:28.209252 21498 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b582ab8d8f044d9b9038a28e958feeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:28.210296 21579 tablet_service.cc:1467] Processing CreateTablet for tablet ab6c3fcdf226433c9e1303004f6c96b4 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20250109 08:57:28.221273 21579 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab6c3fcdf226433c9e1303004f6c96b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:28.221755 21577 tablet_service.cc:1467] Processing CreateTablet for tablet a3aa271f2f0144eea9261e0e07ab833b (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20250109 08:57:28.222916 21577 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3aa271f2f0144eea9261e0e07ab833b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:28.212450 21574 tablet_service.cc:1467] Processing CreateTablet for tablet f961d3f357944118a69f905621731552 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20250109 08:57:28.225271 21574 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f961d3f357944118a69f905621731552. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:28.225891 21578 tablet_service.cc:1467] Processing CreateTablet for tablet 1b582ab8d8f044d9b9038a28e958feeb (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20250109 08:57:28.226218 21496 tablet_service.cc:1467] Processing CreateTablet for tablet d52f79ecf037457cbcc09d5fa43456cf (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20250109 08:57:28.227130 21578 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b582ab8d8f044d9b9038a28e958feeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:28.227859 21497 tablet_service.cc:1467] Processing CreateTablet for tablet 1d3d5d2a2c88417a97c9d57262a356dd (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20250109 08:57:28.216033 21504 tablet_service.cc:1467] Processing CreateTablet for tablet 56bbc306275e4a409e028a7ca1dd1602 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20250109 08:57:28.219800 21576 tablet_service.cc:1467] Processing CreateTablet for tablet 1d3d5d2a2c88417a97c9d57262a356dd (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20250109 08:57:28.230734 21576 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d3d5d2a2c88417a97c9d57262a356dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:28.214684 21571 tablet_service.cc:1467] Processing CreateTablet for tablet 7781c00473024307987096b596da12ac (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20250109 08:57:28.233156 21571 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7781c00473024307987096b596da12ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:28.236804 21504 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56bbc306275e4a409e028a7ca1dd1602. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:28.247696 21575 tablet_service.cc:1467] Processing CreateTablet for tablet d015a10f95eb4f19b6d7c53cb201fa80 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20250109 08:57:28.248732 21496 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d52f79ecf037457cbcc09d5fa43456cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:28.249017 21575 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d015a10f95eb4f19b6d7c53cb201fa80. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:28.257357 21495 tablet_service.cc:1467] Processing CreateTablet for tablet 5717a02ef53a4b22a96f431ca6c20d93 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20250109 08:57:28.258775 21495 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5717a02ef53a4b22a96f431ca6c20d93. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:28.260701 21497 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d3d5d2a2c88417a97c9d57262a356dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:28.214718 21499 tablet_service.cc:1467] Processing CreateTablet for tablet 64c667f887ea41b4a484ee565b21fa16 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20250109 08:57:28.265739 21499 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 64c667f887ea41b4a484ee565b21fa16. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:28.271824 21504 tablet_service.cc:1467] Processing CreateTablet for tablet 5d31dbfb2b4a4a6ca4500f92df9dbb98 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20250109 08:57:28.289260 21504 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d31dbfb2b4a4a6ca4500f92df9dbb98. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:28.290625 21645 tablet_service.cc:1467] Processing CreateTablet for tablet c18c1d46a6ca47598d7b161ed70a4e7a (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20250109 08:57:28.293140 21502 tablet_service.cc:1467] Processing CreateTablet for tablet 2935d69446cd458e92046d2b2f1b74ab (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20250109 08:57:28.315959 21502 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2935d69446cd458e92046d2b2f1b74ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:28.305538 21500 tablet_service.cc:1467] Processing CreateTablet for tablet edc9db96e0ff4aa5839a6922f3eb5f24 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20250109 08:57:28.321233 21500 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet edc9db96e0ff4aa5839a6922f3eb5f24. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:28.313841 21496 tablet_service.cc:1467] Processing CreateTablet for tablet 7c19ed9fdde5463f84e43bea7b860d03 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20250109 08:57:28.314648 21579 tablet_service.cc:1467] Processing CreateTablet for tablet 64c667f887ea41b4a484ee565b21fa16 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20250109 08:57:28.323695 21496 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c19ed9fdde5463f84e43bea7b860d03. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:28.297535 21574 tablet_service.cc:1467] Processing CreateTablet for tablet f5f85ad7c7c94acfaf9d550d55fce48c (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20250109 08:57:28.323834 21579 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 64c667f887ea41b4a484ee565b21fa16. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:28.326857 21574 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5f85ad7c7c94acfaf9d550d55fce48c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:28.329037 21645 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c18c1d46a6ca47598d7b161ed70a4e7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:28.334213 21647 tablet_service.cc:1467] Processing CreateTablet for tablet 90978f185a374feba3bc19b54f9d97f5 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20250109 08:57:28.335562 21647 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90978f185a374feba3bc19b54f9d97f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:28.338775 21648 tablet_service.cc:1467] Processing CreateTablet for tablet c99aacce09d44f889be5480854a22760 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20250109 08:57:28.340065 21648 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c99aacce09d44f889be5480854a22760. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:28.357601 21644 tablet_service.cc:1467] Processing CreateTablet for tablet eb4516b9a16041c2adf34be520768f3f (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20250109 08:57:28.358850 21644 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb4516b9a16041c2adf34be520768f3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:28.362258 21702 tablet_bootstrap.cc:492] T 206785f29c824cddbe9bbc00d365ffb3 P a2b1a692435241ee988e0bd1661cd4c3: Bootstrap starting.
I20250109 08:57:28.368641 21651 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b00671c87aac405a81c6543f4defd722. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:28.368798 21710 raft_consensus.cc:491] T e601a034e10e418bac8683443d86a1c1 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:28.369534 21710 raft_consensus.cc:513] T e601a034e10e418bac8683443d86a1c1 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:28.370723 21704 tablet_bootstrap.cc:492] T 20ea30dba4314290ae45d9b4c13bebdd P 787ddcc824c2490a808fc5a53452dc5e: Bootstrap starting.
I20250109 08:57:28.372413 21709 raft_consensus.cc:491] T 37280454805d47f98a415a8f0c16d869 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:28.372957 21710 leader_election.cc:290] T e601a034e10e418bac8683443d86a1c1 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
I20250109 08:57:28.372895 21709 raft_consensus.cc:513] T 37280454805d47f98a415a8f0c16d869 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:28.375048 21702 tablet_bootstrap.cc:654] T 206785f29c824cddbe9bbc00d365ffb3 P a2b1a692435241ee988e0bd1661cd4c3: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:28.375481 21709 leader_election.cc:290] T 37280454805d47f98a415a8f0c16d869 P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:28.377295 21703 tablet_bootstrap.cc:492] T 610441ef4b1d40c284eff3a714e4214e P 8bf6e26e98b14566899578a990b27d47: No bootstrap required, opened a new log
I20250109 08:57:28.377918 21703 ts_tablet_manager.cc:1397] T 610441ef4b1d40c284eff3a714e4214e P 8bf6e26e98b14566899578a990b27d47: Time spent bootstrapping tablet: real 0.349s user 0.004s sys 0.008s
I20250109 08:57:28.381297 21703 raft_consensus.cc:357] T 610441ef4b1d40c284eff3a714e4214e P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:28.387194 21703 raft_consensus.cc:383] T 610441ef4b1d40c284eff3a714e4214e P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:28.387972 21711 raft_consensus.cc:491] T 206785f29c824cddbe9bbc00d365ffb3 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:28.388494 21711 raft_consensus.cc:513] T 206785f29c824cddbe9bbc00d365ffb3 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } }
I20250109 08:57:28.386390 21704 tablet_bootstrap.cc:654] T 20ea30dba4314290ae45d9b4c13bebdd P 787ddcc824c2490a808fc5a53452dc5e: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:28.390998 21711 leader_election.cc:290] T 206785f29c824cddbe9bbc00d365ffb3 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:28.413007 21646 tablet_service.cc:1467] Processing CreateTablet for tablet 126a163a89c448be89ccb9490e9f0168 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20250109 08:57:28.435034 21579 tablet_service.cc:1467] Processing CreateTablet for tablet d52f79ecf037457cbcc09d5fa43456cf (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20250109 08:57:28.436398 21579 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d52f79ecf037457cbcc09d5fa43456cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:28.439399 21576 tablet_service.cc:1467] Processing CreateTablet for tablet 5717a02ef53a4b22a96f431ca6c20d93 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20250109 08:57:28.396916 21703 raft_consensus.cc:738] T 610441ef4b1d40c284eff3a714e4214e P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bf6e26e98b14566899578a990b27d47, State: Initialized, Role: FOLLOWER
I20250109 08:57:28.384182 21574 tablet_service.cc:1467] Processing CreateTablet for tablet 56bbc306275e4a409e028a7ca1dd1602 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20250109 08:57:28.448933 21576 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5717a02ef53a4b22a96f431ca6c20d93. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:28.457257 21646 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 126a163a89c448be89ccb9490e9f0168. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:28.458437 21703 consensus_queue.cc:260] T 610441ef4b1d40c284eff3a714e4214e P 8bf6e26e98b14566899578a990b27d47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:28.463269 21571 tablet_service.cc:1467] Processing CreateTablet for tablet 5d31dbfb2b4a4a6ca4500f92df9dbb98 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20250109 08:57:28.464010 21703 ts_tablet_manager.cc:1428] T 610441ef4b1d40c284eff3a714e4214e P 8bf6e26e98b14566899578a990b27d47: Time spent starting tablet: real 0.086s user 0.005s sys 0.002s
I20250109 08:57:28.465052 21652 tablet_service.cc:1467] Processing CreateTablet for tablet 7781c00473024307987096b596da12ac (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20250109 08:57:28.464999 21703 tablet_bootstrap.cc:492] T db34a00c02b84ee988edc7d6e344c1f5 P 8bf6e26e98b14566899578a990b27d47: Bootstrap starting.
I20250109 08:57:28.467301 21574 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56bbc306275e4a409e028a7ca1dd1602. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:28.467914 21571 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d31dbfb2b4a4a6ca4500f92df9dbb98. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:28.473781 21649 tablet_service.cc:1467] Processing CreateTablet for tablet 5d31dbfb2b4a4a6ca4500f92df9dbb98 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20250109 08:57:28.466934 21653 tablet_service.cc:1467] Processing CreateTablet for tablet 8c0459d2eee64f0a83bd05ad5974f506 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20250109 08:57:28.477046 21703 tablet_bootstrap.cc:654] T db34a00c02b84ee988edc7d6e344c1f5 P 8bf6e26e98b14566899578a990b27d47: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:28.477385 21645 tablet_service.cc:1467] Processing CreateTablet for tablet f961d3f357944118a69f905621731552 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20250109 08:57:28.481846 21704 tablet_bootstrap.cc:492] T 20ea30dba4314290ae45d9b4c13bebdd P 787ddcc824c2490a808fc5a53452dc5e: No bootstrap required, opened a new log
I20250109 08:57:28.482527 21704 ts_tablet_manager.cc:1397] T 20ea30dba4314290ae45d9b4c13bebdd P 787ddcc824c2490a808fc5a53452dc5e: Time spent bootstrapping tablet: real 0.112s user 0.017s sys 0.039s
I20250109 08:57:28.483042 21650 tablet_service.cc:1467] Processing CreateTablet for tablet ab6c3fcdf226433c9e1303004f6c96b4 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20250109 08:57:28.479851 21572 tablet_service.cc:1467] Processing CreateTablet for tablet 2935d69446cd458e92046d2b2f1b74ab (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20250109 08:57:28.499711 21652 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7781c00473024307987096b596da12ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:28.502404 21653 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8c0459d2eee64f0a83bd05ad5974f506. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:28.504786 21649 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d31dbfb2b4a4a6ca4500f92df9dbb98. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:28.509950 21650 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab6c3fcdf226433c9e1303004f6c96b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:28.514555 21570 tablet_service.cc:1467] Processing CreateTablet for tablet edc9db96e0ff4aa5839a6922f3eb5f24 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20250109 08:57:28.515916 21570 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet edc9db96e0ff4aa5839a6922f3eb5f24. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:28.517388 21573 tablet_service.cc:1467] Processing CreateTablet for tablet 7c19ed9fdde5463f84e43bea7b860d03 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20250109 08:57:28.518654 21573 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c19ed9fdde5463f84e43bea7b860d03. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:28.521778 21644 tablet_service.cc:1467] Processing CreateTablet for tablet a3aa271f2f0144eea9261e0e07ab833b (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20250109 08:57:28.524281 21572 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2935d69446cd458e92046d2b2f1b74ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:28.520087 21647 tablet_service.cc:1467] Processing CreateTablet for tablet 64c667f887ea41b4a484ee565b21fa16 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20250109 08:57:28.528854 21644 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3aa271f2f0144eea9261e0e07ab833b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:28.530005 21704 raft_consensus.cc:357] T 20ea30dba4314290ae45d9b4c13bebdd P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:28.530756 21704 raft_consensus.cc:383] T 20ea30dba4314290ae45d9b4c13bebdd P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:28.531047 21704 raft_consensus.cc:738] T 20ea30dba4314290ae45d9b4c13bebdd P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Initialized, Role: FOLLOWER
I20250109 08:57:28.534814 21647 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 64c667f887ea41b4a484ee565b21fa16. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:28.539695 21702 tablet_bootstrap.cc:492] T 206785f29c824cddbe9bbc00d365ffb3 P a2b1a692435241ee988e0bd1661cd4c3: No bootstrap required, opened a new log
I20250109 08:57:28.540200 21702 ts_tablet_manager.cc:1397] T 206785f29c824cddbe9bbc00d365ffb3 P a2b1a692435241ee988e0bd1661cd4c3: Time spent bootstrapping tablet: real 0.178s user 0.031s sys 0.033s
I20250109 08:57:28.531718 21704 consensus_queue.cc:260] T 20ea30dba4314290ae45d9b4c13bebdd P 787ddcc824c2490a808fc5a53452dc5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:28.542447 21704 ts_tablet_manager.cc:1428] T 20ea30dba4314290ae45d9b4c13bebdd P 787ddcc824c2490a808fc5a53452dc5e: Time spent starting tablet: real 0.059s user 0.005s sys 0.000s
I20250109 08:57:28.543402 21704 tablet_bootstrap.cc:492] T 610441ef4b1d40c284eff3a714e4214e P 787ddcc824c2490a808fc5a53452dc5e: Bootstrap starting.
I20250109 08:57:28.543052 21702 raft_consensus.cc:357] T 206785f29c824cddbe9bbc00d365ffb3 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } }
I20250109 08:57:28.543792 21702 raft_consensus.cc:383] T 206785f29c824cddbe9bbc00d365ffb3 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:28.544070 21702 raft_consensus.cc:738] T 206785f29c824cddbe9bbc00d365ffb3 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2b1a692435241ee988e0bd1661cd4c3, State: Initialized, Role: FOLLOWER
I20250109 08:57:28.545015 21702 consensus_queue.cc:260] T 206785f29c824cddbe9bbc00d365ffb3 P a2b1a692435241ee988e0bd1661cd4c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } }
I20250109 08:57:28.547240 21702 ts_tablet_manager.cc:1428] T 206785f29c824cddbe9bbc00d365ffb3 P a2b1a692435241ee988e0bd1661cd4c3: Time spent starting tablet: real 0.007s user 0.004s sys 0.000s
I20250109 08:57:28.547402 21645 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f961d3f357944118a69f905621731552. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:28.551236 21703 tablet_bootstrap.cc:492] T db34a00c02b84ee988edc7d6e344c1f5 P 8bf6e26e98b14566899578a990b27d47: No bootstrap required, opened a new log
I20250109 08:57:28.551748 21703 ts_tablet_manager.cc:1397] T db34a00c02b84ee988edc7d6e344c1f5 P 8bf6e26e98b14566899578a990b27d47: Time spent bootstrapping tablet: real 0.087s user 0.005s sys 0.016s
I20250109 08:57:28.556984 21703 raft_consensus.cc:357] T db34a00c02b84ee988edc7d6e344c1f5 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:28.557788 21703 raft_consensus.cc:383] T db34a00c02b84ee988edc7d6e344c1f5 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:28.560213 21514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e601a034e10e418bac8683443d86a1c1" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47" is_pre_election: true
W20250109 08:57:28.562044 21626 leader_election.cc:343] T e601a034e10e418bac8683443d86a1c1 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:28.558075 21703 raft_consensus.cc:738] T db34a00c02b84ee988edc7d6e344c1f5 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bf6e26e98b14566899578a990b27d47, State: Initialized, Role: FOLLOWER
I20250109 08:57:28.548171 21702 tablet_bootstrap.cc:492] T db34a00c02b84ee988edc7d6e344c1f5 P a2b1a692435241ee988e0bd1661cd4c3: Bootstrap starting.
I20250109 08:57:28.583298 21702 tablet_bootstrap.cc:654] T db34a00c02b84ee988edc7d6e344c1f5 P a2b1a692435241ee988e0bd1661cd4c3: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:28.606400 21652 tablet_service.cc:1467] Processing CreateTablet for tablet 1b582ab8d8f044d9b9038a28e958feeb (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20250109 08:57:28.607784 21652 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b582ab8d8f044d9b9038a28e958feeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:28.614114 21644 tablet_service.cc:1467] Processing CreateTablet for tablet f5f85ad7c7c94acfaf9d550d55fce48c (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20250109 08:57:28.615360 21514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37280454805d47f98a415a8f0c16d869" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47" is_pre_election: true
I20250109 08:57:28.606583 21651 tablet_service.cc:1467] Processing CreateTablet for tablet d015a10f95eb4f19b6d7c53cb201fa80 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
W20250109 08:57:28.617358 21552 leader_election.cc:343] T 37280454805d47f98a415a8f0c16d869 P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:28.615361 21644 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5f85ad7c7c94acfaf9d550d55fce48c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:28.563513 21703 consensus_queue.cc:260] T db34a00c02b84ee988edc7d6e344c1f5 P 8bf6e26e98b14566899578a990b27d47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:28.627554 21704 tablet_bootstrap.cc:654] T 610441ef4b1d40c284eff3a714e4214e P 787ddcc824c2490a808fc5a53452dc5e: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:28.628767 21703 ts_tablet_manager.cc:1428] T db34a00c02b84ee988edc7d6e344c1f5 P 8bf6e26e98b14566899578a990b27d47: Time spent starting tablet: real 0.077s user 0.005s sys 0.004s
I20250109 08:57:28.629812 21703 tablet_bootstrap.cc:492] T 97304b06c7fa40ef8345a72c796c47f8 P 8bf6e26e98b14566899578a990b27d47: Bootstrap starting.
I20250109 08:57:28.633625 21663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "206785f29c824cddbe9bbc00d365ffb3" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e" is_pre_election: true
W20250109 08:57:28.635210 21477 leader_election.cc:343] T 206785f29c824cddbe9bbc00d365ffb3 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:28.636696 21651 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d015a10f95eb4f19b6d7c53cb201fa80. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:28.638100 21650 tablet_service.cc:1467] Processing CreateTablet for tablet d52f79ecf037457cbcc09d5fa43456cf (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20250109 08:57:28.638489 21653 tablet_service.cc:1467] Processing CreateTablet for tablet 1d3d5d2a2c88417a97c9d57262a356dd (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20250109 08:57:28.639308 21650 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d52f79ecf037457cbcc09d5fa43456cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:28.642074 21702 tablet_bootstrap.cc:492] T db34a00c02b84ee988edc7d6e344c1f5 P a2b1a692435241ee988e0bd1661cd4c3: No bootstrap required, opened a new log
I20250109 08:57:28.642540 21702 ts_tablet_manager.cc:1397] T db34a00c02b84ee988edc7d6e344c1f5 P a2b1a692435241ee988e0bd1661cd4c3: Time spent bootstrapping tablet: real 0.095s user 0.009s sys 0.010s
I20250109 08:57:28.643689 21652 tablet_service.cc:1467] Processing CreateTablet for tablet 5717a02ef53a4b22a96f431ca6c20d93 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20250109 08:57:28.645007 21648 tablet_service.cc:1467] Processing CreateTablet for tablet 56bbc306275e4a409e028a7ca1dd1602 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20250109 08:57:28.648584 21653 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d3d5d2a2c88417a97c9d57262a356dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:28.656895 21649 tablet_service.cc:1467] Processing CreateTablet for tablet 2935d69446cd458e92046d2b2f1b74ab (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20250109 08:57:28.659366 21663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37280454805d47f98a415a8f0c16d869" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e" is_pre_election: true
I20250109 08:57:28.659475 21589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "206785f29c824cddbe9bbc00d365ffb3" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3" is_pre_election: true
I20250109 08:57:28.660140 21589 raft_consensus.cc:2463] T 206785f29c824cddbe9bbc00d365ffb3 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8bf6e26e98b14566899578a990b27d47 in term 0.
W20250109 08:57:28.660717 21552 leader_election.cc:343] T 37280454805d47f98a415a8f0c16d869 P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:28.661180 21552 leader_election.cc:304] T 37280454805d47f98a415a8f0c16d869 P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2b1a692435241ee988e0bd1661cd4c3; no voters: 787ddcc824c2490a808fc5a53452dc5e, 8bf6e26e98b14566899578a990b27d47
I20250109 08:57:28.661465 21478 leader_election.cc:304] T 206785f29c824cddbe9bbc00d365ffb3 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3; no voters: 787ddcc824c2490a808fc5a53452dc5e
I20250109 08:57:28.661545 21702 raft_consensus.cc:357] T db34a00c02b84ee988edc7d6e344c1f5 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:28.662431 21709 raft_consensus.cc:2743] T 37280454805d47f98a415a8f0c16d869 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:57:28.663295 21702 raft_consensus.cc:383] T db34a00c02b84ee988edc7d6e344c1f5 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:28.662719 21711 raft_consensus.cc:2798] T 206785f29c824cddbe9bbc00d365ffb3 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:57:28.663673 21711 raft_consensus.cc:491] T 206785f29c824cddbe9bbc00d365ffb3 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:57:28.663938 21711 raft_consensus.cc:3054] T 206785f29c824cddbe9bbc00d365ffb3 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:28.665381 21703 tablet_bootstrap.cc:654] T 97304b06c7fa40ef8345a72c796c47f8 P 8bf6e26e98b14566899578a990b27d47: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:28.668033 21702 raft_consensus.cc:738] T db34a00c02b84ee988edc7d6e344c1f5 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2b1a692435241ee988e0bd1661cd4c3, State: Initialized, Role: FOLLOWER
I20250109 08:57:28.669883 21702 consensus_queue.cc:260] T db34a00c02b84ee988edc7d6e344c1f5 P a2b1a692435241ee988e0bd1661cd4c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:28.672170 21702 ts_tablet_manager.cc:1428] T db34a00c02b84ee988edc7d6e344c1f5 P a2b1a692435241ee988e0bd1661cd4c3: Time spent starting tablet: real 0.029s user 0.003s sys 0.003s
I20250109 08:57:28.672804 21711 raft_consensus.cc:513] T 206785f29c824cddbe9bbc00d365ffb3 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } }
I20250109 08:57:28.710486 21711 leader_election.cc:290] T 206785f29c824cddbe9bbc00d365ffb3 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 election: Requested vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:28.680549 21652 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5717a02ef53a4b22a96f431ca6c20d93. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:28.681572 21650 tablet_service.cc:1467] Processing CreateTablet for tablet edc9db96e0ff4aa5839a6922f3eb5f24 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20250109 08:57:28.690971 21646 tablet_service.cc:1467] Processing CreateTablet for tablet 7c19ed9fdde5463f84e43bea7b860d03 (DEFAULT_TABLE table=test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20250109 08:57:28.711602 21588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "206785f29c824cddbe9bbc00d365ffb3" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3"
I20250109 08:57:28.693626 21704 tablet_bootstrap.cc:492] T 610441ef4b1d40c284eff3a714e4214e P 787ddcc824c2490a808fc5a53452dc5e: No bootstrap required, opened a new log
I20250109 08:57:28.712750 21663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "206785f29c824cddbe9bbc00d365ffb3" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e"
I20250109 08:57:28.713464 21650 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet edc9db96e0ff4aa5839a6922f3eb5f24. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:28.714223 21702 tablet_bootstrap.cc:492] T 20ea30dba4314290ae45d9b4c13bebdd P a2b1a692435241ee988e0bd1661cd4c3: Bootstrap starting.
I20250109 08:57:28.714541 21646 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c19ed9fdde5463f84e43bea7b860d03. 1 dirs total, 0 dirs full, 0 dirs failed
W20250109 08:57:28.715533 21477 leader_election.cc:343] T 206785f29c824cddbe9bbc00d365ffb3 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:28.712313 21588 raft_consensus.cc:3054] T 206785f29c824cddbe9bbc00d365ffb3 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:28.712692 21704 ts_tablet_manager.cc:1397] T 610441ef4b1d40c284eff3a714e4214e P 787ddcc824c2490a808fc5a53452dc5e: Time spent bootstrapping tablet: real 0.169s user 0.008s sys 0.006s
I20250109 08:57:28.720299 21704 raft_consensus.cc:357] T 610441ef4b1d40c284eff3a714e4214e P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:28.721084 21704 raft_consensus.cc:383] T 610441ef4b1d40c284eff3a714e4214e P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:28.721388 21704 raft_consensus.cc:738] T 610441ef4b1d40c284eff3a714e4214e P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Initialized, Role: FOLLOWER
I20250109 08:57:28.722091 21704 consensus_queue.cc:260] T 610441ef4b1d40c284eff3a714e4214e P 787ddcc824c2490a808fc5a53452dc5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:28.723361 21711 raft_consensus.cc:491] T db34a00c02b84ee988edc7d6e344c1f5 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:28.723812 21711 raft_consensus.cc:513] T db34a00c02b84ee988edc7d6e344c1f5 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:28.726855 21587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db34a00c02b84ee988edc7d6e344c1f5" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3" is_pre_election: true
I20250109 08:57:28.727540 21587 raft_consensus.cc:2463] T db34a00c02b84ee988edc7d6e344c1f5 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8bf6e26e98b14566899578a990b27d47 in term 0.
I20250109 08:57:28.727927 21663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db34a00c02b84ee988edc7d6e344c1f5" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e" is_pre_election: true
I20250109 08:57:28.698369 21589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e601a034e10e418bac8683443d86a1c1" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3" is_pre_election: true
I20250109 08:57:28.729128 21478 leader_election.cc:304] T db34a00c02b84ee988edc7d6e344c1f5 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3; no voters:
I20250109 08:57:28.729722 21710 raft_consensus.cc:491] T 610441ef4b1d40c284eff3a714e4214e P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:28.730182 21710 raft_consensus.cc:513] T 610441ef4b1d40c284eff3a714e4214e P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:28.730587 21704 ts_tablet_manager.cc:1428] T 610441ef4b1d40c284eff3a714e4214e P 787ddcc824c2490a808fc5a53452dc5e: Time spent starting tablet: real 0.013s user 0.005s sys 0.000s
W20250109 08:57:28.731042 21477 leader_election.cc:343] T db34a00c02b84ee988edc7d6e344c1f5 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731): Illegal state: must be running to vote when last-logged opid is not known
W20250109 08:57:28.730715 21627 leader_election.cc:343] T e601a034e10e418bac8683443d86a1c1 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:28.732008 21627 leader_election.cc:304] T e601a034e10e418bac8683443d86a1c1 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e; no voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:57:28.732858 21514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "610441ef4b1d40c284eff3a714e4214e" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47" is_pre_election: true
I20250109 08:57:28.732146 21710 leader_election.cc:290] T 610441ef4b1d40c284eff3a714e4214e P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
I20250109 08:57:28.733387 21514 raft_consensus.cc:2463] T 610441ef4b1d40c284eff3a714e4214e P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 0.
I20250109 08:57:28.734684 21626 leader_election.cc:304] T 610441ef4b1d40c284eff3a714e4214e P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, 8bf6e26e98b14566899578a990b27d47; no voters:
I20250109 08:57:28.735417 21589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "610441ef4b1d40c284eff3a714e4214e" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3" is_pre_election: true
I20250109 08:57:28.736006 21732 raft_consensus.cc:2798] T 610441ef4b1d40c284eff3a714e4214e P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:57:28.733438 21710 raft_consensus.cc:2743] T e601a034e10e418bac8683443d86a1c1 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:57:28.736487 21732 raft_consensus.cc:491] T 610441ef4b1d40c284eff3a714e4214e P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:57:28.736971 21732 raft_consensus.cc:3054] T 610441ef4b1d40c284eff3a714e4214e P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Advancing to term 1
W20250109 08:57:28.737196 21627 leader_election.cc:343] T 610441ef4b1d40c284eff3a714e4214e P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:28.727623 21711 leader_election.cc:290] T db34a00c02b84ee988edc7d6e344c1f5 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:28.738226 21711 raft_consensus.cc:2798] T db34a00c02b84ee988edc7d6e344c1f5 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:57:28.738565 21711 raft_consensus.cc:491] T db34a00c02b84ee988edc7d6e344c1f5 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:57:28.738956 21711 raft_consensus.cc:3054] T db34a00c02b84ee988edc7d6e344c1f5 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:28.720790 21703 tablet_bootstrap.cc:492] T 97304b06c7fa40ef8345a72c796c47f8 P 8bf6e26e98b14566899578a990b27d47: No bootstrap required, opened a new log
I20250109 08:57:28.740687 21703 ts_tablet_manager.cc:1397] T 97304b06c7fa40ef8345a72c796c47f8 P 8bf6e26e98b14566899578a990b27d47: Time spent bootstrapping tablet: real 0.111s user 0.012s sys 0.002s
I20250109 08:57:28.740777 21649 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2935d69446cd458e92046d2b2f1b74ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:28.731555 21704 tablet_bootstrap.cc:492] T 80123146836349938344efc61333695f P 787ddcc824c2490a808fc5a53452dc5e: Bootstrap starting.
I20250109 08:57:28.744331 21732 raft_consensus.cc:513] T 610441ef4b1d40c284eff3a714e4214e P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:28.745256 21711 raft_consensus.cc:513] T db34a00c02b84ee988edc7d6e344c1f5 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:28.746364 21588 raft_consensus.cc:2463] T 206785f29c824cddbe9bbc00d365ffb3 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8bf6e26e98b14566899578a990b27d47 in term 1.
I20250109 08:57:28.747206 21711 leader_election.cc:290] T db34a00c02b84ee988edc7d6e344c1f5 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 election: Requested vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:28.747480 21514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "610441ef4b1d40c284eff3a714e4214e" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47"
I20250109 08:57:28.747670 21732 leader_election.cc:290] T 610441ef4b1d40c284eff3a714e4214e P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 election: Requested vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
I20250109 08:57:28.748073 21514 raft_consensus.cc:3054] T 610441ef4b1d40c284eff3a714e4214e P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:28.748236 21663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db34a00c02b84ee988edc7d6e344c1f5" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e"
W20250109 08:57:28.751088 21477 leader_election.cc:343] T db34a00c02b84ee988edc7d6e344c1f5 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:28.752175 21589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "610441ef4b1d40c284eff3a714e4214e" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3"
W20250109 08:57:28.753552 21627 leader_election.cc:343] T 610441ef4b1d40c284eff3a714e4214e P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:28.754714 21514 raft_consensus.cc:2463] T 610441ef4b1d40c284eff3a714e4214e P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 1.
I20250109 08:57:28.755821 21626 leader_election.cc:304] T 610441ef4b1d40c284eff3a714e4214e P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, 8bf6e26e98b14566899578a990b27d47; no voters: a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:57:28.750969 21703 raft_consensus.cc:357] T 97304b06c7fa40ef8345a72c796c47f8 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:28.756614 21648 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56bbc306275e4a409e028a7ca1dd1602. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:28.756727 21703 raft_consensus.cc:383] T 97304b06c7fa40ef8345a72c796c47f8 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:28.760558 21703 raft_consensus.cc:738] T 97304b06c7fa40ef8345a72c796c47f8 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bf6e26e98b14566899578a990b27d47, State: Initialized, Role: FOLLOWER
I20250109 08:57:28.761255 21703 consensus_queue.cc:260] T 97304b06c7fa40ef8345a72c796c47f8 P 8bf6e26e98b14566899578a990b27d47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:28.763559 21703 ts_tablet_manager.cc:1428] T 97304b06c7fa40ef8345a72c796c47f8 P 8bf6e26e98b14566899578a990b27d47: Time spent starting tablet: real 0.015s user 0.004s sys 0.000s
I20250109 08:57:28.756991 21732 raft_consensus.cc:2798] T 610441ef4b1d40c284eff3a714e4214e P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:57:28.759624 21704 tablet_bootstrap.cc:654] T 80123146836349938344efc61333695f P 787ddcc824c2490a808fc5a53452dc5e: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:28.764652 21732 raft_consensus.cc:695] T 610441ef4b1d40c284eff3a714e4214e P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Becoming Leader. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Running, Role: LEADER
I20250109 08:57:28.738538 21702 tablet_bootstrap.cc:654] T 20ea30dba4314290ae45d9b4c13bebdd P a2b1a692435241ee988e0bd1661cd4c3: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:28.774737 21702 tablet_bootstrap.cc:492] T 20ea30dba4314290ae45d9b4c13bebdd P a2b1a692435241ee988e0bd1661cd4c3: No bootstrap required, opened a new log
I20250109 08:57:28.775224 21702 ts_tablet_manager.cc:1397] T 20ea30dba4314290ae45d9b4c13bebdd P a2b1a692435241ee988e0bd1661cd4c3: Time spent bootstrapping tablet: real 0.061s user 0.010s sys 0.000s
I20250109 08:57:28.776090 21732 consensus_queue.cc:237] T 610441ef4b1d40c284eff3a714e4214e P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:28.777990 21702 raft_consensus.cc:357] T 20ea30dba4314290ae45d9b4c13bebdd P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:28.778761 21702 raft_consensus.cc:383] T 20ea30dba4314290ae45d9b4c13bebdd P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:28.779053 21702 raft_consensus.cc:738] T 20ea30dba4314290ae45d9b4c13bebdd P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2b1a692435241ee988e0bd1661cd4c3, State: Initialized, Role: FOLLOWER
I20250109 08:57:28.780012 21703 tablet_bootstrap.cc:492] T 20ea30dba4314290ae45d9b4c13bebdd P 8bf6e26e98b14566899578a990b27d47: Bootstrap starting.
W20250109 08:57:28.781664 21478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.226.194:45527, user_credentials={real_user=slave}} blocked reactor thread for 52623.8us
I20250109 08:57:28.783609 21588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db34a00c02b84ee988edc7d6e344c1f5" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3"
I20250109 08:57:28.784220 21588 raft_consensus.cc:3054] T db34a00c02b84ee988edc7d6e344c1f5 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:28.786387 21702 consensus_queue.cc:260] T 20ea30dba4314290ae45d9b4c13bebdd P a2b1a692435241ee988e0bd1661cd4c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:28.787307 21703 tablet_bootstrap.cc:654] T 20ea30dba4314290ae45d9b4c13bebdd P 8bf6e26e98b14566899578a990b27d47: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:28.788918 21702 ts_tablet_manager.cc:1428] T 20ea30dba4314290ae45d9b4c13bebdd P a2b1a692435241ee988e0bd1661cd4c3: Time spent starting tablet: real 0.013s user 0.008s sys 0.000s
I20250109 08:57:28.789911 21702 tablet_bootstrap.cc:492] T c761cb691bd241ae825917a3fbc3ad24 P a2b1a692435241ee988e0bd1661cd4c3: Bootstrap starting.
I20250109 08:57:28.795949 21702 tablet_bootstrap.cc:654] T c761cb691bd241ae825917a3fbc3ad24 P a2b1a692435241ee988e0bd1661cd4c3: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:28.799579 21703 tablet_bootstrap.cc:492] T 20ea30dba4314290ae45d9b4c13bebdd P 8bf6e26e98b14566899578a990b27d47: No bootstrap required, opened a new log
I20250109 08:57:28.800061 21703 ts_tablet_manager.cc:1397] T 20ea30dba4314290ae45d9b4c13bebdd P 8bf6e26e98b14566899578a990b27d47: Time spent bootstrapping tablet: real 0.036s user 0.005s sys 0.012s
I20250109 08:57:28.801018 21478 leader_election.cc:304] T 206785f29c824cddbe9bbc00d365ffb3 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3; no voters: 787ddcc824c2490a808fc5a53452dc5e
I20250109 08:57:28.796084 21411 catalog_manager.cc:5526] T 610441ef4b1d40c284eff3a714e4214e P 787ddcc824c2490a808fc5a53452dc5e reported cstate change: term changed from 0 to 1, leader changed from <none> to 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195). New cstate: current_term: 1 leader_uuid: "787ddcc824c2490a808fc5a53452dc5e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: HEALTHY } } }
I20250109 08:57:28.802901 21703 raft_consensus.cc:357] T 20ea30dba4314290ae45d9b4c13bebdd P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:28.803681 21703 raft_consensus.cc:383] T 20ea30dba4314290ae45d9b4c13bebdd P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:28.804001 21703 raft_consensus.cc:738] T 20ea30dba4314290ae45d9b4c13bebdd P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bf6e26e98b14566899578a990b27d47, State: Initialized, Role: FOLLOWER
I20250109 08:57:28.804040 21711 raft_consensus.cc:2798] T 206785f29c824cddbe9bbc00d365ffb3 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:57:28.808660 21731 raft_consensus.cc:491] T 97304b06c7fa40ef8345a72c796c47f8 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:28.809821 21702 tablet_bootstrap.cc:492] T c761cb691bd241ae825917a3fbc3ad24 P a2b1a692435241ee988e0bd1661cd4c3: No bootstrap required, opened a new log
I20250109 08:57:28.812630 21731 raft_consensus.cc:513] T 97304b06c7fa40ef8345a72c796c47f8 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:28.813339 21703 consensus_queue.cc:260] T 20ea30dba4314290ae45d9b4c13bebdd P 8bf6e26e98b14566899578a990b27d47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:28.814335 21702 ts_tablet_manager.cc:1397] T c761cb691bd241ae825917a3fbc3ad24 P a2b1a692435241ee988e0bd1661cd4c3: Time spent bootstrapping tablet: real 0.025s user 0.008s sys 0.005s
I20250109 08:57:28.816629 21588 raft_consensus.cc:2463] T db34a00c02b84ee988edc7d6e344c1f5 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8bf6e26e98b14566899578a990b27d47 in term 1.
I20250109 08:57:28.818074 21589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97304b06c7fa40ef8345a72c796c47f8" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3" is_pre_election: true
I20250109 08:57:28.818181 21478 leader_election.cc:304] T db34a00c02b84ee988edc7d6e344c1f5 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3; no voters: 787ddcc824c2490a808fc5a53452dc5e
I20250109 08:57:28.819957 21663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97304b06c7fa40ef8345a72c796c47f8" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e" is_pre_election: true
W20250109 08:57:28.821481 21477 leader_election.cc:343] T 97304b06c7fa40ef8345a72c796c47f8 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:28.819453 21731 leader_election.cc:290] T 97304b06c7fa40ef8345a72c796c47f8 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:28.821558 21702 raft_consensus.cc:357] T c761cb691bd241ae825917a3fbc3ad24 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:28.822625 21702 raft_consensus.cc:383] T c761cb691bd241ae825917a3fbc3ad24 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:28.822990 21702 raft_consensus.cc:738] T c761cb691bd241ae825917a3fbc3ad24 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2b1a692435241ee988e0bd1661cd4c3, State: Initialized, Role: FOLLOWER
I20250109 08:57:28.823290 21731 raft_consensus.cc:2798] T db34a00c02b84ee988edc7d6e344c1f5 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:57:28.823746 21731 raft_consensus.cc:695] T db34a00c02b84ee988edc7d6e344c1f5 P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Becoming Leader. State: Replica: 8bf6e26e98b14566899578a990b27d47, State: Running, Role: LEADER
I20250109 08:57:28.823836 21702 consensus_queue.cc:260] T c761cb691bd241ae825917a3fbc3ad24 P a2b1a692435241ee988e0bd1661cd4c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:28.824620 21731 consensus_queue.cc:237] T db34a00c02b84ee988edc7d6e344c1f5 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:28.826777 21702 ts_tablet_manager.cc:1428] T c761cb691bd241ae825917a3fbc3ad24 P a2b1a692435241ee988e0bd1661cd4c3: Time spent starting tablet: real 0.011s user 0.006s sys 0.000s
I20250109 08:57:28.832028 21704 tablet_bootstrap.cc:492] T 80123146836349938344efc61333695f P 787ddcc824c2490a808fc5a53452dc5e: No bootstrap required, opened a new log
I20250109 08:57:28.835072 21704 ts_tablet_manager.cc:1397] T 80123146836349938344efc61333695f P 787ddcc824c2490a808fc5a53452dc5e: Time spent bootstrapping tablet: real 0.104s user 0.016s sys 0.023s
I20250109 08:57:28.838780 21704 raft_consensus.cc:357] T 80123146836349938344efc61333695f P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:28.843375 21704 raft_consensus.cc:383] T 80123146836349938344efc61333695f P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:28.844272 21704 raft_consensus.cc:738] T 80123146836349938344efc61333695f P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Initialized, Role: FOLLOWER
I20250109 08:57:28.827776 21711 raft_consensus.cc:695] T 206785f29c824cddbe9bbc00d365ffb3 P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Becoming Leader. State: Replica: 8bf6e26e98b14566899578a990b27d47, State: Running, Role: LEADER
I20250109 08:57:28.846146 21704 consensus_queue.cc:260] T 80123146836349938344efc61333695f P 787ddcc824c2490a808fc5a53452dc5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:28.851099 21702 tablet_bootstrap.cc:492] T 7a7211b8d4374ad1afd82e8cf85bad81 P a2b1a692435241ee988e0bd1661cd4c3: Bootstrap starting.
I20250109 08:57:28.850482 21704 ts_tablet_manager.cc:1428] T 80123146836349938344efc61333695f P 787ddcc824c2490a808fc5a53452dc5e: Time spent starting tablet: real 0.014s user 0.006s sys 0.000s
I20250109 08:57:28.856150 21704 tablet_bootstrap.cc:492] T 7a7211b8d4374ad1afd82e8cf85bad81 P 787ddcc824c2490a808fc5a53452dc5e: Bootstrap starting.
I20250109 08:57:28.857383 21711 consensus_queue.cc:237] T 206785f29c824cddbe9bbc00d365ffb3 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } }
I20250109 08:57:28.860991 21703 ts_tablet_manager.cc:1428] T 20ea30dba4314290ae45d9b4c13bebdd P 8bf6e26e98b14566899578a990b27d47: Time spent starting tablet: real 0.061s user 0.012s sys 0.015s
I20250109 08:57:28.861972 21703 tablet_bootstrap.cc:492] T 37280454805d47f98a415a8f0c16d869 P 8bf6e26e98b14566899578a990b27d47: Bootstrap starting.
I20250109 08:57:28.862160 21704 tablet_bootstrap.cc:654] T 7a7211b8d4374ad1afd82e8cf85bad81 P 787ddcc824c2490a808fc5a53452dc5e: Neither blocks nor log segments found. Creating new log.
W20250109 08:57:28.865489 21478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.226.194:45527, user_credentials={real_user=slave}} blocked reactor thread for 47267.9us
W20250109 08:57:28.866212 21478 leader_election.cc:343] T 97304b06c7fa40ef8345a72c796c47f8 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:28.866519 21478 leader_election.cc:304] T 97304b06c7fa40ef8345a72c796c47f8 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8bf6e26e98b14566899578a990b27d47; no voters: 787ddcc824c2490a808fc5a53452dc5e, a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:57:28.867229 21740 raft_consensus.cc:2743] T 97304b06c7fa40ef8345a72c796c47f8 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:57:28.868110 21703 tablet_bootstrap.cc:654] T 37280454805d47f98a415a8f0c16d869 P 8bf6e26e98b14566899578a990b27d47: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:28.870592 21702 tablet_bootstrap.cc:654] T 7a7211b8d4374ad1afd82e8cf85bad81 P a2b1a692435241ee988e0bd1661cd4c3: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:28.880343 21704 tablet_bootstrap.cc:492] T 7a7211b8d4374ad1afd82e8cf85bad81 P 787ddcc824c2490a808fc5a53452dc5e: No bootstrap required, opened a new log
I20250109 08:57:28.880916 21704 ts_tablet_manager.cc:1397] T 7a7211b8d4374ad1afd82e8cf85bad81 P 787ddcc824c2490a808fc5a53452dc5e: Time spent bootstrapping tablet: real 0.025s user 0.007s sys 0.005s
I20250109 08:57:28.881009 21702 tablet_bootstrap.cc:492] T 7a7211b8d4374ad1afd82e8cf85bad81 P a2b1a692435241ee988e0bd1661cd4c3: No bootstrap required, opened a new log
I20250109 08:57:28.881814 21703 tablet_bootstrap.cc:492] T 37280454805d47f98a415a8f0c16d869 P 8bf6e26e98b14566899578a990b27d47: No bootstrap required, opened a new log
I20250109 08:57:28.882226 21702 ts_tablet_manager.cc:1397] T 7a7211b8d4374ad1afd82e8cf85bad81 P a2b1a692435241ee988e0bd1661cd4c3: Time spent bootstrapping tablet: real 0.037s user 0.012s sys 0.002s
I20250109 08:57:28.882352 21703 ts_tablet_manager.cc:1397] T 37280454805d47f98a415a8f0c16d869 P 8bf6e26e98b14566899578a990b27d47: Time spent bootstrapping tablet: real 0.021s user 0.009s sys 0.008s
I20250109 08:57:28.884071 21704 raft_consensus.cc:357] T 7a7211b8d4374ad1afd82e8cf85bad81 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:28.884938 21704 raft_consensus.cc:383] T 7a7211b8d4374ad1afd82e8cf85bad81 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:28.885363 21704 raft_consensus.cc:738] T 7a7211b8d4374ad1afd82e8cf85bad81 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Initialized, Role: FOLLOWER
I20250109 08:57:28.885021 21702 raft_consensus.cc:357] T 7a7211b8d4374ad1afd82e8cf85bad81 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:28.885175 21703 raft_consensus.cc:357] T 37280454805d47f98a415a8f0c16d869 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:28.885913 21702 raft_consensus.cc:383] T 7a7211b8d4374ad1afd82e8cf85bad81 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:28.885955 21703 raft_consensus.cc:383] T 37280454805d47f98a415a8f0c16d869 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:28.886325 21702 raft_consensus.cc:738] T 7a7211b8d4374ad1afd82e8cf85bad81 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2b1a692435241ee988e0bd1661cd4c3, State: Initialized, Role: FOLLOWER
I20250109 08:57:28.886360 21703 raft_consensus.cc:738] T 37280454805d47f98a415a8f0c16d869 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bf6e26e98b14566899578a990b27d47, State: Initialized, Role: FOLLOWER
I20250109 08:57:28.886484 21704 consensus_queue.cc:260] T 7a7211b8d4374ad1afd82e8cf85bad81 P 787ddcc824c2490a808fc5a53452dc5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:28.887331 21702 consensus_queue.cc:260] T 7a7211b8d4374ad1afd82e8cf85bad81 P a2b1a692435241ee988e0bd1661cd4c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:28.887313 21703 consensus_queue.cc:260] T 37280454805d47f98a415a8f0c16d869 P 8bf6e26e98b14566899578a990b27d47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:28.889739 21704 ts_tablet_manager.cc:1428] T 7a7211b8d4374ad1afd82e8cf85bad81 P 787ddcc824c2490a808fc5a53452dc5e: Time spent starting tablet: real 0.008s user 0.005s sys 0.002s
I20250109 08:57:28.890621 21703 ts_tablet_manager.cc:1428] T 37280454805d47f98a415a8f0c16d869 P 8bf6e26e98b14566899578a990b27d47: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20250109 08:57:28.890749 21704 tablet_bootstrap.cc:492] T c761cb691bd241ae825917a3fbc3ad24 P 787ddcc824c2490a808fc5a53452dc5e: Bootstrap starting.
I20250109 08:57:28.891562 21703 tablet_bootstrap.cc:492] T 7a7211b8d4374ad1afd82e8cf85bad81 P 8bf6e26e98b14566899578a990b27d47: Bootstrap starting.
W20250109 08:57:28.891865 21460 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20250109 08:57:28.892601 21702 ts_tablet_manager.cc:1428] T 7a7211b8d4374ad1afd82e8cf85bad81 P a2b1a692435241ee988e0bd1661cd4c3: Time spent starting tablet: real 0.010s user 0.006s sys 0.002s
I20250109 08:57:28.894563 21410 catalog_manager.cc:5526] T 206785f29c824cddbe9bbc00d365ffb3 P 8bf6e26e98b14566899578a990b27d47 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8bf6e26e98b14566899578a990b27d47 (127.19.226.193). New cstate: current_term: 1 leader_uuid: "8bf6e26e98b14566899578a990b27d47" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: HEALTHY } } }
I20250109 08:57:28.896701 21703 tablet_bootstrap.cc:654] T 7a7211b8d4374ad1afd82e8cf85bad81 P 8bf6e26e98b14566899578a990b27d47: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:28.897418 21704 tablet_bootstrap.cc:654] T c761cb691bd241ae825917a3fbc3ad24 P 787ddcc824c2490a808fc5a53452dc5e: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:28.902284 21702 tablet_bootstrap.cc:492] T 610441ef4b1d40c284eff3a714e4214e P a2b1a692435241ee988e0bd1661cd4c3: Bootstrap starting.
I20250109 08:57:28.908955 21702 tablet_bootstrap.cc:654] T 610441ef4b1d40c284eff3a714e4214e P a2b1a692435241ee988e0bd1661cd4c3: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:28.914739 21703 tablet_bootstrap.cc:492] T 7a7211b8d4374ad1afd82e8cf85bad81 P 8bf6e26e98b14566899578a990b27d47: No bootstrap required, opened a new log
I20250109 08:57:28.915257 21703 ts_tablet_manager.cc:1397] T 7a7211b8d4374ad1afd82e8cf85bad81 P 8bf6e26e98b14566899578a990b27d47: Time spent bootstrapping tablet: real 0.024s user 0.011s sys 0.011s
I20250109 08:57:28.915716 21704 tablet_bootstrap.cc:492] T c761cb691bd241ae825917a3fbc3ad24 P 787ddcc824c2490a808fc5a53452dc5e: No bootstrap required, opened a new log
I20250109 08:57:28.916179 21704 ts_tablet_manager.cc:1397] T c761cb691bd241ae825917a3fbc3ad24 P 787ddcc824c2490a808fc5a53452dc5e: Time spent bootstrapping tablet: real 0.026s user 0.016s sys 0.007s
I20250109 08:57:28.916978 21702 tablet_bootstrap.cc:492] T 610441ef4b1d40c284eff3a714e4214e P a2b1a692435241ee988e0bd1661cd4c3: No bootstrap required, opened a new log
I20250109 08:57:28.917426 21702 ts_tablet_manager.cc:1397] T 610441ef4b1d40c284eff3a714e4214e P a2b1a692435241ee988e0bd1661cd4c3: Time spent bootstrapping tablet: real 0.015s user 0.010s sys 0.001s
I20250109 08:57:28.917918 21703 raft_consensus.cc:357] T 7a7211b8d4374ad1afd82e8cf85bad81 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:28.918665 21703 raft_consensus.cc:383] T 7a7211b8d4374ad1afd82e8cf85bad81 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:28.919004 21703 raft_consensus.cc:738] T 7a7211b8d4374ad1afd82e8cf85bad81 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bf6e26e98b14566899578a990b27d47, State: Initialized, Role: FOLLOWER
I20250109 08:57:28.918742 21704 raft_consensus.cc:357] T c761cb691bd241ae825917a3fbc3ad24 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:28.919530 21704 raft_consensus.cc:383] T c761cb691bd241ae825917a3fbc3ad24 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:28.919833 21704 raft_consensus.cc:738] T c761cb691bd241ae825917a3fbc3ad24 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Initialized, Role: FOLLOWER
I20250109 08:57:28.919696 21703 consensus_queue.cc:260] T 7a7211b8d4374ad1afd82e8cf85bad81 P 8bf6e26e98b14566899578a990b27d47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:28.920176 21702 raft_consensus.cc:357] T 610441ef4b1d40c284eff3a714e4214e P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:28.920997 21702 raft_consensus.cc:383] T 610441ef4b1d40c284eff3a714e4214e P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:28.921301 21702 raft_consensus.cc:738] T 610441ef4b1d40c284eff3a714e4214e P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2b1a692435241ee988e0bd1661cd4c3, State: Initialized, Role: FOLLOWER
I20250109 08:57:28.922013 21703 ts_tablet_manager.cc:1428] T 7a7211b8d4374ad1afd82e8cf85bad81 P 8bf6e26e98b14566899578a990b27d47: Time spent starting tablet: real 0.006s user 0.004s sys 0.004s
I20250109 08:57:28.923416 21703 tablet_bootstrap.cc:492] T 79c1ed8ab551478291f1f24cc940cb27 P 8bf6e26e98b14566899578a990b27d47: Bootstrap starting.
I20250109 08:57:28.923141 21704 consensus_queue.cc:260] T c761cb691bd241ae825917a3fbc3ad24 P 787ddcc824c2490a808fc5a53452dc5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:28.923170 21702 consensus_queue.cc:260] T 610441ef4b1d40c284eff3a714e4214e P a2b1a692435241ee988e0bd1661cd4c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:28.925854 21704 ts_tablet_manager.cc:1428] T c761cb691bd241ae825917a3fbc3ad24 P 787ddcc824c2490a808fc5a53452dc5e: Time spent starting tablet: real 0.009s user 0.002s sys 0.004s
I20250109 08:57:28.927222 21704 tablet_bootstrap.cc:492] T 206785f29c824cddbe9bbc00d365ffb3 P 787ddcc824c2490a808fc5a53452dc5e: Bootstrap starting.
I20250109 08:57:28.928812 21702 ts_tablet_manager.cc:1428] T 610441ef4b1d40c284eff3a714e4214e P a2b1a692435241ee988e0bd1661cd4c3: Time spent starting tablet: real 0.011s user 0.002s sys 0.005s
I20250109 08:57:28.929991 21703 tablet_bootstrap.cc:654] T 79c1ed8ab551478291f1f24cc940cb27 P 8bf6e26e98b14566899578a990b27d47: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:28.930116 21702 tablet_bootstrap.cc:492] T 71ac0243d72845659291184835652dde P a2b1a692435241ee988e0bd1661cd4c3: Bootstrap starting.
I20250109 08:57:28.936511 21704 tablet_bootstrap.cc:654] T 206785f29c824cddbe9bbc00d365ffb3 P 787ddcc824c2490a808fc5a53452dc5e: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:28.937935 21702 tablet_bootstrap.cc:654] T 71ac0243d72845659291184835652dde P a2b1a692435241ee988e0bd1661cd4c3: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:28.944294 21703 tablet_bootstrap.cc:492] T 79c1ed8ab551478291f1f24cc940cb27 P 8bf6e26e98b14566899578a990b27d47: No bootstrap required, opened a new log
I20250109 08:57:28.944778 21703 ts_tablet_manager.cc:1397] T 79c1ed8ab551478291f1f24cc940cb27 P 8bf6e26e98b14566899578a990b27d47: Time spent bootstrapping tablet: real 0.022s user 0.006s sys 0.015s
I20250109 08:57:28.947161 21703 raft_consensus.cc:357] T 79c1ed8ab551478291f1f24cc940cb27 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:28.947844 21703 raft_consensus.cc:383] T 79c1ed8ab551478291f1f24cc940cb27 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:28.948100 21703 raft_consensus.cc:738] T 79c1ed8ab551478291f1f24cc940cb27 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bf6e26e98b14566899578a990b27d47, State: Initialized, Role: FOLLOWER
I20250109 08:57:28.948786 21703 consensus_queue.cc:260] T 79c1ed8ab551478291f1f24cc940cb27 P 8bf6e26e98b14566899578a990b27d47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:28.951203 21703 ts_tablet_manager.cc:1428] T 79c1ed8ab551478291f1f24cc940cb27 P 8bf6e26e98b14566899578a990b27d47: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20250109 08:57:28.952096 21703 tablet_bootstrap.cc:492] T 363fa693a5114f2bb22f4ab67722a1c6 P 8bf6e26e98b14566899578a990b27d47: Bootstrap starting.
I20250109 08:57:28.957374 21703 tablet_bootstrap.cc:654] T 363fa693a5114f2bb22f4ab67722a1c6 P 8bf6e26e98b14566899578a990b27d47: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:28.958465 21711 raft_consensus.cc:491] T 7a7211b8d4374ad1afd82e8cf85bad81 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:28.959033 21711 raft_consensus.cc:513] T 7a7211b8d4374ad1afd82e8cf85bad81 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:28.963032 21704 tablet_bootstrap.cc:492] T 206785f29c824cddbe9bbc00d365ffb3 P 787ddcc824c2490a808fc5a53452dc5e: No bootstrap required, opened a new log
I20250109 08:57:28.963543 21704 ts_tablet_manager.cc:1397] T 206785f29c824cddbe9bbc00d365ffb3 P 787ddcc824c2490a808fc5a53452dc5e: Time spent bootstrapping tablet: real 0.036s user 0.016s sys 0.013s
I20250109 08:57:28.966578 21732 raft_consensus.cc:491] T 80123146836349938344efc61333695f P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:28.967165 21711 leader_election.cc:290] T 7a7211b8d4374ad1afd82e8cf85bad81 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:28.967069 21732 raft_consensus.cc:513] T 80123146836349938344efc61333695f P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:28.968938 21663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a7211b8d4374ad1afd82e8cf85bad81" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e" is_pre_election: true
I20250109 08:57:28.969223 21732 leader_election.cc:290] T 80123146836349938344efc61333695f P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
I20250109 08:57:28.969667 21663 raft_consensus.cc:2463] T 7a7211b8d4374ad1afd82e8cf85bad81 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8bf6e26e98b14566899578a990b27d47 in term 0.
I20250109 08:57:28.971189 21589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a7211b8d4374ad1afd82e8cf85bad81" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3" is_pre_election: true
I20250109 08:57:28.971545 21588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80123146836349938344efc61333695f" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3" is_pre_election: true
I20250109 08:57:28.971828 21589 raft_consensus.cc:2463] T 7a7211b8d4374ad1afd82e8cf85bad81 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8bf6e26e98b14566899578a990b27d47 in term 0.
I20250109 08:57:28.972340 21514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80123146836349938344efc61333695f" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47" is_pre_election: true
I20250109 08:57:28.971560 21477 leader_election.cc:304] T 7a7211b8d4374ad1afd82e8cf85bad81 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, 8bf6e26e98b14566899578a990b27d47; no voters:
I20250109 08:57:28.972965 21732 raft_consensus.cc:491] T 20ea30dba4314290ae45d9b4c13bebdd P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20250109 08:57:28.973642 21627 leader_election.cc:343] T 80123146836349938344efc61333695f P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:28.973455 21732 raft_consensus.cc:513] T 20ea30dba4314290ae45d9b4c13bebdd P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:28.977134 21513 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20ea30dba4314290ae45d9b4c13bebdd" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47" is_pre_election: true
I20250109 08:57:28.977952 21513 raft_consensus.cc:2463] T 20ea30dba4314290ae45d9b4c13bebdd P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 0.
I20250109 08:57:28.979348 21626 leader_election.cc:304] T 20ea30dba4314290ae45d9b4c13bebdd P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, 8bf6e26e98b14566899578a990b27d47; no voters:
I20250109 08:57:28.980312 21710 raft_consensus.cc:2798] T 20ea30dba4314290ae45d9b4c13bebdd P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:57:28.980787 21710 raft_consensus.cc:491] T 20ea30dba4314290ae45d9b4c13bebdd P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:57:28.981189 21710 raft_consensus.cc:3054] T 20ea30dba4314290ae45d9b4c13bebdd P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:28.980552 21711 raft_consensus.cc:2798] T 7a7211b8d4374ad1afd82e8cf85bad81 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:57:28.982815 21711 raft_consensus.cc:491] T 7a7211b8d4374ad1afd82e8cf85bad81 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:57:28.983182 21711 raft_consensus.cc:3054] T 7a7211b8d4374ad1afd82e8cf85bad81 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:28.982250 21410 catalog_manager.cc:5526] T db34a00c02b84ee988edc7d6e344c1f5 P 8bf6e26e98b14566899578a990b27d47 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8bf6e26e98b14566899578a990b27d47 (127.19.226.193). New cstate: current_term: 1 leader_uuid: "8bf6e26e98b14566899578a990b27d47" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: UNKNOWN } } }
W20250109 08:57:28.985754 21626 leader_election.cc:343] T 80123146836349938344efc61333695f P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:28.986233 21626 leader_election.cc:304] T 80123146836349938344efc61333695f P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e; no voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:57:28.994093 21589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20ea30dba4314290ae45d9b4c13bebdd" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3" is_pre_election: true
I20250109 08:57:28.994843 21589 raft_consensus.cc:2463] T 20ea30dba4314290ae45d9b4c13bebdd P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 0.
I20250109 08:57:28.996356 21732 leader_election.cc:290] T 20ea30dba4314290ae45d9b4c13bebdd P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
I20250109 08:57:28.996735 21709 raft_consensus.cc:491] T c761cb691bd241ae825917a3fbc3ad24 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:28.997349 21732 raft_consensus.cc:2743] T 80123146836349938344efc61333695f P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:57:28.997296 21709 raft_consensus.cc:513] T c761cb691bd241ae825917a3fbc3ad24 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:28.992182 21710 raft_consensus.cc:513] T 20ea30dba4314290ae45d9b4c13bebdd P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:29.003376 21514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c761cb691bd241ae825917a3fbc3ad24" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47" is_pre_election: true
I20250109 08:57:29.001375 21704 raft_consensus.cc:357] T 206785f29c824cddbe9bbc00d365ffb3 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } }
W20250109 08:57:29.004885 21552 leader_election.cc:343] T c761cb691bd241ae825917a3fbc3ad24 P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:29.006251 21709 leader_election.cc:290] T c761cb691bd241ae825917a3fbc3ad24 P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:29.007220 21709 raft_consensus.cc:491] T 610441ef4b1d40c284eff3a714e4214e P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:29.007751 21663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c761cb691bd241ae825917a3fbc3ad24" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e" is_pre_election: true
I20250109 08:57:29.007705 21709 raft_consensus.cc:513] T 610441ef4b1d40c284eff3a714e4214e P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:29.008564 21663 raft_consensus.cc:2463] T c761cb691bd241ae825917a3fbc3ad24 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2b1a692435241ee988e0bd1661cd4c3 in term 0.
I20250109 08:57:29.009785 21552 leader_election.cc:304] T c761cb691bd241ae825917a3fbc3ad24 P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, a2b1a692435241ee988e0bd1661cd4c3; no voters: 8bf6e26e98b14566899578a990b27d47
I20250109 08:57:29.010617 21709 leader_election.cc:290] T 610441ef4b1d40c284eff3a714e4214e P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:29.011221 21709 raft_consensus.cc:2798] T c761cb691bd241ae825917a3fbc3ad24 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:57:29.011593 21709 raft_consensus.cc:491] T c761cb691bd241ae825917a3fbc3ad24 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:57:29.011950 21709 raft_consensus.cc:3054] T c761cb691bd241ae825917a3fbc3ad24 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:29.012044 21663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "610441ef4b1d40c284eff3a714e4214e" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e" is_pre_election: true
I20250109 08:57:29.014128 21514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "610441ef4b1d40c284eff3a714e4214e" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47" is_pre_election: true
I20250109 08:57:29.014732 21704 raft_consensus.cc:383] T 206785f29c824cddbe9bbc00d365ffb3 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:29.015525 21514 raft_consensus.cc:2388] T 610441ef4b1d40c284eff3a714e4214e P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2b1a692435241ee988e0bd1661cd4c3 in current term 1: Already voted for candidate 787ddcc824c2490a808fc5a53452dc5e in this term.
I20250109 08:57:29.016317 21589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20ea30dba4314290ae45d9b4c13bebdd" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3"
I20250109 08:57:29.021214 21589 raft_consensus.cc:3054] T 20ea30dba4314290ae45d9b4c13bebdd P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:29.021556 21552 leader_election.cc:304] T 610441ef4b1d40c284eff3a714e4214e P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2b1a692435241ee988e0bd1661cd4c3; no voters: 787ddcc824c2490a808fc5a53452dc5e, 8bf6e26e98b14566899578a990b27d47
I20250109 08:57:29.024245 21711 raft_consensus.cc:513] T 7a7211b8d4374ad1afd82e8cf85bad81 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:29.027038 21588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a7211b8d4374ad1afd82e8cf85bad81" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3"
I20250109 08:57:29.027642 21588 raft_consensus.cc:3054] T 7a7211b8d4374ad1afd82e8cf85bad81 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:29.029021 21589 raft_consensus.cc:2463] T 20ea30dba4314290ae45d9b4c13bebdd P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 1.
I20250109 08:57:29.030665 21627 leader_election.cc:304] T 20ea30dba4314290ae45d9b4c13bebdd P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, a2b1a692435241ee988e0bd1661cd4c3; no voters:
I20250109 08:57:29.031602 21732 raft_consensus.cc:2798] T 20ea30dba4314290ae45d9b4c13bebdd P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:57:29.032096 21732 raft_consensus.cc:695] T 20ea30dba4314290ae45d9b4c13bebdd P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Becoming Leader. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Running, Role: LEADER
I20250109 08:57:29.033578 21732 consensus_queue.cc:237] T 20ea30dba4314290ae45d9b4c13bebdd P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:29.016865 21710 leader_election.cc:290] T 20ea30dba4314290ae45d9b4c13bebdd P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 election: Requested vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
I20250109 08:57:29.029546 21704 raft_consensus.cc:738] T 206785f29c824cddbe9bbc00d365ffb3 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Initialized, Role: FOLLOWER
I20250109 08:57:29.036932 21588 raft_consensus.cc:2463] T 7a7211b8d4374ad1afd82e8cf85bad81 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8bf6e26e98b14566899578a990b27d47 in term 1.
I20250109 08:57:29.040324 21478 leader_election.cc:304] T 7a7211b8d4374ad1afd82e8cf85bad81 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3; no voters:
I20250109 08:57:29.041507 21740 raft_consensus.cc:2798] T 7a7211b8d4374ad1afd82e8cf85bad81 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:57:29.042009 21740 raft_consensus.cc:695] T 7a7211b8d4374ad1afd82e8cf85bad81 P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Becoming Leader. State: Replica: 8bf6e26e98b14566899578a990b27d47, State: Running, Role: LEADER
I20250109 08:57:29.042814 21740 consensus_queue.cc:237] T 7a7211b8d4374ad1afd82e8cf85bad81 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:29.045581 21709 raft_consensus.cc:513] T c761cb691bd241ae825917a3fbc3ad24 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:29.064868 21711 leader_election.cc:290] T 7a7211b8d4374ad1afd82e8cf85bad81 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 election: Requested vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:29.066833 21514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c761cb691bd241ae825917a3fbc3ad24" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47"
I20250109 08:57:29.065670 21709 leader_election.cc:290] T c761cb691bd241ae825917a3fbc3ad24 P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 election: Requested vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:29.072016 21709 raft_consensus.cc:3054] T 610441ef4b1d40c284eff3a714e4214e P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:29.068791 21663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a7211b8d4374ad1afd82e8cf85bad81" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e"
I20250109 08:57:29.076848 21663 raft_consensus.cc:3054] T 7a7211b8d4374ad1afd82e8cf85bad81 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:29.070727 21662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c761cb691bd241ae825917a3fbc3ad24" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e"
I20250109 08:57:29.080862 21662 raft_consensus.cc:3054] T c761cb691bd241ae825917a3fbc3ad24 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Advancing to term 1
W20250109 08:57:29.071419 21552 leader_election.cc:343] T c761cb691bd241ae825917a3fbc3ad24 P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Illegal state: must be running to vote when last-logged opid is not known
W20250109 08:57:29.087719 21626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.226.193:42725, user_credentials={real_user=slave}} blocked reactor thread for 102001us
I20250109 08:57:29.089766 21514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20ea30dba4314290ae45d9b4c13bebdd" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47"
I20250109 08:57:29.090444 21514 raft_consensus.cc:3054] T 20ea30dba4314290ae45d9b4c13bebdd P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:29.039775 21704 consensus_queue.cc:260] T 206785f29c824cddbe9bbc00d365ffb3 P 787ddcc824c2490a808fc5a53452dc5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } }
I20250109 08:57:29.101455 21412 catalog_manager.cc:5526] T 20ea30dba4314290ae45d9b4c13bebdd P 787ddcc824c2490a808fc5a53452dc5e reported cstate change: term changed from 0 to 1, leader changed from <none> to 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195). New cstate: current_term: 1 leader_uuid: "787ddcc824c2490a808fc5a53452dc5e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: HEALTHY } } }
I20250109 08:57:29.103122 21709 raft_consensus.cc:2743] T 610441ef4b1d40c284eff3a714e4214e P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:57:29.099340 21662 raft_consensus.cc:2463] T c761cb691bd241ae825917a3fbc3ad24 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2b1a692435241ee988e0bd1661cd4c3 in term 1.
I20250109 08:57:29.107759 21552 leader_election.cc:304] T c761cb691bd241ae825917a3fbc3ad24 P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, a2b1a692435241ee988e0bd1661cd4c3; no voters: 8bf6e26e98b14566899578a990b27d47
I20250109 08:57:29.110123 21663 raft_consensus.cc:2463] T 7a7211b8d4374ad1afd82e8cf85bad81 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8bf6e26e98b14566899578a990b27d47 in term 1.
I20250109 08:57:29.110666 21514 raft_consensus.cc:2463] T 20ea30dba4314290ae45d9b4c13bebdd P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 1.
I20250109 08:57:29.110951 21709 raft_consensus.cc:2798] T c761cb691bd241ae825917a3fbc3ad24 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:57:29.113919 21711 raft_consensus.cc:491] T 37280454805d47f98a415a8f0c16d869 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:29.114558 21704 ts_tablet_manager.cc:1428] T 206785f29c824cddbe9bbc00d365ffb3 P 787ddcc824c2490a808fc5a53452dc5e: Time spent starting tablet: real 0.151s user 0.007s sys 0.003s
I20250109 08:57:29.114390 21711 raft_consensus.cc:513] T 37280454805d47f98a415a8f0c16d869 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:29.113339 21709 raft_consensus.cc:695] T c761cb691bd241ae825917a3fbc3ad24 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 LEADER]: Becoming Leader. State: Replica: a2b1a692435241ee988e0bd1661cd4c3, State: Running, Role: LEADER
I20250109 08:57:29.116819 21709 consensus_queue.cc:237] T c761cb691bd241ae825917a3fbc3ad24 P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:29.117458 21703 tablet_bootstrap.cc:492] T 363fa693a5114f2bb22f4ab67722a1c6 P 8bf6e26e98b14566899578a990b27d47: No bootstrap required, opened a new log
I20250109 08:57:29.117964 21703 ts_tablet_manager.cc:1397] T 363fa693a5114f2bb22f4ab67722a1c6 P 8bf6e26e98b14566899578a990b27d47: Time spent bootstrapping tablet: real 0.166s user 0.067s sys 0.076s
I20250109 08:57:29.118528 21704 tablet_bootstrap.cc:492] T 97304b06c7fa40ef8345a72c796c47f8 P 787ddcc824c2490a808fc5a53452dc5e: Bootstrap starting.
I20250109 08:57:29.120481 21711 leader_election.cc:290] T 37280454805d47f98a415a8f0c16d869 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:29.121027 21703 raft_consensus.cc:357] T 363fa693a5114f2bb22f4ab67722a1c6 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:29.121876 21703 raft_consensus.cc:383] T 363fa693a5114f2bb22f4ab67722a1c6 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:29.122215 21703 raft_consensus.cc:738] T 363fa693a5114f2bb22f4ab67722a1c6 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bf6e26e98b14566899578a990b27d47, State: Initialized, Role: FOLLOWER
I20250109 08:57:29.122550 21588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37280454805d47f98a415a8f0c16d869" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3" is_pre_election: true
I20250109 08:57:29.123225 21588 raft_consensus.cc:2463] T 37280454805d47f98a415a8f0c16d869 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8bf6e26e98b14566899578a990b27d47 in term 0.
I20250109 08:57:29.124457 21478 leader_election.cc:304] T 37280454805d47f98a415a8f0c16d869 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3; no voters:
I20250109 08:57:29.125567 21704 tablet_bootstrap.cc:654] T 97304b06c7fa40ef8345a72c796c47f8 P 787ddcc824c2490a808fc5a53452dc5e: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:29.122915 21703 consensus_queue.cc:260] T 363fa693a5114f2bb22f4ab67722a1c6 P 8bf6e26e98b14566899578a990b27d47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:29.127144 21740 raft_consensus.cc:2798] T 37280454805d47f98a415a8f0c16d869 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:57:29.127689 21740 raft_consensus.cc:491] T 37280454805d47f98a415a8f0c16d869 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:57:29.128103 21740 raft_consensus.cc:3054] T 37280454805d47f98a415a8f0c16d869 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:29.128099 21663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37280454805d47f98a415a8f0c16d869" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e" is_pre_election: true
W20250109 08:57:29.130321 21477 leader_election.cc:343] T 37280454805d47f98a415a8f0c16d869 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:29.130929 21703 ts_tablet_manager.cc:1428] T 363fa693a5114f2bb22f4ab67722a1c6 P 8bf6e26e98b14566899578a990b27d47: Time spent starting tablet: real 0.013s user 0.006s sys 0.000s
I20250109 08:57:29.136488 21703 tablet_bootstrap.cc:492] T 80123146836349938344efc61333695f P 8bf6e26e98b14566899578a990b27d47: Bootstrap starting.
I20250109 08:57:29.137506 21702 tablet_bootstrap.cc:492] T 71ac0243d72845659291184835652dde P a2b1a692435241ee988e0bd1661cd4c3: No bootstrap required, opened a new log
I20250109 08:57:29.137990 21702 ts_tablet_manager.cc:1397] T 71ac0243d72845659291184835652dde P a2b1a692435241ee988e0bd1661cd4c3: Time spent bootstrapping tablet: real 0.208s user 0.050s sys 0.120s
I20250109 08:57:29.142732 21703 tablet_bootstrap.cc:654] T 80123146836349938344efc61333695f P 8bf6e26e98b14566899578a990b27d47: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:29.149519 21412 catalog_manager.cc:5526] T 7a7211b8d4374ad1afd82e8cf85bad81 P 8bf6e26e98b14566899578a990b27d47 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8bf6e26e98b14566899578a990b27d47 (127.19.226.193). New cstate: current_term: 1 leader_uuid: "8bf6e26e98b14566899578a990b27d47" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: UNKNOWN } } }
I20250109 08:57:29.155421 21702 raft_consensus.cc:357] T 71ac0243d72845659291184835652dde P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:29.156190 21702 raft_consensus.cc:383] T 71ac0243d72845659291184835652dde P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:29.156476 21702 raft_consensus.cc:738] T 71ac0243d72845659291184835652dde P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2b1a692435241ee988e0bd1661cd4c3, State: Initialized, Role: FOLLOWER
I20250109 08:57:29.157207 21740 raft_consensus.cc:513] T 37280454805d47f98a415a8f0c16d869 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:29.159387 21702 consensus_queue.cc:260] T 71ac0243d72845659291184835652dde P a2b1a692435241ee988e0bd1661cd4c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:29.160334 21410 catalog_manager.cc:5526] T c761cb691bd241ae825917a3fbc3ad24 P a2b1a692435241ee988e0bd1661cd4c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194). New cstate: current_term: 1 leader_uuid: "a2b1a692435241ee988e0bd1661cd4c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: UNKNOWN } } }
I20250109 08:57:29.161579 21702 ts_tablet_manager.cc:1428] T 71ac0243d72845659291184835652dde P a2b1a692435241ee988e0bd1661cd4c3: Time spent starting tablet: real 0.023s user 0.007s sys 0.000s
I20250109 08:57:29.162449 21702 tablet_bootstrap.cc:492] T 97304b06c7fa40ef8345a72c796c47f8 P a2b1a692435241ee988e0bd1661cd4c3: Bootstrap starting.
I20250109 08:57:29.167829 21588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37280454805d47f98a415a8f0c16d869" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3"
I20250109 08:57:29.168437 21588 raft_consensus.cc:3054] T 37280454805d47f98a415a8f0c16d869 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:29.168393 21740 leader_election.cc:290] T 37280454805d47f98a415a8f0c16d869 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 election: Requested vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:29.171048 21702 tablet_bootstrap.cc:654] T 97304b06c7fa40ef8345a72c796c47f8 P a2b1a692435241ee988e0bd1661cd4c3: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:29.170909 21663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37280454805d47f98a415a8f0c16d869" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e"
I20250109 08:57:29.171097 21704 tablet_bootstrap.cc:492] T 97304b06c7fa40ef8345a72c796c47f8 P 787ddcc824c2490a808fc5a53452dc5e: No bootstrap required, opened a new log
I20250109 08:57:29.171649 21704 ts_tablet_manager.cc:1397] T 97304b06c7fa40ef8345a72c796c47f8 P 787ddcc824c2490a808fc5a53452dc5e: Time spent bootstrapping tablet: real 0.053s user 0.015s sys 0.031s
W20250109 08:57:29.172242 21477 leader_election.cc:343] T 37280454805d47f98a415a8f0c16d869 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:29.174511 21704 raft_consensus.cc:357] T 97304b06c7fa40ef8345a72c796c47f8 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:29.175179 21704 raft_consensus.cc:383] T 97304b06c7fa40ef8345a72c796c47f8 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:29.175444 21704 raft_consensus.cc:738] T 97304b06c7fa40ef8345a72c796c47f8 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Initialized, Role: FOLLOWER
I20250109 08:57:29.176178 21704 consensus_queue.cc:260] T 97304b06c7fa40ef8345a72c796c47f8 P 787ddcc824c2490a808fc5a53452dc5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:29.182564 21588 raft_consensus.cc:2463] T 37280454805d47f98a415a8f0c16d869 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8bf6e26e98b14566899578a990b27d47 in term 1.
I20250109 08:57:29.184293 21704 ts_tablet_manager.cc:1428] T 97304b06c7fa40ef8345a72c796c47f8 P 787ddcc824c2490a808fc5a53452dc5e: Time spent starting tablet: real 0.012s user 0.003s sys 0.003s
I20250109 08:57:29.184646 21478 leader_election.cc:304] T 37280454805d47f98a415a8f0c16d869 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3; no voters: 787ddcc824c2490a808fc5a53452dc5e
I20250109 08:57:29.185227 21704 tablet_bootstrap.cc:492] T 79c1ed8ab551478291f1f24cc940cb27 P 787ddcc824c2490a808fc5a53452dc5e: Bootstrap starting.
I20250109 08:57:29.185844 21740 raft_consensus.cc:2798] T 37280454805d47f98a415a8f0c16d869 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:57:29.186357 21740 raft_consensus.cc:695] T 37280454805d47f98a415a8f0c16d869 P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Becoming Leader. State: Replica: 8bf6e26e98b14566899578a990b27d47, State: Running, Role: LEADER
I20250109 08:57:29.187197 21740 consensus_queue.cc:237] T 37280454805d47f98a415a8f0c16d869 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:29.191947 21711 raft_consensus.cc:491] T 363fa693a5114f2bb22f4ab67722a1c6 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:29.192489 21711 raft_consensus.cc:513] T 363fa693a5114f2bb22f4ab67722a1c6 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:29.194494 21711 leader_election.cc:290] T 363fa693a5114f2bb22f4ab67722a1c6 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:29.196041 21703 tablet_bootstrap.cc:492] T 80123146836349938344efc61333695f P 8bf6e26e98b14566899578a990b27d47: No bootstrap required, opened a new log
I20250109 08:57:29.196516 21703 ts_tablet_manager.cc:1397] T 80123146836349938344efc61333695f P 8bf6e26e98b14566899578a990b27d47: Time spent bootstrapping tablet: real 0.060s user 0.015s sys 0.002s
I20250109 08:57:29.196897 21702 tablet_bootstrap.cc:492] T 97304b06c7fa40ef8345a72c796c47f8 P a2b1a692435241ee988e0bd1661cd4c3: No bootstrap required, opened a new log
I20250109 08:57:29.196938 21663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "363fa693a5114f2bb22f4ab67722a1c6" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e" is_pre_election: true
I20250109 08:57:29.197360 21702 ts_tablet_manager.cc:1397] T 97304b06c7fa40ef8345a72c796c47f8 P a2b1a692435241ee988e0bd1661cd4c3: Time spent bootstrapping tablet: real 0.035s user 0.016s sys 0.009s
W20250109 08:57:29.198390 21477 leader_election.cc:343] T 363fa693a5114f2bb22f4ab67722a1c6 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:29.199599 21702 raft_consensus.cc:357] T 97304b06c7fa40ef8345a72c796c47f8 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:29.200263 21702 raft_consensus.cc:383] T 97304b06c7fa40ef8345a72c796c47f8 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:29.200609 21702 raft_consensus.cc:738] T 97304b06c7fa40ef8345a72c796c47f8 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2b1a692435241ee988e0bd1661cd4c3, State: Initialized, Role: FOLLOWER
I20250109 08:57:29.200187 21703 raft_consensus.cc:357] T 80123146836349938344efc61333695f P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:29.201231 21703 raft_consensus.cc:383] T 80123146836349938344efc61333695f P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:29.201462 21703 raft_consensus.cc:738] T 80123146836349938344efc61333695f P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bf6e26e98b14566899578a990b27d47, State: Initialized, Role: FOLLOWER
I20250109 08:57:29.201522 21702 consensus_queue.cc:260] T 97304b06c7fa40ef8345a72c796c47f8 P a2b1a692435241ee988e0bd1661cd4c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:29.201911 21703 consensus_queue.cc:260] T 80123146836349938344efc61333695f P 8bf6e26e98b14566899578a990b27d47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:29.202811 21588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "363fa693a5114f2bb22f4ab67722a1c6" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3" is_pre_election: true
I20250109 08:57:29.204528 21704 tablet_bootstrap.cc:654] T 79c1ed8ab551478291f1f24cc940cb27 P 787ddcc824c2490a808fc5a53452dc5e: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:29.205158 21702 ts_tablet_manager.cc:1428] T 97304b06c7fa40ef8345a72c796c47f8 P a2b1a692435241ee988e0bd1661cd4c3: Time spent starting tablet: real 0.008s user 0.005s sys 0.001s
I20250109 08:57:29.206055 21702 tablet_bootstrap.cc:492] T 79c1ed8ab551478291f1f24cc940cb27 P a2b1a692435241ee988e0bd1661cd4c3: Bootstrap starting.
W20250109 08:57:29.206645 21478 leader_election.cc:343] T 363fa693a5114f2bb22f4ab67722a1c6 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:29.207062 21478 leader_election.cc:304] T 363fa693a5114f2bb22f4ab67722a1c6 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8bf6e26e98b14566899578a990b27d47; no voters: 787ddcc824c2490a808fc5a53452dc5e, a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:57:29.209302 21740 raft_consensus.cc:2743] T 363fa693a5114f2bb22f4ab67722a1c6 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:57:29.209892 21703 ts_tablet_manager.cc:1428] T 80123146836349938344efc61333695f P 8bf6e26e98b14566899578a990b27d47: Time spent starting tablet: real 0.013s user 0.006s sys 0.000s
I20250109 08:57:29.210450 21758 raft_consensus.cc:491] T 206785f29c824cddbe9bbc00d365ffb3 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:29.210906 21703 tablet_bootstrap.cc:492] T e601a034e10e418bac8683443d86a1c1 P 8bf6e26e98b14566899578a990b27d47: Bootstrap starting.
I20250109 08:57:29.210897 21758 raft_consensus.cc:513] T 206785f29c824cddbe9bbc00d365ffb3 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } }
I20250109 08:57:29.211431 21702 tablet_bootstrap.cc:654] T 79c1ed8ab551478291f1f24cc940cb27 P a2b1a692435241ee988e0bd1661cd4c3: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:29.212956 21758 leader_election.cc:290] T 206785f29c824cddbe9bbc00d365ffb3 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725)
I20250109 08:57:29.214828 21588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "206785f29c824cddbe9bbc00d365ffb3" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3" is_pre_election: true
I20250109 08:57:29.215319 21514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "206785f29c824cddbe9bbc00d365ffb3" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47" is_pre_election: true
I20250109 08:57:29.216920 21703 tablet_bootstrap.cc:654] T e601a034e10e418bac8683443d86a1c1 P 8bf6e26e98b14566899578a990b27d47: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:29.215587 21588 raft_consensus.cc:2388] T 206785f29c824cddbe9bbc00d365ffb3 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 787ddcc824c2490a808fc5a53452dc5e in current term 1: Already voted for candidate 8bf6e26e98b14566899578a990b27d47 in this term.
I20250109 08:57:29.217790 21704 tablet_bootstrap.cc:492] T 79c1ed8ab551478291f1f24cc940cb27 P 787ddcc824c2490a808fc5a53452dc5e: No bootstrap required, opened a new log
I20250109 08:57:29.218205 21704 ts_tablet_manager.cc:1397] T 79c1ed8ab551478291f1f24cc940cb27 P 787ddcc824c2490a808fc5a53452dc5e: Time spent bootstrapping tablet: real 0.033s user 0.002s sys 0.009s
I20250109 08:57:29.218434 21627 leader_election.cc:304] T 206785f29c824cddbe9bbc00d365ffb3 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e; no voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:57:29.219146 21758 raft_consensus.cc:3054] T 206785f29c824cddbe9bbc00d365ffb3 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:29.226117 21758 raft_consensus.cc:2743] T 206785f29c824cddbe9bbc00d365ffb3 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:57:29.228138 21703 tablet_bootstrap.cc:492] T e601a034e10e418bac8683443d86a1c1 P 8bf6e26e98b14566899578a990b27d47: No bootstrap required, opened a new log
I20250109 08:57:29.228715 21703 ts_tablet_manager.cc:1397] T e601a034e10e418bac8683443d86a1c1 P 8bf6e26e98b14566899578a990b27d47: Time spent bootstrapping tablet: real 0.018s user 0.013s sys 0.004s
I20250109 08:57:29.228242 21704 raft_consensus.cc:357] T 79c1ed8ab551478291f1f24cc940cb27 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:29.229110 21704 raft_consensus.cc:383] T 79c1ed8ab551478291f1f24cc940cb27 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:29.229372 21704 raft_consensus.cc:738] T 79c1ed8ab551478291f1f24cc940cb27 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Initialized, Role: FOLLOWER
I20250109 08:57:29.230053 21704 consensus_queue.cc:260] T 79c1ed8ab551478291f1f24cc940cb27 P 787ddcc824c2490a808fc5a53452dc5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:29.231159 21703 raft_consensus.cc:357] T e601a034e10e418bac8683443d86a1c1 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:29.231822 21703 raft_consensus.cc:383] T e601a034e10e418bac8683443d86a1c1 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:29.232129 21703 raft_consensus.cc:738] T e601a034e10e418bac8683443d86a1c1 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bf6e26e98b14566899578a990b27d47, State: Initialized, Role: FOLLOWER
I20250109 08:57:29.232801 21703 consensus_queue.cc:260] T e601a034e10e418bac8683443d86a1c1 P 8bf6e26e98b14566899578a990b27d47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:29.233518 21411 catalog_manager.cc:5526] T 37280454805d47f98a415a8f0c16d869 P 8bf6e26e98b14566899578a990b27d47 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8bf6e26e98b14566899578a990b27d47 (127.19.226.193). New cstate: current_term: 1 leader_uuid: "8bf6e26e98b14566899578a990b27d47" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: UNKNOWN } } }
I20250109 08:57:29.235365 21703 ts_tablet_manager.cc:1428] T e601a034e10e418bac8683443d86a1c1 P 8bf6e26e98b14566899578a990b27d47: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20250109 08:57:29.236236 21703 tablet_bootstrap.cc:492] T 81d74e7d4472497aac5463a4b47e4fb9 P 8bf6e26e98b14566899578a990b27d47: Bootstrap starting.
I20250109 08:57:29.239506 21704 ts_tablet_manager.cc:1428] T 79c1ed8ab551478291f1f24cc940cb27 P 787ddcc824c2490a808fc5a53452dc5e: Time spent starting tablet: real 0.021s user 0.006s sys 0.001s
I20250109 08:57:29.240409 21704 tablet_bootstrap.cc:492] T 37280454805d47f98a415a8f0c16d869 P 787ddcc824c2490a808fc5a53452dc5e: Bootstrap starting.
I20250109 08:57:29.241497 21703 tablet_bootstrap.cc:654] T 81d74e7d4472497aac5463a4b47e4fb9 P 8bf6e26e98b14566899578a990b27d47: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:29.248196 21704 tablet_bootstrap.cc:654] T 37280454805d47f98a415a8f0c16d869 P 787ddcc824c2490a808fc5a53452dc5e: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:29.249426 21703 tablet_bootstrap.cc:492] T 81d74e7d4472497aac5463a4b47e4fb9 P 8bf6e26e98b14566899578a990b27d47: No bootstrap required, opened a new log
I20250109 08:57:29.249877 21703 ts_tablet_manager.cc:1397] T 81d74e7d4472497aac5463a4b47e4fb9 P 8bf6e26e98b14566899578a990b27d47: Time spent bootstrapping tablet: real 0.014s user 0.010s sys 0.000s
I20250109 08:57:29.251619 21703 raft_consensus.cc:357] T 81d74e7d4472497aac5463a4b47e4fb9 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:29.252094 21703 raft_consensus.cc:383] T 81d74e7d4472497aac5463a4b47e4fb9 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:29.252245 21703 raft_consensus.cc:738] T 81d74e7d4472497aac5463a4b47e4fb9 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bf6e26e98b14566899578a990b27d47, State: Initialized, Role: FOLLOWER
I20250109 08:57:29.253397 21702 tablet_bootstrap.cc:492] T 79c1ed8ab551478291f1f24cc940cb27 P a2b1a692435241ee988e0bd1661cd4c3: No bootstrap required, opened a new log
I20250109 08:57:29.253172 21703 consensus_queue.cc:260] T 81d74e7d4472497aac5463a4b47e4fb9 P 8bf6e26e98b14566899578a990b27d47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:29.253878 21702 ts_tablet_manager.cc:1397] T 79c1ed8ab551478291f1f24cc940cb27 P a2b1a692435241ee988e0bd1661cd4c3: Time spent bootstrapping tablet: real 0.048s user 0.009s sys 0.015s
I20250109 08:57:29.255203 21703 ts_tablet_manager.cc:1428] T 81d74e7d4472497aac5463a4b47e4fb9 P 8bf6e26e98b14566899578a990b27d47: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20250109 08:57:29.256029 21703 tablet_bootstrap.cc:492] T c761cb691bd241ae825917a3fbc3ad24 P 8bf6e26e98b14566899578a990b27d47: Bootstrap starting.
I20250109 08:57:29.256470 21702 raft_consensus.cc:357] T 79c1ed8ab551478291f1f24cc940cb27 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:29.257129 21702 raft_consensus.cc:383] T 79c1ed8ab551478291f1f24cc940cb27 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:29.257385 21702 raft_consensus.cc:738] T 79c1ed8ab551478291f1f24cc940cb27 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2b1a692435241ee988e0bd1661cd4c3, State: Initialized, Role: FOLLOWER
I20250109 08:57:29.257951 21702 consensus_queue.cc:260] T 79c1ed8ab551478291f1f24cc940cb27 P a2b1a692435241ee988e0bd1661cd4c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:29.258862 21704 tablet_bootstrap.cc:492] T 37280454805d47f98a415a8f0c16d869 P 787ddcc824c2490a808fc5a53452dc5e: No bootstrap required, opened a new log
I20250109 08:57:29.259284 21704 ts_tablet_manager.cc:1397] T 37280454805d47f98a415a8f0c16d869 P 787ddcc824c2490a808fc5a53452dc5e: Time spent bootstrapping tablet: real 0.019s user 0.007s sys 0.007s
I20250109 08:57:29.259418 21740 raft_consensus.cc:491] T 79c1ed8ab551478291f1f24cc940cb27 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:29.259840 21740 raft_consensus.cc:513] T 79c1ed8ab551478291f1f24cc940cb27 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:29.261629 21704 raft_consensus.cc:357] T 37280454805d47f98a415a8f0c16d869 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:29.261690 21740 leader_election.cc:290] T 79c1ed8ab551478291f1f24cc940cb27 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:29.262811 21663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79c1ed8ab551478291f1f24cc940cb27" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e" is_pre_election: true
I20250109 08:57:29.263036 21703 tablet_bootstrap.cc:654] T c761cb691bd241ae825917a3fbc3ad24 P 8bf6e26e98b14566899578a990b27d47: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:29.263422 21663 raft_consensus.cc:2463] T 79c1ed8ab551478291f1f24cc940cb27 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8bf6e26e98b14566899578a990b27d47 in term 0.
I20250109 08:57:29.264137 21702 ts_tablet_manager.cc:1428] T 79c1ed8ab551478291f1f24cc940cb27 P a2b1a692435241ee988e0bd1661cd4c3: Time spent starting tablet: real 0.010s user 0.004s sys 0.000s
I20250109 08:57:29.265166 21477 leader_election.cc:304] T 79c1ed8ab551478291f1f24cc940cb27 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, 8bf6e26e98b14566899578a990b27d47; no voters:
I20250109 08:57:29.265440 21702 tablet_bootstrap.cc:492] T 80123146836349938344efc61333695f P a2b1a692435241ee988e0bd1661cd4c3: Bootstrap starting.
I20250109 08:57:29.266038 21740 raft_consensus.cc:2798] T 79c1ed8ab551478291f1f24cc940cb27 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:57:29.266376 21740 raft_consensus.cc:491] T 79c1ed8ab551478291f1f24cc940cb27 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:57:29.266726 21740 raft_consensus.cc:3054] T 79c1ed8ab551478291f1f24cc940cb27 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:29.271313 21702 tablet_bootstrap.cc:654] T 80123146836349938344efc61333695f P a2b1a692435241ee988e0bd1661cd4c3: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:29.275842 21711 consensus_queue.cc:1035] T 206785f29c824cddbe9bbc00d365ffb3 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:57:29.282742 21740 raft_consensus.cc:513] T 79c1ed8ab551478291f1f24cc940cb27 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:29.285961 21758 consensus_queue.cc:1035] T 610441ef4b1d40c284eff3a714e4214e P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250109 08:57:29.262266 21704 raft_consensus.cc:383] T 37280454805d47f98a415a8f0c16d869 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:29.290230 21704 raft_consensus.cc:738] T 37280454805d47f98a415a8f0c16d869 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Initialized, Role: FOLLOWER
I20250109 08:57:29.290915 21704 consensus_queue.cc:260] T 37280454805d47f98a415a8f0c16d869 P 787ddcc824c2490a808fc5a53452dc5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:29.262439 21588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79c1ed8ab551478291f1f24cc940cb27" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3" is_pre_election: true
I20250109 08:57:29.292104 21588 raft_consensus.cc:2463] T 79c1ed8ab551478291f1f24cc940cb27 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8bf6e26e98b14566899578a990b27d47 in term 0.
I20250109 08:57:29.292650 21740 leader_election.cc:290] T 79c1ed8ab551478291f1f24cc940cb27 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 election: Requested vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:29.294541 21704 ts_tablet_manager.cc:1428] T 37280454805d47f98a415a8f0c16d869 P 787ddcc824c2490a808fc5a53452dc5e: Time spent starting tablet: real 0.035s user 0.005s sys 0.001s
I20250109 08:57:29.293373 21662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79c1ed8ab551478291f1f24cc940cb27" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e"
I20250109 08:57:29.294682 21588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79c1ed8ab551478291f1f24cc940cb27" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3"
I20250109 08:57:29.295030 21662 raft_consensus.cc:3054] T 79c1ed8ab551478291f1f24cc940cb27 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:29.295437 21704 tablet_bootstrap.cc:492] T 363fa693a5114f2bb22f4ab67722a1c6 P 787ddcc824c2490a808fc5a53452dc5e: Bootstrap starting.
I20250109 08:57:29.295221 21588 raft_consensus.cc:3054] T 79c1ed8ab551478291f1f24cc940cb27 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:29.301419 21704 tablet_bootstrap.cc:654] T 363fa693a5114f2bb22f4ab67722a1c6 P 787ddcc824c2490a808fc5a53452dc5e: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:29.303187 21662 raft_consensus.cc:2463] T 79c1ed8ab551478291f1f24cc940cb27 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8bf6e26e98b14566899578a990b27d47 in term 1.
I20250109 08:57:29.304461 21477 leader_election.cc:304] T 79c1ed8ab551478291f1f24cc940cb27 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, 8bf6e26e98b14566899578a990b27d47; no voters:
I20250109 08:57:29.305186 21740 raft_consensus.cc:2798] T 79c1ed8ab551478291f1f24cc940cb27 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:57:29.305559 21740 raft_consensus.cc:695] T 79c1ed8ab551478291f1f24cc940cb27 P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Becoming Leader. State: Replica: 8bf6e26e98b14566899578a990b27d47, State: Running, Role: LEADER
I20250109 08:57:29.306557 21703 tablet_bootstrap.cc:492] T c761cb691bd241ae825917a3fbc3ad24 P 8bf6e26e98b14566899578a990b27d47: No bootstrap required, opened a new log
I20250109 08:57:29.306236 21740 consensus_queue.cc:237] T 79c1ed8ab551478291f1f24cc940cb27 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:29.307036 21703 ts_tablet_manager.cc:1397] T c761cb691bd241ae825917a3fbc3ad24 P 8bf6e26e98b14566899578a990b27d47: Time spent bootstrapping tablet: real 0.051s user 0.027s sys 0.022s
I20250109 08:57:29.306249 21731 raft_consensus.cc:491] T 81d74e7d4472497aac5463a4b47e4fb9 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:29.308513 21731 raft_consensus.cc:513] T 81d74e7d4472497aac5463a4b47e4fb9 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:29.309516 21703 raft_consensus.cc:357] T c761cb691bd241ae825917a3fbc3ad24 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:29.310182 21703 raft_consensus.cc:383] T c761cb691bd241ae825917a3fbc3ad24 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:29.310425 21703 raft_consensus.cc:738] T c761cb691bd241ae825917a3fbc3ad24 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bf6e26e98b14566899578a990b27d47, State: Initialized, Role: FOLLOWER
I20250109 08:57:29.310513 21731 leader_election.cc:290] T 81d74e7d4472497aac5463a4b47e4fb9 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:29.311252 21703 consensus_queue.cc:260] T c761cb691bd241ae825917a3fbc3ad24 P 8bf6e26e98b14566899578a990b27d47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:29.312748 21587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81d74e7d4472497aac5463a4b47e4fb9" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3" is_pre_election: true
I20250109 08:57:29.313328 21703 ts_tablet_manager.cc:1428] T c761cb691bd241ae825917a3fbc3ad24 P 8bf6e26e98b14566899578a990b27d47: Time spent starting tablet: real 0.006s user 0.006s sys 0.001s
I20250109 08:57:29.314175 21703 tablet_bootstrap.cc:492] T 306c4892bbff4dbfbe5a0645294aa627 P 8bf6e26e98b14566899578a990b27d47: Bootstrap starting.
W20250109 08:57:29.314661 21478 leader_election.cc:343] T 81d74e7d4472497aac5463a4b47e4fb9 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:29.318862 21662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81d74e7d4472497aac5463a4b47e4fb9" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e" is_pre_election: true
I20250109 08:57:29.319500 21703 tablet_bootstrap.cc:654] T 306c4892bbff4dbfbe5a0645294aa627 P 8bf6e26e98b14566899578a990b27d47: Neither blocks nor log segments found. Creating new log.
W20250109 08:57:29.332129 21477 leader_election.cc:343] T 81d74e7d4472497aac5463a4b47e4fb9 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:29.332712 21477 leader_election.cc:304] T 81d74e7d4472497aac5463a4b47e4fb9 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8bf6e26e98b14566899578a990b27d47; no voters: 787ddcc824c2490a808fc5a53452dc5e, a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:57:29.333925 21731 raft_consensus.cc:2743] T 81d74e7d4472497aac5463a4b47e4fb9 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:57:29.306391 21588 raft_consensus.cc:2463] T 79c1ed8ab551478291f1f24cc940cb27 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8bf6e26e98b14566899578a990b27d47 in term 1.
I20250109 08:57:29.329348 21758 raft_consensus.cc:491] T 37280454805d47f98a415a8f0c16d869 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:29.335220 21758 raft_consensus.cc:513] T 37280454805d47f98a415a8f0c16d869 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:29.337538 21758 leader_election.cc:290] T 37280454805d47f98a415a8f0c16d869 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
I20250109 08:57:29.339331 21514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37280454805d47f98a415a8f0c16d869" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47" is_pre_election: true
I20250109 08:57:29.345598 21781 consensus_queue.cc:1035] T db34a00c02b84ee988edc7d6e344c1f5 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250109 08:57:29.347467 21711 consensus_queue.cc:1035] T 206785f29c824cddbe9bbc00d365ffb3 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:57:29.351321 21585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37280454805d47f98a415a8f0c16d869" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3" is_pre_election: true
I20250109 08:57:29.352146 21585 raft_consensus.cc:2388] T 37280454805d47f98a415a8f0c16d869 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 787ddcc824c2490a808fc5a53452dc5e in current term 1: Already voted for candidate 8bf6e26e98b14566899578a990b27d47 in this term.
I20250109 08:57:29.353511 21627 leader_election.cc:304] T 37280454805d47f98a415a8f0c16d869 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e; no voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:57:29.354491 21758 raft_consensus.cc:3054] T 37280454805d47f98a415a8f0c16d869 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:29.361697 21409 catalog_manager.cc:5526] T 79c1ed8ab551478291f1f24cc940cb27 P 8bf6e26e98b14566899578a990b27d47 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8bf6e26e98b14566899578a990b27d47 (127.19.226.193). New cstate: current_term: 1 leader_uuid: "8bf6e26e98b14566899578a990b27d47" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: UNKNOWN } } }
I20250109 08:57:29.368649 21709 raft_consensus.cc:491] T 97304b06c7fa40ef8345a72c796c47f8 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:29.369287 21709 raft_consensus.cc:513] T 97304b06c7fa40ef8345a72c796c47f8 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:29.371302 21709 leader_election.cc:290] T 97304b06c7fa40ef8345a72c796c47f8 P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:29.373165 21514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97304b06c7fa40ef8345a72c796c47f8" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47" is_pre_election: true
I20250109 08:57:29.373890 21514 raft_consensus.cc:2463] T 97304b06c7fa40ef8345a72c796c47f8 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2b1a692435241ee988e0bd1661cd4c3 in term 0.
I20250109 08:57:29.374956 21552 leader_election.cc:304] T 97304b06c7fa40ef8345a72c796c47f8 P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3; no voters:
I20250109 08:57:29.376132 21709 raft_consensus.cc:2798] T 97304b06c7fa40ef8345a72c796c47f8 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:57:29.376506 21709 raft_consensus.cc:491] T 97304b06c7fa40ef8345a72c796c47f8 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:57:29.376839 21709 raft_consensus.cc:3054] T 97304b06c7fa40ef8345a72c796c47f8 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Advancing to term 1
W20250109 08:57:29.383004 21477 consensus_peers.cc:487] T db34a00c02b84ee988edc7d6e344c1f5 P 8bf6e26e98b14566899578a990b27d47 -> Peer 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731): Couldn't send request to peer 787ddcc824c2490a808fc5a53452dc5e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:57:29.387096 21663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97304b06c7fa40ef8345a72c796c47f8" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e" is_pre_election: true
I20250109 08:57:29.391919 21663 raft_consensus.cc:2463] T 97304b06c7fa40ef8345a72c796c47f8 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2b1a692435241ee988e0bd1661cd4c3 in term 0.
I20250109 08:57:29.393801 21732 consensus_queue.cc:1035] T 610441ef4b1d40c284eff3a714e4214e P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:57:29.386111 21709 raft_consensus.cc:513] T 97304b06c7fa40ef8345a72c796c47f8 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:29.390244 21758 raft_consensus.cc:2743] T 37280454805d47f98a415a8f0c16d869 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:57:29.406004 21781 raft_consensus.cc:491] T e601a034e10e418bac8683443d86a1c1 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:29.406570 21781 raft_consensus.cc:513] T e601a034e10e418bac8683443d86a1c1 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:29.407714 21709 leader_election.cc:290] T 97304b06c7fa40ef8345a72c796c47f8 P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 election: Requested vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:29.408691 21663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97304b06c7fa40ef8345a72c796c47f8" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e"
I20250109 08:57:29.409384 21663 raft_consensus.cc:3054] T 97304b06c7fa40ef8345a72c796c47f8 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:29.409627 21781 leader_election.cc:290] T e601a034e10e418bac8683443d86a1c1 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:29.410966 21662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e601a034e10e418bac8683443d86a1c1" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e" is_pre_election: true
I20250109 08:57:29.411732 21662 raft_consensus.cc:2463] T e601a034e10e418bac8683443d86a1c1 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8bf6e26e98b14566899578a990b27d47 in term 0.
I20250109 08:57:29.411840 21588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e601a034e10e418bac8683443d86a1c1" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3" is_pre_election: true
I20250109 08:57:29.412910 21477 leader_election.cc:304] T e601a034e10e418bac8683443d86a1c1 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, 8bf6e26e98b14566899578a990b27d47; no voters:
W20250109 08:57:29.413481 21478 leader_election.cc:343] T e601a034e10e418bac8683443d86a1c1 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:29.410969 21514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97304b06c7fa40ef8345a72c796c47f8" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47"
I20250109 08:57:29.414255 21514 raft_consensus.cc:3054] T 97304b06c7fa40ef8345a72c796c47f8 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:29.415792 21702 tablet_bootstrap.cc:492] T 80123146836349938344efc61333695f P a2b1a692435241ee988e0bd1661cd4c3: No bootstrap required, opened a new log
I20250109 08:57:29.416146 21702 ts_tablet_manager.cc:1397] T 80123146836349938344efc61333695f P a2b1a692435241ee988e0bd1661cd4c3: Time spent bootstrapping tablet: real 0.151s user 0.016s sys 0.010s
I20250109 08:57:29.417716 21703 tablet_bootstrap.cc:492] T 306c4892bbff4dbfbe5a0645294aa627 P 8bf6e26e98b14566899578a990b27d47: No bootstrap required, opened a new log
I20250109 08:57:29.418138 21703 ts_tablet_manager.cc:1397] T 306c4892bbff4dbfbe5a0645294aa627 P 8bf6e26e98b14566899578a990b27d47: Time spent bootstrapping tablet: real 0.104s user 0.029s sys 0.047s
I20250109 08:57:29.418480 21702 raft_consensus.cc:357] T 80123146836349938344efc61333695f P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:29.419934 21711 raft_consensus.cc:2798] T e601a034e10e418bac8683443d86a1c1 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:57:29.420320 21711 raft_consensus.cc:491] T e601a034e10e418bac8683443d86a1c1 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:57:29.420663 21711 raft_consensus.cc:3054] T e601a034e10e418bac8683443d86a1c1 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:29.420907 21703 raft_consensus.cc:357] T 306c4892bbff4dbfbe5a0645294aa627 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:29.421550 21703 raft_consensus.cc:383] T 306c4892bbff4dbfbe5a0645294aa627 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:29.421823 21703 raft_consensus.cc:738] T 306c4892bbff4dbfbe5a0645294aa627 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bf6e26e98b14566899578a990b27d47, State: Initialized, Role: FOLLOWER
I20250109 08:57:29.422540 21704 tablet_bootstrap.cc:492] T 363fa693a5114f2bb22f4ab67722a1c6 P 787ddcc824c2490a808fc5a53452dc5e: No bootstrap required, opened a new log
I20250109 08:57:29.425295 21663 raft_consensus.cc:2463] T 97304b06c7fa40ef8345a72c796c47f8 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2b1a692435241ee988e0bd1661cd4c3 in term 1.
I20250109 08:57:29.425995 21704 ts_tablet_manager.cc:1397] T 363fa693a5114f2bb22f4ab67722a1c6 P 787ddcc824c2490a808fc5a53452dc5e: Time spent bootstrapping tablet: real 0.131s user 0.027s sys 0.031s
I20250109 08:57:29.426558 21552 leader_election.cc:304] T 97304b06c7fa40ef8345a72c796c47f8 P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, a2b1a692435241ee988e0bd1661cd4c3; no voters:
I20250109 08:57:29.427392 21709 raft_consensus.cc:2798] T 97304b06c7fa40ef8345a72c796c47f8 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:57:29.425295 21514 raft_consensus.cc:2463] T 97304b06c7fa40ef8345a72c796c47f8 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2b1a692435241ee988e0bd1661cd4c3 in term 1.
I20250109 08:57:29.427847 21709 raft_consensus.cc:695] T 97304b06c7fa40ef8345a72c796c47f8 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 LEADER]: Becoming Leader. State: Replica: a2b1a692435241ee988e0bd1661cd4c3, State: Running, Role: LEADER
I20250109 08:57:29.422487 21703 consensus_queue.cc:260] T 306c4892bbff4dbfbe5a0645294aa627 P 8bf6e26e98b14566899578a990b27d47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:29.428617 21704 raft_consensus.cc:357] T 363fa693a5114f2bb22f4ab67722a1c6 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:29.429303 21704 raft_consensus.cc:383] T 363fa693a5114f2bb22f4ab67722a1c6 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:29.429574 21704 raft_consensus.cc:738] T 363fa693a5114f2bb22f4ab67722a1c6 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Initialized, Role: FOLLOWER
I20250109 08:57:29.426555 21711 raft_consensus.cc:513] T e601a034e10e418bac8683443d86a1c1 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:29.430217 21703 ts_tablet_manager.cc:1428] T 306c4892bbff4dbfbe5a0645294aa627 P 8bf6e26e98b14566899578a990b27d47: Time spent starting tablet: real 0.012s user 0.006s sys 0.002s
I20250109 08:57:29.430191 21704 consensus_queue.cc:260] T 363fa693a5114f2bb22f4ab67722a1c6 P 787ddcc824c2490a808fc5a53452dc5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:29.431394 21703 tablet_bootstrap.cc:492] T 3b5049c00b0b4dd89c49fa5d13db169e P 8bf6e26e98b14566899578a990b27d47: Bootstrap starting.
I20250109 08:57:29.419147 21702 raft_consensus.cc:383] T 80123146836349938344efc61333695f P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:29.432329 21704 ts_tablet_manager.cc:1428] T 363fa693a5114f2bb22f4ab67722a1c6 P 787ddcc824c2490a808fc5a53452dc5e: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20250109 08:57:29.440634 21702 raft_consensus.cc:738] T 80123146836349938344efc61333695f P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2b1a692435241ee988e0bd1661cd4c3, State: Initialized, Role: FOLLOWER
I20250109 08:57:29.428704 21709 consensus_queue.cc:237] T 97304b06c7fa40ef8345a72c796c47f8 P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:29.441366 21702 consensus_queue.cc:260] T 80123146836349938344efc61333695f P a2b1a692435241ee988e0bd1661cd4c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:29.444111 21702 ts_tablet_manager.cc:1428] T 80123146836349938344efc61333695f P a2b1a692435241ee988e0bd1661cd4c3: Time spent starting tablet: real 0.028s user 0.009s sys 0.001s
I20250109 08:57:29.444994 21702 tablet_bootstrap.cc:492] T 363fa693a5114f2bb22f4ab67722a1c6 P a2b1a692435241ee988e0bd1661cd4c3: Bootstrap starting.
I20250109 08:57:29.446034 21586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e601a034e10e418bac8683443d86a1c1" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3"
W20250109 08:57:29.447269 21478 leader_election.cc:343] T e601a034e10e418bac8683443d86a1c1 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:29.447674 21703 tablet_bootstrap.cc:654] T 3b5049c00b0b4dd89c49fa5d13db169e P 8bf6e26e98b14566899578a990b27d47: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:29.448262 21711 leader_election.cc:290] T e601a034e10e418bac8683443d86a1c1 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 election: Requested vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:29.450557 21704 tablet_bootstrap.cc:492] T db34a00c02b84ee988edc7d6e344c1f5 P 787ddcc824c2490a808fc5a53452dc5e: Bootstrap starting.
I20250109 08:57:29.458180 21704 tablet_bootstrap.cc:654] T db34a00c02b84ee988edc7d6e344c1f5 P 787ddcc824c2490a808fc5a53452dc5e: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:29.451188 21663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e601a034e10e418bac8683443d86a1c1" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e"
I20250109 08:57:29.461414 21663 raft_consensus.cc:3054] T e601a034e10e418bac8683443d86a1c1 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:29.467175 21411 catalog_manager.cc:5526] T 97304b06c7fa40ef8345a72c796c47f8 P a2b1a692435241ee988e0bd1661cd4c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194). New cstate: current_term: 1 leader_uuid: "a2b1a692435241ee988e0bd1661cd4c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: UNKNOWN } } }
I20250109 08:57:29.471172 21663 raft_consensus.cc:2463] T e601a034e10e418bac8683443d86a1c1 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8bf6e26e98b14566899578a990b27d47 in term 1.
I20250109 08:57:29.472276 21477 leader_election.cc:304] T e601a034e10e418bac8683443d86a1c1 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, 8bf6e26e98b14566899578a990b27d47; no voters: a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:57:29.473948 21711 raft_consensus.cc:2798] T e601a034e10e418bac8683443d86a1c1 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:57:29.474375 21711 raft_consensus.cc:695] T e601a034e10e418bac8683443d86a1c1 P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Becoming Leader. State: Replica: 8bf6e26e98b14566899578a990b27d47, State: Running, Role: LEADER
I20250109 08:57:29.475121 21711 consensus_queue.cc:237] T e601a034e10e418bac8683443d86a1c1 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:29.483225 21704 tablet_bootstrap.cc:492] T db34a00c02b84ee988edc7d6e344c1f5 P 787ddcc824c2490a808fc5a53452dc5e: No bootstrap required, opened a new log
I20250109 08:57:29.485177 21704 ts_tablet_manager.cc:1397] T db34a00c02b84ee988edc7d6e344c1f5 P 787ddcc824c2490a808fc5a53452dc5e: Time spent bootstrapping tablet: real 0.035s user 0.011s sys 0.000s
I20250109 08:57:29.489822 21704 raft_consensus.cc:357] T db34a00c02b84ee988edc7d6e344c1f5 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:29.490576 21704 raft_consensus.cc:383] T db34a00c02b84ee988edc7d6e344c1f5 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:29.490904 21704 raft_consensus.cc:738] T db34a00c02b84ee988edc7d6e344c1f5 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Initialized, Role: FOLLOWER
I20250109 08:57:29.491902 21704 consensus_queue.cc:260] T db34a00c02b84ee988edc7d6e344c1f5 P 787ddcc824c2490a808fc5a53452dc5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:29.494702 21704 ts_tablet_manager.cc:1428] T db34a00c02b84ee988edc7d6e344c1f5 P 787ddcc824c2490a808fc5a53452dc5e: Time spent starting tablet: real 0.008s user 0.005s sys 0.000s
I20250109 08:57:29.495568 21704 tablet_bootstrap.cc:492] T 81d74e7d4472497aac5463a4b47e4fb9 P 787ddcc824c2490a808fc5a53452dc5e: Bootstrap starting.
I20250109 08:57:29.501296 21704 tablet_bootstrap.cc:654] T 81d74e7d4472497aac5463a4b47e4fb9 P 787ddcc824c2490a808fc5a53452dc5e: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:29.506904 21702 tablet_bootstrap.cc:654] T 363fa693a5114f2bb22f4ab67722a1c6 P a2b1a692435241ee988e0bd1661cd4c3: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:29.508392 21411 catalog_manager.cc:5526] T e601a034e10e418bac8683443d86a1c1 P 8bf6e26e98b14566899578a990b27d47 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8bf6e26e98b14566899578a990b27d47 (127.19.226.193). New cstate: current_term: 1 leader_uuid: "8bf6e26e98b14566899578a990b27d47" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: UNKNOWN } } }
I20250109 08:57:29.511920 21703 tablet_bootstrap.cc:492] T 3b5049c00b0b4dd89c49fa5d13db169e P 8bf6e26e98b14566899578a990b27d47: No bootstrap required, opened a new log
I20250109 08:57:29.512285 21703 ts_tablet_manager.cc:1397] T 3b5049c00b0b4dd89c49fa5d13db169e P 8bf6e26e98b14566899578a990b27d47: Time spent bootstrapping tablet: real 0.081s user 0.036s sys 0.040s
I20250109 08:57:29.514595 21703 raft_consensus.cc:357] T 3b5049c00b0b4dd89c49fa5d13db169e P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:29.515266 21703 raft_consensus.cc:383] T 3b5049c00b0b4dd89c49fa5d13db169e P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:29.515528 21703 raft_consensus.cc:738] T 3b5049c00b0b4dd89c49fa5d13db169e P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bf6e26e98b14566899578a990b27d47, State: Initialized, Role: FOLLOWER
I20250109 08:57:29.516090 21703 consensus_queue.cc:260] T 3b5049c00b0b4dd89c49fa5d13db169e P 8bf6e26e98b14566899578a990b27d47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:29.518229 21703 ts_tablet_manager.cc:1428] T 3b5049c00b0b4dd89c49fa5d13db169e P 8bf6e26e98b14566899578a990b27d47: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20250109 08:57:29.519109 21703 tablet_bootstrap.cc:492] T 71ac0243d72845659291184835652dde P 8bf6e26e98b14566899578a990b27d47: Bootstrap starting.
I20250109 08:57:29.521272 21702 tablet_bootstrap.cc:492] T 363fa693a5114f2bb22f4ab67722a1c6 P a2b1a692435241ee988e0bd1661cd4c3: No bootstrap required, opened a new log
I20250109 08:57:29.521683 21702 ts_tablet_manager.cc:1397] T 363fa693a5114f2bb22f4ab67722a1c6 P a2b1a692435241ee988e0bd1661cd4c3: Time spent bootstrapping tablet: real 0.077s user 0.010s sys 0.006s
I20250109 08:57:29.522327 21704 tablet_bootstrap.cc:492] T 81d74e7d4472497aac5463a4b47e4fb9 P 787ddcc824c2490a808fc5a53452dc5e: No bootstrap required, opened a new log
I20250109 08:57:29.522724 21704 ts_tablet_manager.cc:1397] T 81d74e7d4472497aac5463a4b47e4fb9 P 787ddcc824c2490a808fc5a53452dc5e: Time spent bootstrapping tablet: real 0.027s user 0.008s sys 0.015s
I20250109 08:57:29.524044 21702 raft_consensus.cc:357] T 363fa693a5114f2bb22f4ab67722a1c6 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:29.524498 21703 tablet_bootstrap.cc:654] T 71ac0243d72845659291184835652dde P 8bf6e26e98b14566899578a990b27d47: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:29.524760 21702 raft_consensus.cc:383] T 363fa693a5114f2bb22f4ab67722a1c6 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:29.525144 21702 raft_consensus.cc:738] T 363fa693a5114f2bb22f4ab67722a1c6 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2b1a692435241ee988e0bd1661cd4c3, State: Initialized, Role: FOLLOWER
I20250109 08:57:29.524963 21704 raft_consensus.cc:357] T 81d74e7d4472497aac5463a4b47e4fb9 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:29.525704 21704 raft_consensus.cc:383] T 81d74e7d4472497aac5463a4b47e4fb9 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:29.525995 21704 raft_consensus.cc:738] T 81d74e7d4472497aac5463a4b47e4fb9 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Initialized, Role: FOLLOWER
I20250109 08:57:29.525784 21702 consensus_queue.cc:260] T 363fa693a5114f2bb22f4ab67722a1c6 P a2b1a692435241ee988e0bd1661cd4c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:29.526669 21704 consensus_queue.cc:260] T 81d74e7d4472497aac5463a4b47e4fb9 P 787ddcc824c2490a808fc5a53452dc5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:29.527614 21709 raft_consensus.cc:491] T 71ac0243d72845659291184835652dde P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:29.527799 21702 ts_tablet_manager.cc:1428] T 363fa693a5114f2bb22f4ab67722a1c6 P a2b1a692435241ee988e0bd1661cd4c3: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20250109 08:57:29.528088 21709 raft_consensus.cc:513] T 71ac0243d72845659291184835652dde P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:29.529917 21702 tablet_bootstrap.cc:492] T e601a034e10e418bac8683443d86a1c1 P a2b1a692435241ee988e0bd1661cd4c3: Bootstrap starting.
I20250109 08:57:29.530076 21709 leader_election.cc:290] T 71ac0243d72845659291184835652dde P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:29.532790 21663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71ac0243d72845659291184835652dde" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e" is_pre_election: true
I20250109 08:57:29.534379 21704 ts_tablet_manager.cc:1428] T 81d74e7d4472497aac5463a4b47e4fb9 P 787ddcc824c2490a808fc5a53452dc5e: Time spent starting tablet: real 0.011s user 0.005s sys 0.001s
I20250109 08:57:29.534765 21513 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71ac0243d72845659291184835652dde" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47" is_pre_election: true
W20250109 08:57:29.535399 21552 leader_election.cc:343] T 71ac0243d72845659291184835652dde P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:29.535996 21703 tablet_bootstrap.cc:492] T 71ac0243d72845659291184835652dde P 8bf6e26e98b14566899578a990b27d47: No bootstrap required, opened a new log
I20250109 08:57:29.536399 21703 ts_tablet_manager.cc:1397] T 71ac0243d72845659291184835652dde P 8bf6e26e98b14566899578a990b27d47: Time spent bootstrapping tablet: real 0.017s user 0.011s sys 0.004s
I20250109 08:57:29.537253 21704 tablet_bootstrap.cc:492] T 180b4d539a144a38905bf340ecb24ef4 P 787ddcc824c2490a808fc5a53452dc5e: Bootstrap starting.
W20250109 08:57:29.538841 21552 leader_election.cc:343] T 71ac0243d72845659291184835652dde P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:29.539256 21552 leader_election.cc:304] T 71ac0243d72845659291184835652dde P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2b1a692435241ee988e0bd1661cd4c3; no voters: 787ddcc824c2490a808fc5a53452dc5e, 8bf6e26e98b14566899578a990b27d47
I20250109 08:57:29.540310 21758 consensus_queue.cc:1035] T 20ea30dba4314290ae45d9b4c13bebdd P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:57:29.539292 21703 raft_consensus.cc:357] T 71ac0243d72845659291184835652dde P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:29.541200 21703 raft_consensus.cc:383] T 71ac0243d72845659291184835652dde P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:29.541467 21703 raft_consensus.cc:738] T 71ac0243d72845659291184835652dde P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bf6e26e98b14566899578a990b27d47, State: Initialized, Role: FOLLOWER
I20250109 08:57:29.542104 21703 consensus_queue.cc:260] T 71ac0243d72845659291184835652dde P 8bf6e26e98b14566899578a990b27d47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:29.543375 21704 tablet_bootstrap.cc:654] T 180b4d539a144a38905bf340ecb24ef4 P 787ddcc824c2490a808fc5a53452dc5e: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:29.544265 21703 ts_tablet_manager.cc:1428] T 71ac0243d72845659291184835652dde P 8bf6e26e98b14566899578a990b27d47: Time spent starting tablet: real 0.008s user 0.004s sys 0.001s
I20250109 08:57:29.546104 21703 tablet_bootstrap.cc:492] T 180b4d539a144a38905bf340ecb24ef4 P 8bf6e26e98b14566899578a990b27d47: Bootstrap starting.
I20250109 08:57:29.546319 21759 raft_consensus.cc:2743] T 71ac0243d72845659291184835652dde P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:57:29.550105 21711 consensus_queue.cc:1035] T 7a7211b8d4374ad1afd82e8cf85bad81 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:57:29.551358 21742 raft_consensus.cc:491] T 80123146836349938344efc61333695f P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:29.551900 21731 consensus_queue.cc:1035] T 7a7211b8d4374ad1afd82e8cf85bad81 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250109 08:57:29.551796 21742 raft_consensus.cc:513] T 80123146836349938344efc61333695f P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:29.565660 21662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80123146836349938344efc61333695f" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e" is_pre_election: true
I20250109 08:57:29.566397 21662 raft_consensus.cc:2463] T 80123146836349938344efc61333695f P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8bf6e26e98b14566899578a990b27d47 in term 0.
I20250109 08:57:29.566941 21731 raft_consensus.cc:491] T c761cb691bd241ae825917a3fbc3ad24 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:29.560210 21742 leader_election.cc:290] T 80123146836349938344efc61333695f P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:29.567538 21477 leader_election.cc:304] T 80123146836349938344efc61333695f P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, 8bf6e26e98b14566899578a990b27d47; no voters:
I20250109 08:57:29.567394 21731 raft_consensus.cc:513] T c761cb691bd241ae825917a3fbc3ad24 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:29.571509 21587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80123146836349938344efc61333695f" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3" is_pre_election: true
I20250109 08:57:29.571774 21588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c761cb691bd241ae825917a3fbc3ad24" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3" is_pre_election: true
I20250109 08:57:29.573236 21742 raft_consensus.cc:2798] T 80123146836349938344efc61333695f P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:57:29.570318 21731 leader_election.cc:290] T c761cb691bd241ae825917a3fbc3ad24 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:29.573642 21742 raft_consensus.cc:491] T 80123146836349938344efc61333695f P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:57:29.574463 21742 raft_consensus.cc:3054] T 80123146836349938344efc61333695f P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:29.572196 21587 raft_consensus.cc:2463] T 80123146836349938344efc61333695f P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8bf6e26e98b14566899578a990b27d47 in term 0.
I20250109 08:57:29.580875 21758 consensus_queue.cc:1035] T 20ea30dba4314290ae45d9b4c13bebdd P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:57:29.581362 21662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c761cb691bd241ae825917a3fbc3ad24" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e" is_pre_election: true
I20250109 08:57:29.582062 21662 raft_consensus.cc:2388] T c761cb691bd241ae825917a3fbc3ad24 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8bf6e26e98b14566899578a990b27d47 in current term 1: Already voted for candidate a2b1a692435241ee988e0bd1661cd4c3 in this term.
I20250109 08:57:29.587361 21709 consensus_queue.cc:1035] T c761cb691bd241ae825917a3fbc3ad24 P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:57:29.590975 21702 tablet_bootstrap.cc:654] T e601a034e10e418bac8683443d86a1c1 P a2b1a692435241ee988e0bd1661cd4c3: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:29.590576 21477 leader_election.cc:304] T c761cb691bd241ae825917a3fbc3ad24 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8bf6e26e98b14566899578a990b27d47; no voters: 787ddcc824c2490a808fc5a53452dc5e, a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:57:29.593469 21731 raft_consensus.cc:3054] T c761cb691bd241ae825917a3fbc3ad24 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:29.623628 21742 raft_consensus.cc:513] T 80123146836349938344efc61333695f P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:29.633893 21731 raft_consensus.cc:2743] T c761cb691bd241ae825917a3fbc3ad24 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:57:29.638532 21703 tablet_bootstrap.cc:654] T 180b4d539a144a38905bf340ecb24ef4 P 8bf6e26e98b14566899578a990b27d47: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:29.642299 21589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80123146836349938344efc61333695f" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3"
I20250109 08:57:29.642925 21589 raft_consensus.cc:3054] T 80123146836349938344efc61333695f P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:29.664036 21759 consensus_queue.cc:1035] T c761cb691bd241ae825917a3fbc3ad24 P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250109 08:57:29.681149 21589 raft_consensus.cc:2463] T 80123146836349938344efc61333695f P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8bf6e26e98b14566899578a990b27d47 in term 1.
I20250109 08:57:29.669873 21742 leader_election.cc:290] T 80123146836349938344efc61333695f P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 election: Requested vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:29.682317 21478 leader_election.cc:304] T 80123146836349938344efc61333695f P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3; no voters:
I20250109 08:57:29.684340 21731 raft_consensus.cc:2798] T 80123146836349938344efc61333695f P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:57:29.684764 21662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80123146836349938344efc61333695f" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e"
I20250109 08:57:29.684873 21731 raft_consensus.cc:695] T 80123146836349938344efc61333695f P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Becoming Leader. State: Replica: 8bf6e26e98b14566899578a990b27d47, State: Running, Role: LEADER
I20250109 08:57:29.685374 21662 raft_consensus.cc:3054] T 80123146836349938344efc61333695f P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:29.686079 21731 consensus_queue.cc:237] T 80123146836349938344efc61333695f P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:29.691401 21702 tablet_bootstrap.cc:492] T e601a034e10e418bac8683443d86a1c1 P a2b1a692435241ee988e0bd1661cd4c3: No bootstrap required, opened a new log
I20250109 08:57:29.691877 21702 ts_tablet_manager.cc:1397] T e601a034e10e418bac8683443d86a1c1 P a2b1a692435241ee988e0bd1661cd4c3: Time spent bootstrapping tablet: real 0.162s user 0.014s sys 0.043s
I20250109 08:57:29.694872 21702 raft_consensus.cc:357] T e601a034e10e418bac8683443d86a1c1 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:29.695937 21702 raft_consensus.cc:383] T e601a034e10e418bac8683443d86a1c1 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:29.696231 21702 raft_consensus.cc:738] T e601a034e10e418bac8683443d86a1c1 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2b1a692435241ee988e0bd1661cd4c3, State: Initialized, Role: FOLLOWER
I20250109 08:57:29.699905 21662 raft_consensus.cc:2463] T 80123146836349938344efc61333695f P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8bf6e26e98b14566899578a990b27d47 in term 1.
I20250109 08:57:29.701797 21704 tablet_bootstrap.cc:492] T 180b4d539a144a38905bf340ecb24ef4 P 787ddcc824c2490a808fc5a53452dc5e: No bootstrap required, opened a new log
I20250109 08:57:29.702282 21704 ts_tablet_manager.cc:1397] T 180b4d539a144a38905bf340ecb24ef4 P 787ddcc824c2490a808fc5a53452dc5e: Time spent bootstrapping tablet: real 0.165s user 0.023s sys 0.031s
I20250109 08:57:29.708489 21702 consensus_queue.cc:260] T e601a034e10e418bac8683443d86a1c1 P a2b1a692435241ee988e0bd1661cd4c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:29.713707 21702 ts_tablet_manager.cc:1428] T e601a034e10e418bac8683443d86a1c1 P a2b1a692435241ee988e0bd1661cd4c3: Time spent starting tablet: real 0.022s user 0.006s sys 0.001s
I20250109 08:57:29.716993 21702 tablet_bootstrap.cc:492] T 0e06021cd6624e50af2c914399ebd6c4 P a2b1a692435241ee988e0bd1661cd4c3: Bootstrap starting.
I20250109 08:57:29.721452 21703 tablet_bootstrap.cc:492] T 180b4d539a144a38905bf340ecb24ef4 P 8bf6e26e98b14566899578a990b27d47: No bootstrap required, opened a new log
I20250109 08:57:29.721959 21703 ts_tablet_manager.cc:1397] T 180b4d539a144a38905bf340ecb24ef4 P 8bf6e26e98b14566899578a990b27d47: Time spent bootstrapping tablet: real 0.176s user 0.028s sys 0.045s
I20250109 08:57:29.723868 21702 tablet_bootstrap.cc:654] T 0e06021cd6624e50af2c914399ebd6c4 P a2b1a692435241ee988e0bd1661cd4c3: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:29.724324 21703 raft_consensus.cc:357] T 180b4d539a144a38905bf340ecb24ef4 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:29.725103 21703 raft_consensus.cc:383] T 180b4d539a144a38905bf340ecb24ef4 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:29.725366 21703 raft_consensus.cc:738] T 180b4d539a144a38905bf340ecb24ef4 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bf6e26e98b14566899578a990b27d47, State: Initialized, Role: FOLLOWER
I20250109 08:57:29.725953 21703 consensus_queue.cc:260] T 180b4d539a144a38905bf340ecb24ef4 P 8bf6e26e98b14566899578a990b27d47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:29.728518 21704 raft_consensus.cc:357] T 180b4d539a144a38905bf340ecb24ef4 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:29.729226 21704 raft_consensus.cc:383] T 180b4d539a144a38905bf340ecb24ef4 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:29.729472 21704 raft_consensus.cc:738] T 180b4d539a144a38905bf340ecb24ef4 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Initialized, Role: FOLLOWER
I20250109 08:57:29.730069 21702 tablet_bootstrap.cc:492] T 0e06021cd6624e50af2c914399ebd6c4 P a2b1a692435241ee988e0bd1661cd4c3: No bootstrap required, opened a new log
I20250109 08:57:29.730572 21702 ts_tablet_manager.cc:1397] T 0e06021cd6624e50af2c914399ebd6c4 P a2b1a692435241ee988e0bd1661cd4c3: Time spent bootstrapping tablet: real 0.014s user 0.007s sys 0.004s
I20250109 08:57:29.731269 21742 raft_consensus.cc:491] T 71ac0243d72845659291184835652dde P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:29.731724 21742 raft_consensus.cc:513] T 71ac0243d72845659291184835652dde P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:29.732281 21703 ts_tablet_manager.cc:1428] T 180b4d539a144a38905bf340ecb24ef4 P 8bf6e26e98b14566899578a990b27d47: Time spent starting tablet: real 0.010s user 0.000s sys 0.006s
I20250109 08:57:29.733173 21703 tablet_bootstrap.cc:492] T 5ac5ffe3cd65463fb448e819e3978841 P 8bf6e26e98b14566899578a990b27d47: Bootstrap starting.
I20250109 08:57:29.732759 21409 catalog_manager.cc:5526] T 80123146836349938344efc61333695f P 8bf6e26e98b14566899578a990b27d47 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8bf6e26e98b14566899578a990b27d47 (127.19.226.193). New cstate: current_term: 1 leader_uuid: "8bf6e26e98b14566899578a990b27d47" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: UNKNOWN } } }
I20250109 08:57:29.733610 21742 leader_election.cc:290] T 71ac0243d72845659291184835652dde P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:29.733472 21702 raft_consensus.cc:357] T 0e06021cd6624e50af2c914399ebd6c4 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:29.734139 21702 raft_consensus.cc:383] T 0e06021cd6624e50af2c914399ebd6c4 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:29.734390 21702 raft_consensus.cc:738] T 0e06021cd6624e50af2c914399ebd6c4 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2b1a692435241ee988e0bd1661cd4c3, State: Initialized, Role: FOLLOWER
I20250109 08:57:29.734947 21702 consensus_queue.cc:260] T 0e06021cd6624e50af2c914399ebd6c4 P a2b1a692435241ee988e0bd1661cd4c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:29.730050 21704 consensus_queue.cc:260] T 180b4d539a144a38905bf340ecb24ef4 P 787ddcc824c2490a808fc5a53452dc5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:29.737134 21704 ts_tablet_manager.cc:1428] T 180b4d539a144a38905bf340ecb24ef4 P 787ddcc824c2490a808fc5a53452dc5e: Time spent starting tablet: real 0.035s user 0.005s sys 0.001s
I20250109 08:57:29.738617 21589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71ac0243d72845659291184835652dde" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3" is_pre_election: true
I20250109 08:57:29.739271 21703 tablet_bootstrap.cc:654] T 5ac5ffe3cd65463fb448e819e3978841 P 8bf6e26e98b14566899578a990b27d47: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:29.739235 21589 raft_consensus.cc:2463] T 71ac0243d72845659291184835652dde P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8bf6e26e98b14566899578a990b27d47 in term 0.
I20250109 08:57:29.739375 21742 consensus_queue.cc:1035] T 37280454805d47f98a415a8f0c16d869 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250109 08:57:29.739221 21704 tablet_bootstrap.cc:492] T 5ac5ffe3cd65463fb448e819e3978841 P 787ddcc824c2490a808fc5a53452dc5e: Bootstrap starting.
I20250109 08:57:29.740988 21662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71ac0243d72845659291184835652dde" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e" is_pre_election: true
I20250109 08:57:29.743780 21702 ts_tablet_manager.cc:1428] T 0e06021cd6624e50af2c914399ebd6c4 P a2b1a692435241ee988e0bd1661cd4c3: Time spent starting tablet: real 0.013s user 0.005s sys 0.000s
I20250109 08:57:29.744666 21702 tablet_bootstrap.cc:492] T d8493108cba04514bd2236192cef1079 P a2b1a692435241ee988e0bd1661cd4c3: Bootstrap starting.
I20250109 08:57:29.744572 21478 leader_election.cc:304] T 71ac0243d72845659291184835652dde P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3; no voters:
I20250109 08:57:29.745496 21742 raft_consensus.cc:2798] T 71ac0243d72845659291184835652dde P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:57:29.745864 21742 raft_consensus.cc:491] T 71ac0243d72845659291184835652dde P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:57:29.746212 21742 raft_consensus.cc:3054] T 71ac0243d72845659291184835652dde P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Advancing to term 1
W20250109 08:57:29.746277 21477 leader_election.cc:343] T 71ac0243d72845659291184835652dde P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:29.747048 21704 tablet_bootstrap.cc:654] T 5ac5ffe3cd65463fb448e819e3978841 P 787ddcc824c2490a808fc5a53452dc5e: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:29.751169 21702 tablet_bootstrap.cc:654] T d8493108cba04514bd2236192cef1079 P a2b1a692435241ee988e0bd1661cd4c3: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:29.755280 21742 raft_consensus.cc:513] T 71ac0243d72845659291184835652dde P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:29.756079 21759 raft_consensus.cc:491] T 363fa693a5114f2bb22f4ab67722a1c6 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:29.756628 21759 raft_consensus.cc:513] T 363fa693a5114f2bb22f4ab67722a1c6 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:29.759807 21663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71ac0243d72845659291184835652dde" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e"
I20250109 08:57:29.760063 21589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71ac0243d72845659291184835652dde" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3"
I20250109 08:57:29.760670 21589 raft_consensus.cc:3054] T 71ac0243d72845659291184835652dde P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:29.761711 21514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "363fa693a5114f2bb22f4ab67722a1c6" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47" is_pre_election: true
I20250109 08:57:29.762413 21514 raft_consensus.cc:2463] T 363fa693a5114f2bb22f4ab67722a1c6 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2b1a692435241ee988e0bd1661cd4c3 in term 0.
I20250109 08:57:29.763559 21552 leader_election.cc:304] T 363fa693a5114f2bb22f4ab67722a1c6 P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3; no voters:
W20250109 08:57:29.765117 21477 leader_election.cc:343] T 71ac0243d72845659291184835652dde P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:29.765713 21731 raft_consensus.cc:491] T 3b5049c00b0b4dd89c49fa5d13db169e P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:29.766407 21709 raft_consensus.cc:2798] T 363fa693a5114f2bb22f4ab67722a1c6 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:57:29.766815 21709 raft_consensus.cc:491] T 363fa693a5114f2bb22f4ab67722a1c6 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:57:29.767045 21759 leader_election.cc:290] T 363fa693a5114f2bb22f4ab67722a1c6 P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:29.767138 21709 raft_consensus.cc:3054] T 363fa693a5114f2bb22f4ab67722a1c6 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:29.757586 21742 leader_election.cc:290] T 71ac0243d72845659291184835652dde P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 election: Requested vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:29.771085 21663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "363fa693a5114f2bb22f4ab67722a1c6" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e" is_pre_election: true
I20250109 08:57:29.771687 21663 raft_consensus.cc:2463] T 363fa693a5114f2bb22f4ab67722a1c6 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2b1a692435241ee988e0bd1661cd4c3 in term 0.
I20250109 08:57:29.766161 21731 raft_consensus.cc:513] T 3b5049c00b0b4dd89c49fa5d13db169e P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:29.772089 21742 consensus_queue.cc:1035] T 79c1ed8ab551478291f1f24cc940cb27 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:57:29.773842 21731 leader_election.cc:290] T 3b5049c00b0b4dd89c49fa5d13db169e P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:29.775377 21586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b5049c00b0b4dd89c49fa5d13db169e" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3" is_pre_election: true
I20250109 08:57:29.775348 21661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b5049c00b0b4dd89c49fa5d13db169e" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e" is_pre_election: true
W20250109 08:57:29.777210 21478 leader_election.cc:343] T 3b5049c00b0b4dd89c49fa5d13db169e P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Illegal state: must be running to vote when last-logged opid is not known
W20250109 08:57:29.778044 21477 leader_election.cc:343] T 3b5049c00b0b4dd89c49fa5d13db169e P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:29.778462 21477 leader_election.cc:304] T 3b5049c00b0b4dd89c49fa5d13db169e P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8bf6e26e98b14566899578a990b27d47; no voters: 787ddcc824c2490a808fc5a53452dc5e, a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:57:29.779560 21731 raft_consensus.cc:2743] T 3b5049c00b0b4dd89c49fa5d13db169e P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:57:29.780683 21709 raft_consensus.cc:513] T 363fa693a5114f2bb22f4ab67722a1c6 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:29.782316 21742 consensus_queue.cc:1035] T 37280454805d47f98a415a8f0c16d869 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:57:29.783242 21514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "363fa693a5114f2bb22f4ab67722a1c6" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47"
I20250109 08:57:29.783787 21514 raft_consensus.cc:3054] T 363fa693a5114f2bb22f4ab67722a1c6 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:29.784053 21704 tablet_bootstrap.cc:492] T 5ac5ffe3cd65463fb448e819e3978841 P 787ddcc824c2490a808fc5a53452dc5e: No bootstrap required, opened a new log
I20250109 08:57:29.784528 21704 ts_tablet_manager.cc:1397] T 5ac5ffe3cd65463fb448e819e3978841 P 787ddcc824c2490a808fc5a53452dc5e: Time spent bootstrapping tablet: real 0.045s user 0.015s sys 0.003s
I20250109 08:57:29.782030 21589 raft_consensus.cc:2463] T 71ac0243d72845659291184835652dde P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8bf6e26e98b14566899578a990b27d47 in term 1.
I20250109 08:57:29.788007 21758 raft_consensus.cc:491] T 81d74e7d4472497aac5463a4b47e4fb9 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:29.788534 21758 raft_consensus.cc:513] T 81d74e7d4472497aac5463a4b47e4fb9 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:29.789637 21704 raft_consensus.cc:357] T 5ac5ffe3cd65463fb448e819e3978841 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:29.790268 21704 raft_consensus.cc:383] T 5ac5ffe3cd65463fb448e819e3978841 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:29.790534 21704 raft_consensus.cc:738] T 5ac5ffe3cd65463fb448e819e3978841 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Initialized, Role: FOLLOWER
I20250109 08:57:29.791378 21704 consensus_queue.cc:260] T 5ac5ffe3cd65463fb448e819e3978841 P 787ddcc824c2490a808fc5a53452dc5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:29.791817 21478 leader_election.cc:304] T 71ac0243d72845659291184835652dde P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3; no voters: 787ddcc824c2490a808fc5a53452dc5e
I20250109 08:57:29.790372 21758 leader_election.cc:290] T 81d74e7d4472497aac5463a4b47e4fb9 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
I20250109 08:57:29.793867 21703 tablet_bootstrap.cc:492] T 5ac5ffe3cd65463fb448e819e3978841 P 8bf6e26e98b14566899578a990b27d47: No bootstrap required, opened a new log
I20250109 08:57:29.795387 21703 ts_tablet_manager.cc:1397] T 5ac5ffe3cd65463fb448e819e3978841 P 8bf6e26e98b14566899578a990b27d47: Time spent bootstrapping tablet: real 0.062s user 0.016s sys 0.019s
I20250109 08:57:29.796301 21709 leader_election.cc:290] T 363fa693a5114f2bb22f4ab67722a1c6 P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 election: Requested vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:29.796568 21513 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81d74e7d4472497aac5463a4b47e4fb9" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47" is_pre_election: true
I20250109 08:57:29.797230 21513 raft_consensus.cc:2463] T 81d74e7d4472497aac5463a4b47e4fb9 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 0.
I20250109 08:57:29.797598 21514 raft_consensus.cc:2463] T 363fa693a5114f2bb22f4ab67722a1c6 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2b1a692435241ee988e0bd1661cd4c3 in term 1.
I20250109 08:57:29.797763 21703 raft_consensus.cc:357] T 5ac5ffe3cd65463fb448e819e3978841 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:29.798413 21703 raft_consensus.cc:383] T 5ac5ffe3cd65463fb448e819e3978841 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:29.798698 21703 raft_consensus.cc:738] T 5ac5ffe3cd65463fb448e819e3978841 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bf6e26e98b14566899578a990b27d47, State: Initialized, Role: FOLLOWER
I20250109 08:57:29.798923 21626 leader_election.cc:304] T 81d74e7d4472497aac5463a4b47e4fb9 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, 8bf6e26e98b14566899578a990b27d47; no voters:
I20250109 08:57:29.798810 21552 leader_election.cc:304] T 363fa693a5114f2bb22f4ab67722a1c6 P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3; no voters:
I20250109 08:57:29.799402 21703 consensus_queue.cc:260] T 5ac5ffe3cd65463fb448e819e3978841 P 8bf6e26e98b14566899578a990b27d47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:29.799799 21709 raft_consensus.cc:2798] T 363fa693a5114f2bb22f4ab67722a1c6 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:57:29.794139 21731 raft_consensus.cc:2798] T 71ac0243d72845659291184835652dde P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:57:29.800607 21731 raft_consensus.cc:695] T 71ac0243d72845659291184835652dde P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Becoming Leader. State: Replica: 8bf6e26e98b14566899578a990b27d47, State: Running, Role: LEADER
I20250109 08:57:29.801797 21703 ts_tablet_manager.cc:1428] T 5ac5ffe3cd65463fb448e819e3978841 P 8bf6e26e98b14566899578a990b27d47: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
I20250109 08:57:29.801411 21731 consensus_queue.cc:237] T 71ac0243d72845659291184835652dde P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:29.803342 21703 tablet_bootstrap.cc:492] T 0e06021cd6624e50af2c914399ebd6c4 P 8bf6e26e98b14566899578a990b27d47: Bootstrap starting.
I20250109 08:57:29.804319 21710 raft_consensus.cc:2798] T 81d74e7d4472497aac5463a4b47e4fb9 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:57:29.804718 21710 raft_consensus.cc:491] T 81d74e7d4472497aac5463a4b47e4fb9 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:57:29.805029 21710 raft_consensus.cc:3054] T 81d74e7d4472497aac5463a4b47e4fb9 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:29.807235 21589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81d74e7d4472497aac5463a4b47e4fb9" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3" is_pre_election: true
I20250109 08:57:29.807828 21702 tablet_bootstrap.cc:492] T d8493108cba04514bd2236192cef1079 P a2b1a692435241ee988e0bd1661cd4c3: No bootstrap required, opened a new log
I20250109 08:57:29.808229 21702 ts_tablet_manager.cc:1397] T d8493108cba04514bd2236192cef1079 P a2b1a692435241ee988e0bd1661cd4c3: Time spent bootstrapping tablet: real 0.064s user 0.005s sys 0.020s
W20250109 08:57:29.808749 21627 leader_election.cc:343] T 81d74e7d4472497aac5463a4b47e4fb9 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:29.804247 21661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "363fa693a5114f2bb22f4ab67722a1c6" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e"
I20250109 08:57:29.809548 21661 raft_consensus.cc:3054] T 363fa693a5114f2bb22f4ab67722a1c6 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:29.810566 21702 raft_consensus.cc:357] T d8493108cba04514bd2236192cef1079 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:29.811236 21702 raft_consensus.cc:383] T d8493108cba04514bd2236192cef1079 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:29.811511 21702 raft_consensus.cc:738] T d8493108cba04514bd2236192cef1079 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2b1a692435241ee988e0bd1661cd4c3, State: Initialized, Role: FOLLOWER
I20250109 08:57:29.811774 21710 raft_consensus.cc:513] T 81d74e7d4472497aac5463a4b47e4fb9 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:29.800393 21709 raft_consensus.cc:695] T 363fa693a5114f2bb22f4ab67722a1c6 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 LEADER]: Becoming Leader. State: Replica: a2b1a692435241ee988e0bd1661cd4c3, State: Running, Role: LEADER
I20250109 08:57:29.813199 21709 consensus_queue.cc:237] T 363fa693a5114f2bb22f4ab67722a1c6 P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:29.814426 21514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81d74e7d4472497aac5463a4b47e4fb9" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47"
I20250109 08:57:29.815014 21514 raft_consensus.cc:3054] T 81d74e7d4472497aac5463a4b47e4fb9 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:29.815968 21703 tablet_bootstrap.cc:654] T 0e06021cd6624e50af2c914399ebd6c4 P 8bf6e26e98b14566899578a990b27d47: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:29.816927 21710 leader_election.cc:290] T 81d74e7d4472497aac5463a4b47e4fb9 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 election: Requested vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
I20250109 08:57:29.818060 21704 ts_tablet_manager.cc:1428] T 5ac5ffe3cd65463fb448e819e3978841 P 787ddcc824c2490a808fc5a53452dc5e: Time spent starting tablet: real 0.033s user 0.006s sys 0.000s
I20250109 08:57:29.819432 21704 tablet_bootstrap.cc:492] T 71ac0243d72845659291184835652dde P 787ddcc824c2490a808fc5a53452dc5e: Bootstrap starting.
I20250109 08:57:29.820350 21702 consensus_queue.cc:260] T d8493108cba04514bd2236192cef1079 P a2b1a692435241ee988e0bd1661cd4c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:29.822013 21702 ts_tablet_manager.cc:1428] T d8493108cba04514bd2236192cef1079 P a2b1a692435241ee988e0bd1661cd4c3: Time spent starting tablet: real 0.014s user 0.004s sys 0.000s
I20250109 08:57:29.828685 21702 tablet_bootstrap.cc:492] T d6f988a231a74f6fa112da9a527d4a0b P a2b1a692435241ee988e0bd1661cd4c3: Bootstrap starting.
I20250109 08:57:29.831550 21781 raft_consensus.cc:491] T 306c4892bbff4dbfbe5a0645294aa627 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:29.832131 21781 raft_consensus.cc:513] T 306c4892bbff4dbfbe5a0645294aa627 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:29.834852 21781 leader_election.cc:290] T 306c4892bbff4dbfbe5a0645294aa627 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:29.835083 21661 raft_consensus.cc:2463] T 363fa693a5114f2bb22f4ab67722a1c6 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2b1a692435241ee988e0bd1661cd4c3 in term 1.
I20250109 08:57:29.836843 21660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "306c4892bbff4dbfbe5a0645294aa627" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e" is_pre_election: true
I20250109 08:57:29.838132 21587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "306c4892bbff4dbfbe5a0645294aa627" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3" is_pre_election: true
W20250109 08:57:29.838405 21477 leader_election.cc:343] T 306c4892bbff4dbfbe5a0645294aa627 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731): Illegal state: must be running to vote when last-logged opid is not known
W20250109 08:57:29.841643 21478 leader_election.cc:343] T 306c4892bbff4dbfbe5a0645294aa627 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:29.842094 21478 leader_election.cc:304] T 306c4892bbff4dbfbe5a0645294aa627 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8bf6e26e98b14566899578a990b27d47; no voters: 787ddcc824c2490a808fc5a53452dc5e, a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:57:29.842868 21742 raft_consensus.cc:2743] T 306c4892bbff4dbfbe5a0645294aa627 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:57:29.850152 21703 tablet_bootstrap.cc:492] T 0e06021cd6624e50af2c914399ebd6c4 P 8bf6e26e98b14566899578a990b27d47: No bootstrap required, opened a new log
I20250109 08:57:29.851007 21703 ts_tablet_manager.cc:1397] T 0e06021cd6624e50af2c914399ebd6c4 P 8bf6e26e98b14566899578a990b27d47: Time spent bootstrapping tablet: real 0.048s user 0.013s sys 0.021s
I20250109 08:57:29.854156 21660 raft_consensus.cc:3054] T db34a00c02b84ee988edc7d6e344c1f5 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:29.855248 21702 tablet_bootstrap.cc:654] T d6f988a231a74f6fa112da9a527d4a0b P a2b1a692435241ee988e0bd1661cd4c3: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:29.855340 21514 raft_consensus.cc:2463] T 81d74e7d4472497aac5463a4b47e4fb9 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 1.
I20250109 08:57:29.856511 21626 leader_election.cc:304] T 81d74e7d4472497aac5463a4b47e4fb9 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, 8bf6e26e98b14566899578a990b27d47; no voters:
I20250109 08:57:29.857240 21710 raft_consensus.cc:2798] T 81d74e7d4472497aac5463a4b47e4fb9 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:57:29.857676 21710 raft_consensus.cc:695] T 81d74e7d4472497aac5463a4b47e4fb9 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Becoming Leader. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Running, Role: LEADER
I20250109 08:57:29.858419 21710 consensus_queue.cc:237] T 81d74e7d4472497aac5463a4b47e4fb9 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:29.862108 21589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81d74e7d4472497aac5463a4b47e4fb9" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3"
W20250109 08:57:29.874022 21627 leader_election.cc:343] T 81d74e7d4472497aac5463a4b47e4fb9 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:29.877404 21703 raft_consensus.cc:357] T 0e06021cd6624e50af2c914399ebd6c4 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:29.878420 21731 consensus_queue.cc:1035] T 79c1ed8ab551478291f1f24cc940cb27 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:57:29.877818 21781 consensus_queue.cc:1035] T e601a034e10e418bac8683443d86a1c1 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:57:29.884953 21703 raft_consensus.cc:383] T 0e06021cd6624e50af2c914399ebd6c4 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:29.885310 21703 raft_consensus.cc:738] T 0e06021cd6624e50af2c914399ebd6c4 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bf6e26e98b14566899578a990b27d47, State: Initialized, Role: FOLLOWER
I20250109 08:57:29.887773 21703 consensus_queue.cc:260] T 0e06021cd6624e50af2c914399ebd6c4 P 8bf6e26e98b14566899578a990b27d47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:29.890123 21703 ts_tablet_manager.cc:1428] T 0e06021cd6624e50af2c914399ebd6c4 P 8bf6e26e98b14566899578a990b27d47: Time spent starting tablet: real 0.039s user 0.005s sys 0.001s
I20250109 08:57:29.878021 21589 raft_consensus.cc:3054] T e601a034e10e418bac8683443d86a1c1 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:29.896813 21412 catalog_manager.cc:5526] T 363fa693a5114f2bb22f4ab67722a1c6 P a2b1a692435241ee988e0bd1661cd4c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194). New cstate: current_term: 1 leader_uuid: "a2b1a692435241ee988e0bd1661cd4c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: UNKNOWN } } }
I20250109 08:57:29.898099 21704 tablet_bootstrap.cc:654] T 71ac0243d72845659291184835652dde P 787ddcc824c2490a808fc5a53452dc5e: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:29.902011 21703 tablet_bootstrap.cc:492] T d6f988a231a74f6fa112da9a527d4a0b P 8bf6e26e98b14566899578a990b27d47: Bootstrap starting.
I20250109 08:57:29.921308 21702 tablet_bootstrap.cc:492] T d6f988a231a74f6fa112da9a527d4a0b P a2b1a692435241ee988e0bd1661cd4c3: No bootstrap required, opened a new log
I20250109 08:57:29.921829 21702 ts_tablet_manager.cc:1397] T d6f988a231a74f6fa112da9a527d4a0b P a2b1a692435241ee988e0bd1661cd4c3: Time spent bootstrapping tablet: real 0.093s user 0.005s sys 0.021s
I20250109 08:57:29.911423 21781 raft_consensus.cc:491] T 180b4d539a144a38905bf340ecb24ef4 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:29.931329 21410 catalog_manager.cc:5526] T 71ac0243d72845659291184835652dde P 8bf6e26e98b14566899578a990b27d47 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8bf6e26e98b14566899578a990b27d47 (127.19.226.193). New cstate: current_term: 1 leader_uuid: "8bf6e26e98b14566899578a990b27d47" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: UNKNOWN } } }
I20250109 08:57:29.932629 21709 raft_consensus.cc:491] T 0e06021cd6624e50af2c914399ebd6c4 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:29.933076 21709 raft_consensus.cc:513] T 0e06021cd6624e50af2c914399ebd6c4 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:29.934900 21709 leader_election.cc:290] T 0e06021cd6624e50af2c914399ebd6c4 P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:29.945804 21514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e06021cd6624e50af2c914399ebd6c4" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47" is_pre_election: true
I20250109 08:57:29.946525 21514 raft_consensus.cc:2463] T 0e06021cd6624e50af2c914399ebd6c4 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2b1a692435241ee988e0bd1661cd4c3 in term 0.
I20250109 08:57:29.947504 21662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e06021cd6624e50af2c914399ebd6c4" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e" is_pre_election: true
I20250109 08:57:29.948917 21552 leader_election.cc:304] T 0e06021cd6624e50af2c914399ebd6c4 P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3; no voters:
W20250109 08:57:29.949898 21552 leader_election.cc:343] T 0e06021cd6624e50af2c914399ebd6c4 P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:29.950914 21759 raft_consensus.cc:491] T e601a034e10e418bac8683443d86a1c1 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:29.951380 21759 raft_consensus.cc:513] T e601a034e10e418bac8683443d86a1c1 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:29.951670 21711 consensus_queue.cc:1035] T e601a034e10e418bac8683443d86a1c1 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:57:29.951515 21781 raft_consensus.cc:513] T 180b4d539a144a38905bf340ecb24ef4 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:29.953115 21709 raft_consensus.cc:2798] T 0e06021cd6624e50af2c914399ebd6c4 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:57:29.953528 21709 raft_consensus.cc:491] T 0e06021cd6624e50af2c914399ebd6c4 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:57:29.953855 21709 raft_consensus.cc:3054] T 0e06021cd6624e50af2c914399ebd6c4 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:29.956580 21514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e601a034e10e418bac8683443d86a1c1" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47" is_pre_election: true
I20250109 08:57:29.957149 21661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "180b4d539a144a38905bf340ecb24ef4" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e" is_pre_election: true
I20250109 08:57:29.957744 21661 raft_consensus.cc:2463] T 180b4d539a144a38905bf340ecb24ef4 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8bf6e26e98b14566899578a990b27d47 in term 0.
I20250109 08:57:29.958396 21759 leader_election.cc:290] T e601a034e10e418bac8683443d86a1c1 P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:29.958950 21477 leader_election.cc:304] T 180b4d539a144a38905bf340ecb24ef4 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, 8bf6e26e98b14566899578a990b27d47; no voters:
I20250109 08:57:29.959710 21711 raft_consensus.cc:2798] T 180b4d539a144a38905bf340ecb24ef4 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:57:29.959560 21759 consensus_queue.cc:1035] T 97304b06c7fa40ef8345a72c796c47f8 P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:57:29.960034 21711 raft_consensus.cc:491] T 180b4d539a144a38905bf340ecb24ef4 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:57:29.960359 21711 raft_consensus.cc:3054] T 180b4d539a144a38905bf340ecb24ef4 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:29.960530 21662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e601a034e10e418bac8683443d86a1c1" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e" is_pre_election: true
I20250109 08:57:29.961416 21587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "180b4d539a144a38905bf340ecb24ef4" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3" is_pre_election: true
W20250109 08:57:29.962836 21478 leader_election.cc:343] T 180b4d539a144a38905bf340ecb24ef4 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:29.966367 21552 leader_election.cc:304] T e601a034e10e418bac8683443d86a1c1 P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2b1a692435241ee988e0bd1661cd4c3; no voters: 787ddcc824c2490a808fc5a53452dc5e, 8bf6e26e98b14566899578a990b27d47
I20250109 08:57:29.964852 21781 leader_election.cc:290] T 180b4d539a144a38905bf340ecb24ef4 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:29.968603 21759 raft_consensus.cc:2743] T e601a034e10e418bac8683443d86a1c1 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250109 08:57:29.974350 21702 raft_consensus.cc:357] T d6f988a231a74f6fa112da9a527d4a0b P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:29.980623 21702 raft_consensus.cc:383] T d6f988a231a74f6fa112da9a527d4a0b P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:29.980916 21702 raft_consensus.cc:738] T d6f988a231a74f6fa112da9a527d4a0b P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2b1a692435241ee988e0bd1661cd4c3, State: Initialized, Role: FOLLOWER
I20250109 08:57:29.981551 21702 consensus_queue.cc:260] T d6f988a231a74f6fa112da9a527d4a0b P a2b1a692435241ee988e0bd1661cd4c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
W20250109 08:57:29.995457 21460 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20250109 08:57:29.996104 21709 raft_consensus.cc:513] T 0e06021cd6624e50af2c914399ebd6c4 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:29.997140 21704 tablet_bootstrap.cc:492] T 71ac0243d72845659291184835652dde P 787ddcc824c2490a808fc5a53452dc5e: No bootstrap required, opened a new log
I20250109 08:57:29.997553 21704 ts_tablet_manager.cc:1397] T 71ac0243d72845659291184835652dde P 787ddcc824c2490a808fc5a53452dc5e: Time spent bootstrapping tablet: real 0.178s user 0.009s sys 0.008s
I20250109 08:57:30.001013 21702 ts_tablet_manager.cc:1428] T d6f988a231a74f6fa112da9a527d4a0b P a2b1a692435241ee988e0bd1661cd4c3: Time spent starting tablet: real 0.079s user 0.005s sys 0.005s
I20250109 08:57:30.002161 21703 tablet_bootstrap.cc:654] T d6f988a231a74f6fa112da9a527d4a0b P 8bf6e26e98b14566899578a990b27d47: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:30.005041 21660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e06021cd6624e50af2c914399ebd6c4" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e"
W20250109 08:57:30.006491 21552 leader_election.cc:343] T 0e06021cd6624e50af2c914399ebd6c4 P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:30.009459 21513 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e06021cd6624e50af2c914399ebd6c4" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47"
I20250109 08:57:30.009302 21711 raft_consensus.cc:513] T 180b4d539a144a38905bf340ecb24ef4 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.010118 21513 raft_consensus.cc:3054] T 0e06021cd6624e50af2c914399ebd6c4 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:30.007846 21709 leader_election.cc:290] T 0e06021cd6624e50af2c914399ebd6c4 P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 election: Requested vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:30.014837 21587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "180b4d539a144a38905bf340ecb24ef4" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3"
I20250109 08:57:30.002911 21759 raft_consensus.cc:491] T d6f988a231a74f6fa112da9a527d4a0b P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20250109 08:57:30.016507 21478 leader_election.cc:343] T 180b4d539a144a38905bf340ecb24ef4 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:30.016789 21759 raft_consensus.cc:513] T d6f988a231a74f6fa112da9a527d4a0b P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.018841 21759 leader_election.cc:290] T d6f988a231a74f6fa112da9a527d4a0b P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:30.019596 21711 leader_election.cc:290] T 180b4d539a144a38905bf340ecb24ef4 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 election: Requested vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:30.020887 21702 tablet_bootstrap.cc:492] T 89e1eeb24fea483eb63bd98af66e0b6b P a2b1a692435241ee988e0bd1661cd4c3: Bootstrap starting.
I20250109 08:57:30.020541 21662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "180b4d539a144a38905bf340ecb24ef4" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e"
I20250109 08:57:30.022406 21662 raft_consensus.cc:3054] T 180b4d539a144a38905bf340ecb24ef4 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:30.026973 21514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6f988a231a74f6fa112da9a527d4a0b" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47" is_pre_election: true
I20250109 08:57:30.024498 21709 consensus_queue.cc:1035] T 97304b06c7fa40ef8345a72c796c47f8 P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
W20250109 08:57:30.030030 21552 leader_election.cc:343] T d6f988a231a74f6fa112da9a527d4a0b P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:30.032119 21704 raft_consensus.cc:357] T 71ac0243d72845659291184835652dde P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.033632 21704 raft_consensus.cc:383] T 71ac0243d72845659291184835652dde P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:30.033950 21704 raft_consensus.cc:738] T 71ac0243d72845659291184835652dde P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Initialized, Role: FOLLOWER
I20250109 08:57:30.034673 21704 consensus_queue.cc:260] T 71ac0243d72845659291184835652dde P 787ddcc824c2490a808fc5a53452dc5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.041215 21660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6f988a231a74f6fa112da9a527d4a0b" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e" is_pre_election: true
W20250109 08:57:30.042505 21552 leader_election.cc:343] T d6f988a231a74f6fa112da9a527d4a0b P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:30.043006 21552 leader_election.cc:304] T d6f988a231a74f6fa112da9a527d4a0b P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2b1a692435241ee988e0bd1661cd4c3; no voters: 787ddcc824c2490a808fc5a53452dc5e, 8bf6e26e98b14566899578a990b27d47
I20250109 08:57:30.045125 21758 raft_consensus.cc:491] T 5ac5ffe3cd65463fb448e819e3978841 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:30.045624 21758 raft_consensus.cc:513] T 5ac5ffe3cd65463fb448e819e3978841 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.049548 21704 ts_tablet_manager.cc:1428] T 71ac0243d72845659291184835652dde P 787ddcc824c2490a808fc5a53452dc5e: Time spent starting tablet: real 0.052s user 0.006s sys 0.000s
I20250109 08:57:30.053452 21512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ac5ffe3cd65463fb448e819e3978841" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47" is_pre_election: true
I20250109 08:57:30.054821 21704 tablet_bootstrap.cc:492] T d8493108cba04514bd2236192cef1079 P 787ddcc824c2490a808fc5a53452dc5e: Bootstrap starting.
I20250109 08:57:30.054118 21512 raft_consensus.cc:2463] T 5ac5ffe3cd65463fb448e819e3978841 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 0.
I20250109 08:57:30.055662 21758 leader_election.cc:290] T 5ac5ffe3cd65463fb448e819e3978841 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
I20250109 08:57:30.056298 21626 leader_election.cc:304] T 5ac5ffe3cd65463fb448e819e3978841 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, 8bf6e26e98b14566899578a990b27d47; no voters:
I20250109 08:57:30.057281 21588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ac5ffe3cd65463fb448e819e3978841" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3" is_pre_election: true
I20250109 08:57:30.057577 21732 raft_consensus.cc:2798] T 5ac5ffe3cd65463fb448e819e3978841 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:57:30.059062 21410 catalog_manager.cc:5526] T 81d74e7d4472497aac5463a4b47e4fb9 P 787ddcc824c2490a808fc5a53452dc5e reported cstate change: term changed from 0 to 1, leader changed from <none> to 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195). New cstate: current_term: 1 leader_uuid: "787ddcc824c2490a808fc5a53452dc5e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: HEALTHY } } }
I20250109 08:57:30.052711 21709 raft_consensus.cc:2743] T d6f988a231a74f6fa112da9a527d4a0b P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:57:30.057909 21732 raft_consensus.cc:491] T 5ac5ffe3cd65463fb448e819e3978841 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:57:30.062463 21732 raft_consensus.cc:3054] T 5ac5ffe3cd65463fb448e819e3978841 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Advancing to term 1
W20250109 08:57:30.063413 21627 leader_election.cc:343] T 5ac5ffe3cd65463fb448e819e3978841 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:30.068571 21662 raft_consensus.cc:2463] T 180b4d539a144a38905bf340ecb24ef4 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8bf6e26e98b14566899578a990b27d47 in term 1.
I20250109 08:57:30.069851 21477 leader_election.cc:304] T 180b4d539a144a38905bf340ecb24ef4 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, 8bf6e26e98b14566899578a990b27d47; no voters: a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:57:30.071206 21711 raft_consensus.cc:2798] T 180b4d539a144a38905bf340ecb24ef4 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:57:30.071640 21711 raft_consensus.cc:695] T 180b4d539a144a38905bf340ecb24ef4 P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Becoming Leader. State: Replica: 8bf6e26e98b14566899578a990b27d47, State: Running, Role: LEADER
I20250109 08:57:30.072491 21732 raft_consensus.cc:513] T 5ac5ffe3cd65463fb448e819e3978841 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.072427 21711 consensus_queue.cc:237] T 180b4d539a144a38905bf340ecb24ef4 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.078039 21512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ac5ffe3cd65463fb448e819e3978841" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47"
I20250109 08:57:30.073763 21513 raft_consensus.cc:2463] T 0e06021cd6624e50af2c914399ebd6c4 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2b1a692435241ee988e0bd1661cd4c3 in term 1.
I20250109 08:57:30.078603 21512 raft_consensus.cc:3054] T 5ac5ffe3cd65463fb448e819e3978841 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:30.090483 21732 leader_election.cc:290] T 5ac5ffe3cd65463fb448e819e3978841 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 election: Requested vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
I20250109 08:57:30.091490 21586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ac5ffe3cd65463fb448e819e3978841" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3"
W20250109 08:57:30.092708 21627 leader_election.cc:343] T 5ac5ffe3cd65463fb448e819e3978841 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:30.093246 21552 leader_election.cc:304] T 0e06021cd6624e50af2c914399ebd6c4 P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3; no voters: 787ddcc824c2490a808fc5a53452dc5e
I20250109 08:57:30.094023 21709 raft_consensus.cc:2798] T 0e06021cd6624e50af2c914399ebd6c4 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:57:30.094571 21709 raft_consensus.cc:695] T 0e06021cd6624e50af2c914399ebd6c4 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 LEADER]: Becoming Leader. State: Replica: a2b1a692435241ee988e0bd1661cd4c3, State: Running, Role: LEADER
I20250109 08:57:30.095317 21709 consensus_queue.cc:237] T 0e06021cd6624e50af2c914399ebd6c4 P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.101811 21704 tablet_bootstrap.cc:654] T d8493108cba04514bd2236192cef1079 P 787ddcc824c2490a808fc5a53452dc5e: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:30.106561 21732 raft_consensus.cc:491] T 71ac0243d72845659291184835652dde P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:30.107060 21732 raft_consensus.cc:513] T 71ac0243d72845659291184835652dde P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.108932 21819 raft_consensus.cc:491] T d8493108cba04514bd2236192cef1079 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:30.109467 21819 raft_consensus.cc:513] T d8493108cba04514bd2236192cef1079 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.111979 21819 leader_election.cc:290] T d8493108cba04514bd2236192cef1079 P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:30.111944 21702 tablet_bootstrap.cc:654] T 89e1eeb24fea483eb63bd98af66e0b6b P a2b1a692435241ee988e0bd1661cd4c3: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:30.108350 21731 consensus_queue.cc:1035] T 80123146836349938344efc61333695f P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20250109 08:57:30.114903 21514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71ac0243d72845659291184835652dde" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47" is_pre_election: true
I20250109 08:57:30.115906 21732 leader_election.cc:290] T 71ac0243d72845659291184835652dde P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
I20250109 08:57:30.116495 21662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8493108cba04514bd2236192cef1079" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e" is_pre_election: true
I20250109 08:57:30.119148 21589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71ac0243d72845659291184835652dde" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3" is_pre_election: true
I20250109 08:57:30.120702 21589 raft_consensus.cc:2388] T 71ac0243d72845659291184835652dde P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 787ddcc824c2490a808fc5a53452dc5e in current term 1: Already voted for candidate 8bf6e26e98b14566899578a990b27d47 in this term.
I20250109 08:57:30.110677 21512 raft_consensus.cc:2463] T 5ac5ffe3cd65463fb448e819e3978841 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 1.
W20250109 08:57:30.123201 21552 leader_election.cc:343] T d8493108cba04514bd2236192cef1079 P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:30.140269 21627 leader_election.cc:304] T 71ac0243d72845659291184835652dde P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e; no voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:57:30.141552 21732 raft_consensus.cc:3054] T 71ac0243d72845659291184835652dde P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:30.125242 21514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8493108cba04514bd2236192cef1079" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47" is_pre_election: true
I20250109 08:57:30.157003 21626 leader_election.cc:304] T 5ac5ffe3cd65463fb448e819e3978841 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, 8bf6e26e98b14566899578a990b27d47; no voters: a2b1a692435241ee988e0bd1661cd4c3
W20250109 08:57:30.157307 21552 leader_election.cc:343] T d8493108cba04514bd2236192cef1079 P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:30.158300 21710 raft_consensus.cc:2798] T 5ac5ffe3cd65463fb448e819e3978841 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:57:30.158766 21710 raft_consensus.cc:695] T 5ac5ffe3cd65463fb448e819e3978841 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Becoming Leader. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Running, Role: LEADER
I20250109 08:57:30.159577 21710 consensus_queue.cc:237] T 5ac5ffe3cd65463fb448e819e3978841 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.163774 21703 tablet_bootstrap.cc:492] T d6f988a231a74f6fa112da9a527d4a0b P 8bf6e26e98b14566899578a990b27d47: No bootstrap required, opened a new log
I20250109 08:57:30.164139 21703 ts_tablet_manager.cc:1397] T d6f988a231a74f6fa112da9a527d4a0b P 8bf6e26e98b14566899578a990b27d47: Time spent bootstrapping tablet: real 0.262s user 0.017s sys 0.007s
I20250109 08:57:30.162998 21742 consensus_queue.cc:1035] T 80123146836349938344efc61333695f P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20250109 08:57:30.158306 21552 leader_election.cc:304] T d8493108cba04514bd2236192cef1079 P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2b1a692435241ee988e0bd1661cd4c3; no voters: 787ddcc824c2490a808fc5a53452dc5e, 8bf6e26e98b14566899578a990b27d47
I20250109 08:57:30.171720 21819 raft_consensus.cc:2743] T d8493108cba04514bd2236192cef1079 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:57:30.178489 21703 raft_consensus.cc:357] T d6f988a231a74f6fa112da9a527d4a0b P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.179046 21732 raft_consensus.cc:2743] T 71ac0243d72845659291184835652dde P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:57:30.180052 21703 raft_consensus.cc:383] T d6f988a231a74f6fa112da9a527d4a0b P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:30.180297 21703 raft_consensus.cc:738] T d6f988a231a74f6fa112da9a527d4a0b P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bf6e26e98b14566899578a990b27d47, State: Initialized, Role: FOLLOWER
I20250109 08:57:30.180783 21703 consensus_queue.cc:260] T d6f988a231a74f6fa112da9a527d4a0b P 8bf6e26e98b14566899578a990b27d47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.193145 21702 tablet_bootstrap.cc:492] T 89e1eeb24fea483eb63bd98af66e0b6b P a2b1a692435241ee988e0bd1661cd4c3: No bootstrap required, opened a new log
I20250109 08:57:30.193177 21704 tablet_bootstrap.cc:492] T d8493108cba04514bd2236192cef1079 P 787ddcc824c2490a808fc5a53452dc5e: No bootstrap required, opened a new log
I20250109 08:57:30.193663 21702 ts_tablet_manager.cc:1397] T 89e1eeb24fea483eb63bd98af66e0b6b P a2b1a692435241ee988e0bd1661cd4c3: Time spent bootstrapping tablet: real 0.180s user 0.016s sys 0.035s
I20250109 08:57:30.193948 21704 ts_tablet_manager.cc:1397] T d8493108cba04514bd2236192cef1079 P 787ddcc824c2490a808fc5a53452dc5e: Time spent bootstrapping tablet: real 0.139s user 0.029s sys 0.044s
I20250109 08:57:30.196219 21702 raft_consensus.cc:357] T 89e1eeb24fea483eb63bd98af66e0b6b P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.197010 21702 raft_consensus.cc:383] T 89e1eeb24fea483eb63bd98af66e0b6b P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:30.197281 21702 raft_consensus.cc:738] T 89e1eeb24fea483eb63bd98af66e0b6b P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2b1a692435241ee988e0bd1661cd4c3, State: Initialized, Role: FOLLOWER
I20250109 08:57:30.197866 21702 consensus_queue.cc:260] T 89e1eeb24fea483eb63bd98af66e0b6b P a2b1a692435241ee988e0bd1661cd4c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.189563 21410 catalog_manager.cc:5526] T 0e06021cd6624e50af2c914399ebd6c4 P a2b1a692435241ee988e0bd1661cd4c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194). New cstate: current_term: 1 leader_uuid: "a2b1a692435241ee988e0bd1661cd4c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: UNKNOWN } } }
I20250109 08:57:30.196472 21704 raft_consensus.cc:357] T d8493108cba04514bd2236192cef1079 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.202888 21704 raft_consensus.cc:383] T d8493108cba04514bd2236192cef1079 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:30.203184 21704 raft_consensus.cc:738] T d8493108cba04514bd2236192cef1079 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Initialized, Role: FOLLOWER
I20250109 08:57:30.203791 21704 consensus_queue.cc:260] T d8493108cba04514bd2236192cef1079 P 787ddcc824c2490a808fc5a53452dc5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.205513 21703 ts_tablet_manager.cc:1428] T d6f988a231a74f6fa112da9a527d4a0b P 8bf6e26e98b14566899578a990b27d47: Time spent starting tablet: real 0.041s user 0.006s sys 0.000s
I20250109 08:57:30.214485 21703 tablet_bootstrap.cc:492] T e38ec0ef84af4b798073eafd4d210c35 P 8bf6e26e98b14566899578a990b27d47: Bootstrap starting.
I20250109 08:57:30.213361 21702 ts_tablet_manager.cc:1428] T 89e1eeb24fea483eb63bd98af66e0b6b P a2b1a692435241ee988e0bd1661cd4c3: Time spent starting tablet: real 0.019s user 0.008s sys 0.000s
I20250109 08:57:30.219173 21702 tablet_bootstrap.cc:492] T 3b5049c00b0b4dd89c49fa5d13db169e P a2b1a692435241ee988e0bd1661cd4c3: Bootstrap starting.
I20250109 08:57:30.219897 21704 ts_tablet_manager.cc:1428] T d8493108cba04514bd2236192cef1079 P 787ddcc824c2490a808fc5a53452dc5e: Time spent starting tablet: real 0.026s user 0.005s sys 0.000s
I20250109 08:57:30.222530 21412 catalog_manager.cc:5526] T 180b4d539a144a38905bf340ecb24ef4 P 8bf6e26e98b14566899578a990b27d47 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8bf6e26e98b14566899578a990b27d47 (127.19.226.193). New cstate: current_term: 1 leader_uuid: "8bf6e26e98b14566899578a990b27d47" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: UNKNOWN } } }
I20250109 08:57:30.226596 21704 tablet_bootstrap.cc:492] T 3b5049c00b0b4dd89c49fa5d13db169e P 787ddcc824c2490a808fc5a53452dc5e: Bootstrap starting.
I20250109 08:57:30.230804 21703 tablet_bootstrap.cc:654] T e38ec0ef84af4b798073eafd4d210c35 P 8bf6e26e98b14566899578a990b27d47: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:30.242439 21702 tablet_bootstrap.cc:654] T 3b5049c00b0b4dd89c49fa5d13db169e P a2b1a692435241ee988e0bd1661cd4c3: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:30.247195 21704 tablet_bootstrap.cc:654] T 3b5049c00b0b4dd89c49fa5d13db169e P 787ddcc824c2490a808fc5a53452dc5e: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:30.236198 21410 catalog_manager.cc:5526] T 5ac5ffe3cd65463fb448e819e3978841 P 787ddcc824c2490a808fc5a53452dc5e reported cstate change: term changed from 0 to 1, leader changed from <none> to 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195). New cstate: current_term: 1 leader_uuid: "787ddcc824c2490a808fc5a53452dc5e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: HEALTHY } } }
I20250109 08:57:30.256572 21702 tablet_bootstrap.cc:492] T 3b5049c00b0b4dd89c49fa5d13db169e P a2b1a692435241ee988e0bd1661cd4c3: No bootstrap required, opened a new log
I20250109 08:57:30.257030 21702 ts_tablet_manager.cc:1397] T 3b5049c00b0b4dd89c49fa5d13db169e P a2b1a692435241ee988e0bd1661cd4c3: Time spent bootstrapping tablet: real 0.038s user 0.011s sys 0.000s
I20250109 08:57:30.261880 21703 tablet_bootstrap.cc:492] T e38ec0ef84af4b798073eafd4d210c35 P 8bf6e26e98b14566899578a990b27d47: No bootstrap required, opened a new log
I20250109 08:57:30.262413 21703 ts_tablet_manager.cc:1397] T e38ec0ef84af4b798073eafd4d210c35 P 8bf6e26e98b14566899578a990b27d47: Time spent bootstrapping tablet: real 0.048s user 0.011s sys 0.015s
I20250109 08:57:30.264959 21703 raft_consensus.cc:357] T e38ec0ef84af4b798073eafd4d210c35 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.265687 21703 raft_consensus.cc:383] T e38ec0ef84af4b798073eafd4d210c35 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:30.265964 21703 raft_consensus.cc:738] T e38ec0ef84af4b798073eafd4d210c35 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bf6e26e98b14566899578a990b27d47, State: Initialized, Role: FOLLOWER
I20250109 08:57:30.266588 21703 consensus_queue.cc:260] T e38ec0ef84af4b798073eafd4d210c35 P 8bf6e26e98b14566899578a990b27d47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.268581 21759 raft_consensus.cc:491] T 89e1eeb24fea483eb63bd98af66e0b6b P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:30.262481 21702 raft_consensus.cc:357] T 3b5049c00b0b4dd89c49fa5d13db169e P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.269073 21703 ts_tablet_manager.cc:1428] T e38ec0ef84af4b798073eafd4d210c35 P 8bf6e26e98b14566899578a990b27d47: Time spent starting tablet: real 0.006s user 0.006s sys 0.001s
I20250109 08:57:30.284967 21703 tablet_bootstrap.cc:492] T d8493108cba04514bd2236192cef1079 P 8bf6e26e98b14566899578a990b27d47: Bootstrap starting.
I20250109 08:57:30.280804 21709 consensus_queue.cc:1035] T 363fa693a5114f2bb22f4ab67722a1c6 P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20250109 08:57:30.272248 21732 consensus_queue.cc:1035] T 81d74e7d4472497aac5463a4b47e4fb9 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.015s
I20250109 08:57:30.269030 21759 raft_consensus.cc:513] T 89e1eeb24fea483eb63bd98af66e0b6b P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.291394 21759 leader_election.cc:290] T 89e1eeb24fea483eb63bd98af66e0b6b P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:30.291455 21702 raft_consensus.cc:383] T 3b5049c00b0b4dd89c49fa5d13db169e P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:30.293216 21702 raft_consensus.cc:738] T 3b5049c00b0b4dd89c49fa5d13db169e P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2b1a692435241ee988e0bd1661cd4c3, State: Initialized, Role: FOLLOWER
I20250109 08:57:30.293818 21702 consensus_queue.cc:260] T 3b5049c00b0b4dd89c49fa5d13db169e P a2b1a692435241ee988e0bd1661cd4c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.294901 21662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89e1eeb24fea483eb63bd98af66e0b6b" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e" is_pre_election: true
I20250109 08:57:30.295856 21702 ts_tablet_manager.cc:1428] T 3b5049c00b0b4dd89c49fa5d13db169e P a2b1a692435241ee988e0bd1661cd4c3: Time spent starting tablet: real 0.039s user 0.011s sys 0.000s
I20250109 08:57:30.296742 21702 tablet_bootstrap.cc:492] T 81d74e7d4472497aac5463a4b47e4fb9 P a2b1a692435241ee988e0bd1661cd4c3: Bootstrap starting.
W20250109 08:57:30.300837 21552 leader_election.cc:343] T 89e1eeb24fea483eb63bd98af66e0b6b P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:30.306360 21512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89e1eeb24fea483eb63bd98af66e0b6b" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47" is_pre_election: true
I20250109 08:57:30.306766 21704 tablet_bootstrap.cc:492] T 3b5049c00b0b4dd89c49fa5d13db169e P 787ddcc824c2490a808fc5a53452dc5e: No bootstrap required, opened a new log
I20250109 08:57:30.307242 21704 ts_tablet_manager.cc:1397] T 3b5049c00b0b4dd89c49fa5d13db169e P 787ddcc824c2490a808fc5a53452dc5e: Time spent bootstrapping tablet: real 0.081s user 0.031s sys 0.024s
W20250109 08:57:30.308521 21552 leader_election.cc:343] T 89e1eeb24fea483eb63bd98af66e0b6b P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:30.308969 21552 leader_election.cc:304] T 89e1eeb24fea483eb63bd98af66e0b6b P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2b1a692435241ee988e0bd1661cd4c3; no voters: 787ddcc824c2490a808fc5a53452dc5e, 8bf6e26e98b14566899578a990b27d47
I20250109 08:57:30.310282 21759 raft_consensus.cc:2743] T 89e1eeb24fea483eb63bd98af66e0b6b P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:57:30.310737 21704 raft_consensus.cc:357] T 3b5049c00b0b4dd89c49fa5d13db169e P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.311483 21704 raft_consensus.cc:383] T 3b5049c00b0b4dd89c49fa5d13db169e P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:30.311777 21704 raft_consensus.cc:738] T 3b5049c00b0b4dd89c49fa5d13db169e P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Initialized, Role: FOLLOWER
I20250109 08:57:30.312544 21704 consensus_queue.cc:260] T 3b5049c00b0b4dd89c49fa5d13db169e P 787ddcc824c2490a808fc5a53452dc5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.315464 21704 ts_tablet_manager.cc:1428] T 3b5049c00b0b4dd89c49fa5d13db169e P 787ddcc824c2490a808fc5a53452dc5e: Time spent starting tablet: real 0.008s user 0.003s sys 0.004s
I20250109 08:57:30.316186 21702 tablet_bootstrap.cc:654] T 81d74e7d4472497aac5463a4b47e4fb9 P a2b1a692435241ee988e0bd1661cd4c3: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:30.316416 21704 tablet_bootstrap.cc:492] T 89e1eeb24fea483eb63bd98af66e0b6b P 787ddcc824c2490a808fc5a53452dc5e: Bootstrap starting.
I20250109 08:57:30.316470 21703 tablet_bootstrap.cc:654] T d8493108cba04514bd2236192cef1079 P 8bf6e26e98b14566899578a990b27d47: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:30.322165 21820 consensus_queue.cc:1035] T 363fa693a5114f2bb22f4ab67722a1c6 P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250109 08:57:30.331444 21704 tablet_bootstrap.cc:654] T 89e1eeb24fea483eb63bd98af66e0b6b P 787ddcc824c2490a808fc5a53452dc5e: Neither blocks nor log segments found. Creating new log.
W20250109 08:57:30.338179 21627 consensus_peers.cc:487] T 81d74e7d4472497aac5463a4b47e4fb9 P 787ddcc824c2490a808fc5a53452dc5e -> Peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Couldn't send request to peer a2b1a692435241ee988e0bd1661cd4c3. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:57:30.348035 21702 tablet_bootstrap.cc:492] T 81d74e7d4472497aac5463a4b47e4fb9 P a2b1a692435241ee988e0bd1661cd4c3: No bootstrap required, opened a new log
I20250109 08:57:30.348505 21702 ts_tablet_manager.cc:1397] T 81d74e7d4472497aac5463a4b47e4fb9 P a2b1a692435241ee988e0bd1661cd4c3: Time spent bootstrapping tablet: real 0.052s user 0.018s sys 0.007s
I20250109 08:57:30.353196 21702 raft_consensus.cc:357] T 81d74e7d4472497aac5463a4b47e4fb9 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.355353 21702 raft_consensus.cc:383] T 81d74e7d4472497aac5463a4b47e4fb9 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:30.355621 21702 raft_consensus.cc:738] T 81d74e7d4472497aac5463a4b47e4fb9 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2b1a692435241ee988e0bd1661cd4c3, State: Initialized, Role: FOLLOWER
I20250109 08:57:30.356266 21702 consensus_queue.cc:260] T 81d74e7d4472497aac5463a4b47e4fb9 P a2b1a692435241ee988e0bd1661cd4c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.358361 21702 ts_tablet_manager.cc:1428] T 81d74e7d4472497aac5463a4b47e4fb9 P a2b1a692435241ee988e0bd1661cd4c3: Time spent starting tablet: real 0.007s user 0.004s sys 0.000s
I20250109 08:57:30.359655 21702 tablet_bootstrap.cc:492] T 306c4892bbff4dbfbe5a0645294aa627 P a2b1a692435241ee988e0bd1661cd4c3: Bootstrap starting.
I20250109 08:57:30.366863 21740 raft_consensus.cc:491] T d6f988a231a74f6fa112da9a527d4a0b P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:30.367259 21740 raft_consensus.cc:513] T d6f988a231a74f6fa112da9a527d4a0b P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.368806 21740 leader_election.cc:290] T d6f988a231a74f6fa112da9a527d4a0b P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:30.369545 21589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6f988a231a74f6fa112da9a527d4a0b" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3" is_pre_election: true
I20250109 08:57:30.369930 21663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6f988a231a74f6fa112da9a527d4a0b" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e" is_pre_election: true
I20250109 08:57:30.370210 21589 raft_consensus.cc:2463] T d6f988a231a74f6fa112da9a527d4a0b P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8bf6e26e98b14566899578a990b27d47 in term 0.
W20250109 08:57:30.371001 21477 leader_election.cc:343] T d6f988a231a74f6fa112da9a527d4a0b P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:30.371649 21478 leader_election.cc:304] T d6f988a231a74f6fa112da9a527d4a0b P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3; no voters: 787ddcc824c2490a808fc5a53452dc5e
I20250109 08:57:30.372357 21742 raft_consensus.cc:2798] T d6f988a231a74f6fa112da9a527d4a0b P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:57:30.372743 21742 raft_consensus.cc:491] T d6f988a231a74f6fa112da9a527d4a0b P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:57:30.373040 21742 raft_consensus.cc:3054] T d6f988a231a74f6fa112da9a527d4a0b P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:30.374257 21740 consensus_queue.cc:1035] T 71ac0243d72845659291184835652dde P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:57:30.389673 21702 tablet_bootstrap.cc:654] T 306c4892bbff4dbfbe5a0645294aa627 P a2b1a692435241ee988e0bd1661cd4c3: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:30.390349 21703 tablet_bootstrap.cc:492] T d8493108cba04514bd2236192cef1079 P 8bf6e26e98b14566899578a990b27d47: No bootstrap required, opened a new log
I20250109 08:57:30.390794 21703 ts_tablet_manager.cc:1397] T d8493108cba04514bd2236192cef1079 P 8bf6e26e98b14566899578a990b27d47: Time spent bootstrapping tablet: real 0.106s user 0.012s sys 0.006s
I20250109 08:57:30.391149 21704 tablet_bootstrap.cc:492] T 89e1eeb24fea483eb63bd98af66e0b6b P 787ddcc824c2490a808fc5a53452dc5e: No bootstrap required, opened a new log
I20250109 08:57:30.391595 21704 ts_tablet_manager.cc:1397] T 89e1eeb24fea483eb63bd98af66e0b6b P 787ddcc824c2490a808fc5a53452dc5e: Time spent bootstrapping tablet: real 0.075s user 0.010s sys 0.022s
I20250109 08:57:30.393204 21703 raft_consensus.cc:357] T d8493108cba04514bd2236192cef1079 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.393882 21703 raft_consensus.cc:383] T d8493108cba04514bd2236192cef1079 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:30.394121 21703 raft_consensus.cc:738] T d8493108cba04514bd2236192cef1079 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bf6e26e98b14566899578a990b27d47, State: Initialized, Role: FOLLOWER
I20250109 08:57:30.394706 21704 raft_consensus.cc:357] T 89e1eeb24fea483eb63bd98af66e0b6b P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.394661 21703 consensus_queue.cc:260] T d8493108cba04514bd2236192cef1079 P 8bf6e26e98b14566899578a990b27d47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.395414 21704 raft_consensus.cc:383] T 89e1eeb24fea483eb63bd98af66e0b6b P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:30.394958 21781 consensus_queue.cc:1035] T 71ac0243d72845659291184835652dde P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250109 08:57:30.395674 21704 raft_consensus.cc:738] T 89e1eeb24fea483eb63bd98af66e0b6b P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Initialized, Role: FOLLOWER
I20250109 08:57:30.396456 21704 consensus_queue.cc:260] T 89e1eeb24fea483eb63bd98af66e0b6b P 787ddcc824c2490a808fc5a53452dc5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.398583 21704 ts_tablet_manager.cc:1428] T 89e1eeb24fea483eb63bd98af66e0b6b P 787ddcc824c2490a808fc5a53452dc5e: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20250109 08:57:30.400336 21704 tablet_bootstrap.cc:492] T 306c4892bbff4dbfbe5a0645294aa627 P 787ddcc824c2490a808fc5a53452dc5e: Bootstrap starting.
I20250109 08:57:30.409019 21704 tablet_bootstrap.cc:654] T 306c4892bbff4dbfbe5a0645294aa627 P 787ddcc824c2490a808fc5a53452dc5e: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:30.409611 21742 raft_consensus.cc:513] T d6f988a231a74f6fa112da9a527d4a0b P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.412750 21586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6f988a231a74f6fa112da9a527d4a0b" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3"
I20250109 08:57:30.413244 21703 ts_tablet_manager.cc:1428] T d8493108cba04514bd2236192cef1079 P 8bf6e26e98b14566899578a990b27d47: Time spent starting tablet: real 0.022s user 0.009s sys 0.004s
I20250109 08:57:30.413347 21586 raft_consensus.cc:3054] T d6f988a231a74f6fa112da9a527d4a0b P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:30.419842 21586 raft_consensus.cc:2463] T d6f988a231a74f6fa112da9a527d4a0b P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8bf6e26e98b14566899578a990b27d47 in term 1.
I20250109 08:57:30.420605 21703 tablet_bootstrap.cc:492] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P 8bf6e26e98b14566899578a990b27d47: Bootstrap starting.
I20250109 08:57:30.420850 21478 leader_election.cc:304] T d6f988a231a74f6fa112da9a527d4a0b P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3; no voters:
I20250109 08:57:30.421638 21740 raft_consensus.cc:2798] T d6f988a231a74f6fa112da9a527d4a0b P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:57:30.422237 21740 raft_consensus.cc:695] T d6f988a231a74f6fa112da9a527d4a0b P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Becoming Leader. State: Replica: 8bf6e26e98b14566899578a990b27d47, State: Running, Role: LEADER
I20250109 08:57:30.422989 21740 consensus_queue.cc:237] T d6f988a231a74f6fa112da9a527d4a0b P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.431959 21703 tablet_bootstrap.cc:654] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P 8bf6e26e98b14566899578a990b27d47: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:30.433552 21663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6f988a231a74f6fa112da9a527d4a0b" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e"
W20250109 08:57:30.435089 21477 leader_election.cc:343] T d6f988a231a74f6fa112da9a527d4a0b P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:30.433800 21742 leader_election.cc:290] T d6f988a231a74f6fa112da9a527d4a0b P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 election: Requested vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:30.447402 21702 tablet_bootstrap.cc:492] T 306c4892bbff4dbfbe5a0645294aa627 P a2b1a692435241ee988e0bd1661cd4c3: No bootstrap required, opened a new log
I20250109 08:57:30.447896 21702 ts_tablet_manager.cc:1397] T 306c4892bbff4dbfbe5a0645294aa627 P a2b1a692435241ee988e0bd1661cd4c3: Time spent bootstrapping tablet: real 0.088s user 0.012s sys 0.004s
I20250109 08:57:30.449209 21410 catalog_manager.cc:5526] T d6f988a231a74f6fa112da9a527d4a0b P 8bf6e26e98b14566899578a990b27d47 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8bf6e26e98b14566899578a990b27d47 (127.19.226.193). New cstate: current_term: 1 leader_uuid: "8bf6e26e98b14566899578a990b27d47" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: UNKNOWN } } }
I20250109 08:57:30.450989 21702 raft_consensus.cc:357] T 306c4892bbff4dbfbe5a0645294aa627 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.451687 21702 raft_consensus.cc:383] T 306c4892bbff4dbfbe5a0645294aa627 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:30.451938 21702 raft_consensus.cc:738] T 306c4892bbff4dbfbe5a0645294aa627 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2b1a692435241ee988e0bd1661cd4c3, State: Initialized, Role: FOLLOWER
I20250109 08:57:30.453122 21704 tablet_bootstrap.cc:492] T 306c4892bbff4dbfbe5a0645294aa627 P 787ddcc824c2490a808fc5a53452dc5e: No bootstrap required, opened a new log
I20250109 08:57:30.453859 21704 ts_tablet_manager.cc:1397] T 306c4892bbff4dbfbe5a0645294aa627 P 787ddcc824c2490a808fc5a53452dc5e: Time spent bootstrapping tablet: real 0.054s user 0.010s sys 0.019s
I20250109 08:57:30.455436 21702 consensus_queue.cc:260] T 306c4892bbff4dbfbe5a0645294aa627 P a2b1a692435241ee988e0bd1661cd4c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.456559 21704 raft_consensus.cc:357] T 306c4892bbff4dbfbe5a0645294aa627 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.457280 21704 raft_consensus.cc:383] T 306c4892bbff4dbfbe5a0645294aa627 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:30.457541 21704 raft_consensus.cc:738] T 306c4892bbff4dbfbe5a0645294aa627 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Initialized, Role: FOLLOWER
I20250109 08:57:30.458062 21702 ts_tablet_manager.cc:1428] T 306c4892bbff4dbfbe5a0645294aa627 P a2b1a692435241ee988e0bd1661cd4c3: Time spent starting tablet: real 0.010s user 0.006s sys 0.002s
I20250109 08:57:30.459715 21702 tablet_bootstrap.cc:492] T 5ac5ffe3cd65463fb448e819e3978841 P a2b1a692435241ee988e0bd1661cd4c3: Bootstrap starting.
I20250109 08:57:30.460314 21703 tablet_bootstrap.cc:492] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P 8bf6e26e98b14566899578a990b27d47: No bootstrap required, opened a new log
I20250109 08:57:30.460752 21703 ts_tablet_manager.cc:1397] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P 8bf6e26e98b14566899578a990b27d47: Time spent bootstrapping tablet: real 0.040s user 0.018s sys 0.015s
I20250109 08:57:30.458263 21704 consensus_queue.cc:260] T 306c4892bbff4dbfbe5a0645294aa627 P 787ddcc824c2490a808fc5a53452dc5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.463225 21703 raft_consensus.cc:357] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.465042 21704 ts_tablet_manager.cc:1428] T 306c4892bbff4dbfbe5a0645294aa627 P 787ddcc824c2490a808fc5a53452dc5e: Time spent starting tablet: real 0.011s user 0.006s sys 0.000s
I20250109 08:57:30.463805 21703 raft_consensus.cc:383] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:30.465471 21703 raft_consensus.cc:738] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bf6e26e98b14566899578a990b27d47, State: Initialized, Role: FOLLOWER
I20250109 08:57:30.465605 21702 tablet_bootstrap.cc:654] T 5ac5ffe3cd65463fb448e819e3978841 P a2b1a692435241ee988e0bd1661cd4c3: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:30.466567 21704 tablet_bootstrap.cc:492] T e38ec0ef84af4b798073eafd4d210c35 P 787ddcc824c2490a808fc5a53452dc5e: Bootstrap starting.
I20250109 08:57:30.469163 21703 consensus_queue.cc:260] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P 8bf6e26e98b14566899578a990b27d47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.471293 21703 ts_tablet_manager.cc:1428] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P 8bf6e26e98b14566899578a990b27d47: Time spent starting tablet: real 0.010s user 0.004s sys 0.001s
I20250109 08:57:30.472992 21703 tablet_bootstrap.cc:492] T d355d166de534fb4a50c9f48f2df7469 P 8bf6e26e98b14566899578a990b27d47: Bootstrap starting.
I20250109 08:57:30.474978 21704 tablet_bootstrap.cc:654] T e38ec0ef84af4b798073eafd4d210c35 P 787ddcc824c2490a808fc5a53452dc5e: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:30.476842 21702 tablet_bootstrap.cc:492] T 5ac5ffe3cd65463fb448e819e3978841 P a2b1a692435241ee988e0bd1661cd4c3: No bootstrap required, opened a new log
I20250109 08:57:30.477253 21702 ts_tablet_manager.cc:1397] T 5ac5ffe3cd65463fb448e819e3978841 P a2b1a692435241ee988e0bd1661cd4c3: Time spent bootstrapping tablet: real 0.018s user 0.008s sys 0.002s
I20250109 08:57:30.478785 21703 tablet_bootstrap.cc:654] T d355d166de534fb4a50c9f48f2df7469 P 8bf6e26e98b14566899578a990b27d47: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:30.479693 21702 raft_consensus.cc:357] T 5ac5ffe3cd65463fb448e819e3978841 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.480404 21702 raft_consensus.cc:383] T 5ac5ffe3cd65463fb448e819e3978841 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:30.480664 21702 raft_consensus.cc:738] T 5ac5ffe3cd65463fb448e819e3978841 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2b1a692435241ee988e0bd1661cd4c3, State: Initialized, Role: FOLLOWER
I20250109 08:57:30.481333 21702 consensus_queue.cc:260] T 5ac5ffe3cd65463fb448e819e3978841 P a2b1a692435241ee988e0bd1661cd4c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.486341 21702 ts_tablet_manager.cc:1428] T 5ac5ffe3cd65463fb448e819e3978841 P a2b1a692435241ee988e0bd1661cd4c3: Time spent starting tablet: real 0.009s user 0.001s sys 0.004s
I20250109 08:57:30.487298 21702 tablet_bootstrap.cc:492] T 180b4d539a144a38905bf340ecb24ef4 P a2b1a692435241ee988e0bd1661cd4c3: Bootstrap starting.
I20250109 08:57:30.494277 21709 raft_consensus.cc:491] T 3b5049c00b0b4dd89c49fa5d13db169e P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:30.494778 21709 raft_consensus.cc:513] T 3b5049c00b0b4dd89c49fa5d13db169e P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.496771 21709 leader_election.cc:290] T 3b5049c00b0b4dd89c49fa5d13db169e P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:30.498656 21663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b5049c00b0b4dd89c49fa5d13db169e" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e" is_pre_election: true
I20250109 08:57:30.499338 21663 raft_consensus.cc:2463] T 3b5049c00b0b4dd89c49fa5d13db169e P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2b1a692435241ee988e0bd1661cd4c3 in term 0.
I20250109 08:57:30.500818 21512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b5049c00b0b4dd89c49fa5d13db169e" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47" is_pre_election: true
I20250109 08:57:30.501286 21702 tablet_bootstrap.cc:654] T 180b4d539a144a38905bf340ecb24ef4 P a2b1a692435241ee988e0bd1661cd4c3: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:30.501475 21512 raft_consensus.cc:2463] T 3b5049c00b0b4dd89c49fa5d13db169e P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2b1a692435241ee988e0bd1661cd4c3 in term 0.
I20250109 08:57:30.502821 21552 leader_election.cc:304] T 3b5049c00b0b4dd89c49fa5d13db169e P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3; no voters:
I20250109 08:57:30.505014 21709 raft_consensus.cc:2798] T 3b5049c00b0b4dd89c49fa5d13db169e P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:57:30.505434 21709 raft_consensus.cc:491] T 3b5049c00b0b4dd89c49fa5d13db169e P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:57:30.505806 21709 raft_consensus.cc:3054] T 3b5049c00b0b4dd89c49fa5d13db169e P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:30.512173 21709 raft_consensus.cc:513] T 3b5049c00b0b4dd89c49fa5d13db169e P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.516019 21663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b5049c00b0b4dd89c49fa5d13db169e" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e"
I20250109 08:57:30.516677 21663 raft_consensus.cc:3054] T 3b5049c00b0b4dd89c49fa5d13db169e P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:30.517203 21512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b5049c00b0b4dd89c49fa5d13db169e" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47"
I20250109 08:57:30.517817 21512 raft_consensus.cc:3054] T 3b5049c00b0b4dd89c49fa5d13db169e P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:30.524087 21663 raft_consensus.cc:2463] T 3b5049c00b0b4dd89c49fa5d13db169e P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2b1a692435241ee988e0bd1661cd4c3 in term 1.
I20250109 08:57:30.525694 21552 leader_election.cc:304] T 3b5049c00b0b4dd89c49fa5d13db169e P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, a2b1a692435241ee988e0bd1661cd4c3; no voters:
I20250109 08:57:30.527052 21820 raft_consensus.cc:2798] T 3b5049c00b0b4dd89c49fa5d13db169e P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:57:30.527783 21820 raft_consensus.cc:695] T 3b5049c00b0b4dd89c49fa5d13db169e P a2b1a692435241ee988e0bd1661cd4c3 [term 1 LEADER]: Becoming Leader. State: Replica: a2b1a692435241ee988e0bd1661cd4c3, State: Running, Role: LEADER
I20250109 08:57:30.529043 21512 raft_consensus.cc:2463] T 3b5049c00b0b4dd89c49fa5d13db169e P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2b1a692435241ee988e0bd1661cd4c3 in term 1.
I20250109 08:57:30.528993 21820 consensus_queue.cc:237] T 3b5049c00b0b4dd89c49fa5d13db169e P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.533809 21709 leader_election.cc:290] T 3b5049c00b0b4dd89c49fa5d13db169e P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 election: Requested vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:30.536448 21704 tablet_bootstrap.cc:492] T e38ec0ef84af4b798073eafd4d210c35 P 787ddcc824c2490a808fc5a53452dc5e: No bootstrap required, opened a new log
I20250109 08:57:30.536898 21704 ts_tablet_manager.cc:1397] T e38ec0ef84af4b798073eafd4d210c35 P 787ddcc824c2490a808fc5a53452dc5e: Time spent bootstrapping tablet: real 0.070s user 0.023s sys 0.004s
I20250109 08:57:30.538156 21703 tablet_bootstrap.cc:492] T d355d166de534fb4a50c9f48f2df7469 P 8bf6e26e98b14566899578a990b27d47: No bootstrap required, opened a new log
I20250109 08:57:30.538573 21703 ts_tablet_manager.cc:1397] T d355d166de534fb4a50c9f48f2df7469 P 8bf6e26e98b14566899578a990b27d47: Time spent bootstrapping tablet: real 0.066s user 0.015s sys 0.037s
I20250109 08:57:30.539695 21704 raft_consensus.cc:357] T e38ec0ef84af4b798073eafd4d210c35 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.540395 21704 raft_consensus.cc:383] T e38ec0ef84af4b798073eafd4d210c35 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:30.540654 21704 raft_consensus.cc:738] T e38ec0ef84af4b798073eafd4d210c35 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Initialized, Role: FOLLOWER
I20250109 08:57:30.541005 21703 raft_consensus.cc:357] T d355d166de534fb4a50c9f48f2df7469 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.541214 21704 consensus_queue.cc:260] T e38ec0ef84af4b798073eafd4d210c35 P 787ddcc824c2490a808fc5a53452dc5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.541678 21703 raft_consensus.cc:383] T d355d166de534fb4a50c9f48f2df7469 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:30.542054 21703 raft_consensus.cc:738] T d355d166de534fb4a50c9f48f2df7469 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bf6e26e98b14566899578a990b27d47, State: Initialized, Role: FOLLOWER
I20250109 08:57:30.542022 21412 catalog_manager.cc:5526] T 3b5049c00b0b4dd89c49fa5d13db169e P a2b1a692435241ee988e0bd1661cd4c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194). New cstate: current_term: 1 leader_uuid: "a2b1a692435241ee988e0bd1661cd4c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: UNKNOWN } } }
I20250109 08:57:30.542765 21703 consensus_queue.cc:260] T d355d166de534fb4a50c9f48f2df7469 P 8bf6e26e98b14566899578a990b27d47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.543924 21702 tablet_bootstrap.cc:492] T 180b4d539a144a38905bf340ecb24ef4 P a2b1a692435241ee988e0bd1661cd4c3: No bootstrap required, opened a new log
I20250109 08:57:30.544338 21702 ts_tablet_manager.cc:1397] T 180b4d539a144a38905bf340ecb24ef4 P a2b1a692435241ee988e0bd1661cd4c3: Time spent bootstrapping tablet: real 0.057s user 0.021s sys 0.009s
I20250109 08:57:30.546571 21702 raft_consensus.cc:357] T 180b4d539a144a38905bf340ecb24ef4 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.547197 21702 raft_consensus.cc:383] T 180b4d539a144a38905bf340ecb24ef4 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:30.547433 21702 raft_consensus.cc:738] T 180b4d539a144a38905bf340ecb24ef4 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2b1a692435241ee988e0bd1661cd4c3, State: Initialized, Role: FOLLOWER
I20250109 08:57:30.547808 21703 ts_tablet_manager.cc:1428] T d355d166de534fb4a50c9f48f2df7469 P 8bf6e26e98b14566899578a990b27d47: Time spent starting tablet: real 0.009s user 0.005s sys 0.001s
I20250109 08:57:30.548727 21703 tablet_bootstrap.cc:492] T 89e1eeb24fea483eb63bd98af66e0b6b P 8bf6e26e98b14566899578a990b27d47: Bootstrap starting.
I20250109 08:57:30.548933 21704 ts_tablet_manager.cc:1428] T e38ec0ef84af4b798073eafd4d210c35 P 787ddcc824c2490a808fc5a53452dc5e: Time spent starting tablet: real 0.012s user 0.006s sys 0.000s
I20250109 08:57:30.549768 21704 tablet_bootstrap.cc:492] T d6f988a231a74f6fa112da9a527d4a0b P 787ddcc824c2490a808fc5a53452dc5e: Bootstrap starting.
I20250109 08:57:30.554476 21703 tablet_bootstrap.cc:654] T 89e1eeb24fea483eb63bd98af66e0b6b P 8bf6e26e98b14566899578a990b27d47: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:30.555431 21704 tablet_bootstrap.cc:654] T d6f988a231a74f6fa112da9a527d4a0b P 787ddcc824c2490a808fc5a53452dc5e: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:30.556536 21709 raft_consensus.cc:491] T 81d74e7d4472497aac5463a4b47e4fb9 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:30.556977 21709 raft_consensus.cc:513] T 81d74e7d4472497aac5463a4b47e4fb9 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.558818 21709 leader_election.cc:290] T 81d74e7d4472497aac5463a4b47e4fb9 P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:30.547958 21702 consensus_queue.cc:260] T 180b4d539a144a38905bf340ecb24ef4 P a2b1a692435241ee988e0bd1661cd4c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.561798 21702 ts_tablet_manager.cc:1428] T 180b4d539a144a38905bf340ecb24ef4 P a2b1a692435241ee988e0bd1661cd4c3: Time spent starting tablet: real 0.017s user 0.001s sys 0.003s
I20250109 08:57:30.562948 21702 tablet_bootstrap.cc:492] T e38ec0ef84af4b798073eafd4d210c35 P a2b1a692435241ee988e0bd1661cd4c3: Bootstrap starting.
I20250109 08:57:30.563839 21663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81d74e7d4472497aac5463a4b47e4fb9" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e" is_pre_election: true
I20250109 08:57:30.567333 21704 tablet_bootstrap.cc:492] T d6f988a231a74f6fa112da9a527d4a0b P 787ddcc824c2490a808fc5a53452dc5e: No bootstrap required, opened a new log
I20250109 08:57:30.567761 21704 ts_tablet_manager.cc:1397] T d6f988a231a74f6fa112da9a527d4a0b P 787ddcc824c2490a808fc5a53452dc5e: Time spent bootstrapping tablet: real 0.018s user 0.011s sys 0.004s
I20250109 08:57:30.568917 21702 tablet_bootstrap.cc:654] T e38ec0ef84af4b798073eafd4d210c35 P a2b1a692435241ee988e0bd1661cd4c3: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:30.569464 21512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81d74e7d4472497aac5463a4b47e4fb9" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47" is_pre_election: true
I20250109 08:57:30.570099 21704 raft_consensus.cc:357] T d6f988a231a74f6fa112da9a527d4a0b P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.570749 21704 raft_consensus.cc:383] T d6f988a231a74f6fa112da9a527d4a0b P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:30.571002 21704 raft_consensus.cc:738] T d6f988a231a74f6fa112da9a527d4a0b P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Initialized, Role: FOLLOWER
I20250109 08:57:30.571586 21704 consensus_queue.cc:260] T d6f988a231a74f6fa112da9a527d4a0b P 787ddcc824c2490a808fc5a53452dc5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.572602 21589 raft_consensus.cc:3054] T 5ac5ffe3cd65463fb448e819e3978841 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:30.573606 21703 tablet_bootstrap.cc:492] T 89e1eeb24fea483eb63bd98af66e0b6b P 8bf6e26e98b14566899578a990b27d47: No bootstrap required, opened a new log
I20250109 08:57:30.573935 21704 ts_tablet_manager.cc:1428] T d6f988a231a74f6fa112da9a527d4a0b P 787ddcc824c2490a808fc5a53452dc5e: Time spent starting tablet: real 0.006s user 0.006s sys 0.001s
I20250109 08:57:30.574011 21703 ts_tablet_manager.cc:1397] T 89e1eeb24fea483eb63bd98af66e0b6b P 8bf6e26e98b14566899578a990b27d47: Time spent bootstrapping tablet: real 0.025s user 0.007s sys 0.009s
I20250109 08:57:30.574846 21704 tablet_bootstrap.cc:492] T ddd14cdbefd64936960e9ffb5b38b5f2 P 787ddcc824c2490a808fc5a53452dc5e: Bootstrap starting.
I20250109 08:57:30.576468 21703 raft_consensus.cc:357] T 89e1eeb24fea483eb63bd98af66e0b6b P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.576753 21552 leader_election.cc:304] T 81d74e7d4472497aac5463a4b47e4fb9 P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2b1a692435241ee988e0bd1661cd4c3; no voters: 787ddcc824c2490a808fc5a53452dc5e, 8bf6e26e98b14566899578a990b27d47
I20250109 08:57:30.577117 21703 raft_consensus.cc:383] T 89e1eeb24fea483eb63bd98af66e0b6b P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:30.577391 21703 raft_consensus.cc:738] T 89e1eeb24fea483eb63bd98af66e0b6b P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bf6e26e98b14566899578a990b27d47, State: Initialized, Role: FOLLOWER
I20250109 08:57:30.577878 21820 raft_consensus.cc:2743] T 81d74e7d4472497aac5463a4b47e4fb9 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:57:30.578558 21703 consensus_queue.cc:260] T 89e1eeb24fea483eb63bd98af66e0b6b P 8bf6e26e98b14566899578a990b27d47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.581535 21704 tablet_bootstrap.cc:654] T ddd14cdbefd64936960e9ffb5b38b5f2 P 787ddcc824c2490a808fc5a53452dc5e: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:30.582803 21710 raft_consensus.cc:491] T d8493108cba04514bd2236192cef1079 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:30.583129 21758 consensus_queue.cc:1035] T 5ac5ffe3cd65463fb448e819e3978841 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:57:30.583242 21710 raft_consensus.cc:513] T d8493108cba04514bd2236192cef1079 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.584700 21703 ts_tablet_manager.cc:1428] T 89e1eeb24fea483eb63bd98af66e0b6b P 8bf6e26e98b14566899578a990b27d47: Time spent starting tablet: real 0.010s user 0.007s sys 0.000s
I20250109 08:57:30.585186 21710 leader_election.cc:290] T d8493108cba04514bd2236192cef1079 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
I20250109 08:57:30.585530 21703 tablet_bootstrap.cc:492] T ddd14cdbefd64936960e9ffb5b38b5f2 P 8bf6e26e98b14566899578a990b27d47: Bootstrap starting.
I20250109 08:57:30.586860 21588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8493108cba04514bd2236192cef1079" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3" is_pre_election: true
I20250109 08:57:30.588003 21588 raft_consensus.cc:2463] T d8493108cba04514bd2236192cef1079 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 0.
I20250109 08:57:30.589108 21627 leader_election.cc:304] T d8493108cba04514bd2236192cef1079 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, a2b1a692435241ee988e0bd1661cd4c3; no voters:
I20250109 08:57:30.590247 21710 raft_consensus.cc:2798] T d8493108cba04514bd2236192cef1079 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:57:30.590575 21710 raft_consensus.cc:491] T d8493108cba04514bd2236192cef1079 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:57:30.590898 21710 raft_consensus.cc:3054] T d8493108cba04514bd2236192cef1079 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:30.591128 21703 tablet_bootstrap.cc:654] T ddd14cdbefd64936960e9ffb5b38b5f2 P 8bf6e26e98b14566899578a990b27d47: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:30.597340 21740 raft_consensus.cc:491] T e38ec0ef84af4b798073eafd4d210c35 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:30.597772 21740 raft_consensus.cc:513] T e38ec0ef84af4b798073eafd4d210c35 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.598816 21710 raft_consensus.cc:513] T d8493108cba04514bd2236192cef1079 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.599737 21740 leader_election.cc:290] T e38ec0ef84af4b798073eafd4d210c35 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:30.601469 21663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e38ec0ef84af4b798073eafd4d210c35" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e" is_pre_election: true
I20250109 08:57:30.601902 21514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8493108cba04514bd2236192cef1079" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47"
I20250109 08:57:30.602090 21663 raft_consensus.cc:2463] T e38ec0ef84af4b798073eafd4d210c35 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8bf6e26e98b14566899578a990b27d47 in term 0.
I20250109 08:57:30.602530 21514 raft_consensus.cc:3054] T d8493108cba04514bd2236192cef1079 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:30.603440 21477 leader_election.cc:304] T e38ec0ef84af4b798073eafd4d210c35 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, 8bf6e26e98b14566899578a990b27d47; no voters:
I20250109 08:57:30.604622 21740 raft_consensus.cc:2798] T e38ec0ef84af4b798073eafd4d210c35 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:57:30.604960 21740 raft_consensus.cc:491] T e38ec0ef84af4b798073eafd4d210c35 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:57:30.605281 21740 raft_consensus.cc:3054] T e38ec0ef84af4b798073eafd4d210c35 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:30.606590 21512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8493108cba04514bd2236192cef1079" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47" is_pre_election: true
I20250109 08:57:30.608632 21758 raft_consensus.cc:491] T 306c4892bbff4dbfbe5a0645294aa627 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:30.609059 21758 raft_consensus.cc:513] T 306c4892bbff4dbfbe5a0645294aa627 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.610127 21589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e38ec0ef84af4b798073eafd4d210c35" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3" is_pre_election: true
W20250109 08:57:30.611207 21478 leader_election.cc:343] T e38ec0ef84af4b798073eafd4d210c35 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:30.611922 21758 leader_election.cc:290] T 306c4892bbff4dbfbe5a0645294aa627 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
I20250109 08:57:30.612250 21514 raft_consensus.cc:2463] T d8493108cba04514bd2236192cef1079 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 1.
I20250109 08:57:30.613356 21626 leader_election.cc:304] T d8493108cba04514bd2236192cef1079 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, 8bf6e26e98b14566899578a990b27d47; no voters:
I20250109 08:57:30.613931 21589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "306c4892bbff4dbfbe5a0645294aa627" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3" is_pre_election: true
I20250109 08:57:30.614487 21589 raft_consensus.cc:2463] T 306c4892bbff4dbfbe5a0645294aa627 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 0.
I20250109 08:57:30.615051 21758 raft_consensus.cc:2798] T d8493108cba04514bd2236192cef1079 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:57:30.613344 21512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "306c4892bbff4dbfbe5a0645294aa627" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47" is_pre_election: true
I20250109 08:57:30.615825 21512 raft_consensus.cc:2463] T 306c4892bbff4dbfbe5a0645294aa627 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 0.
I20250109 08:57:30.616412 21820 consensus_queue.cc:1035] T 0e06021cd6624e50af2c914399ebd6c4 P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:57:30.615491 21627 leader_election.cc:304] T 306c4892bbff4dbfbe5a0645294aa627 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, a2b1a692435241ee988e0bd1661cd4c3; no voters:
I20250109 08:57:30.616396 21710 leader_election.cc:290] T d8493108cba04514bd2236192cef1079 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 election: Requested vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
I20250109 08:57:30.618420 21732 raft_consensus.cc:2798] T 306c4892bbff4dbfbe5a0645294aa627 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:57:30.618773 21732 raft_consensus.cc:491] T 306c4892bbff4dbfbe5a0645294aa627 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:57:30.619087 21732 raft_consensus.cc:3054] T 306c4892bbff4dbfbe5a0645294aa627 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:30.625689 21588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8493108cba04514bd2236192cef1079" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3"
I20250109 08:57:30.626232 21588 raft_consensus.cc:3054] T d8493108cba04514bd2236192cef1079 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:30.616348 21758 raft_consensus.cc:695] T d8493108cba04514bd2236192cef1079 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Becoming Leader. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Running, Role: LEADER
I20250109 08:57:30.629928 21758 consensus_queue.cc:237] T d8493108cba04514bd2236192cef1079 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.636307 21588 raft_consensus.cc:2463] T d8493108cba04514bd2236192cef1079 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 1.
I20250109 08:57:30.624668 21732 raft_consensus.cc:513] T 306c4892bbff4dbfbe5a0645294aa627 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.628959 21740 raft_consensus.cc:513] T e38ec0ef84af4b798073eafd4d210c35 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.642782 21740 leader_election.cc:290] T e38ec0ef84af4b798073eafd4d210c35 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 election: Requested vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:30.647390 21514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "306c4892bbff4dbfbe5a0645294aa627" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47"
I20250109 08:57:30.648001 21514 raft_consensus.cc:3054] T 306c4892bbff4dbfbe5a0645294aa627 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:30.651530 21704 tablet_bootstrap.cc:492] T ddd14cdbefd64936960e9ffb5b38b5f2 P 787ddcc824c2490a808fc5a53452dc5e: No bootstrap required, opened a new log
W20250109 08:57:30.653904 21552 consensus_peers.cc:487] T 0e06021cd6624e50af2c914399ebd6c4 P a2b1a692435241ee988e0bd1661cd4c3 -> Peer 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731): Couldn't send request to peer 787ddcc824c2490a808fc5a53452dc5e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:57:30.656009 21663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e38ec0ef84af4b798073eafd4d210c35" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e"
I20250109 08:57:30.651942 21409 catalog_manager.cc:5526] T d8493108cba04514bd2236192cef1079 P 787ddcc824c2490a808fc5a53452dc5e reported cstate change: term changed from 0 to 1, leader changed from <none> to 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195). New cstate: current_term: 1 leader_uuid: "787ddcc824c2490a808fc5a53452dc5e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: HEALTHY } } }
I20250109 08:57:30.656616 21663 raft_consensus.cc:3054] T e38ec0ef84af4b798073eafd4d210c35 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:30.657126 21732 leader_election.cc:290] T 306c4892bbff4dbfbe5a0645294aa627 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 election: Requested vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
I20250109 08:57:30.658536 21704 ts_tablet_manager.cc:1397] T ddd14cdbefd64936960e9ffb5b38b5f2 P 787ddcc824c2490a808fc5a53452dc5e: Time spent bootstrapping tablet: real 0.081s user 0.012s sys 0.016s
I20250109 08:57:30.658560 21588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "306c4892bbff4dbfbe5a0645294aa627" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3"
I20250109 08:57:30.660946 21703 tablet_bootstrap.cc:492] T ddd14cdbefd64936960e9ffb5b38b5f2 P 8bf6e26e98b14566899578a990b27d47: No bootstrap required, opened a new log
I20250109 08:57:30.662174 21588 raft_consensus.cc:3054] T 306c4892bbff4dbfbe5a0645294aa627 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:30.661981 21703 ts_tablet_manager.cc:1397] T ddd14cdbefd64936960e9ffb5b38b5f2 P 8bf6e26e98b14566899578a990b27d47: Time spent bootstrapping tablet: real 0.077s user 0.030s sys 0.005s
I20250109 08:57:30.666814 21589 raft_consensus.cc:3054] T 180b4d539a144a38905bf340ecb24ef4 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:30.670970 21663 raft_consensus.cc:2463] T e38ec0ef84af4b798073eafd4d210c35 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8bf6e26e98b14566899578a990b27d47 in term 1.
I20250109 08:57:30.671999 21477 leader_election.cc:304] T e38ec0ef84af4b798073eafd4d210c35 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, 8bf6e26e98b14566899578a990b27d47; no voters:
I20250109 08:57:30.673025 21704 raft_consensus.cc:357] T ddd14cdbefd64936960e9ffb5b38b5f2 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.673696 21704 raft_consensus.cc:383] T ddd14cdbefd64936960e9ffb5b38b5f2 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:30.673947 21704 raft_consensus.cc:738] T ddd14cdbefd64936960e9ffb5b38b5f2 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Initialized, Role: FOLLOWER
I20250109 08:57:30.674561 21703 raft_consensus.cc:357] T ddd14cdbefd64936960e9ffb5b38b5f2 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.675194 21703 raft_consensus.cc:383] T ddd14cdbefd64936960e9ffb5b38b5f2 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:30.675365 21703 raft_consensus.cc:738] T ddd14cdbefd64936960e9ffb5b38b5f2 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bf6e26e98b14566899578a990b27d47, State: Initialized, Role: FOLLOWER
I20250109 08:57:30.675761 21703 consensus_queue.cc:260] T ddd14cdbefd64936960e9ffb5b38b5f2 P 8bf6e26e98b14566899578a990b27d47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.677176 21732 consensus_queue.cc:1035] T 5ac5ffe3cd65463fb448e819e3978841 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:57:30.678139 21702 tablet_bootstrap.cc:492] T e38ec0ef84af4b798073eafd4d210c35 P a2b1a692435241ee988e0bd1661cd4c3: No bootstrap required, opened a new log
I20250109 08:57:30.678566 21702 ts_tablet_manager.cc:1397] T e38ec0ef84af4b798073eafd4d210c35 P a2b1a692435241ee988e0bd1661cd4c3: Time spent bootstrapping tablet: real 0.116s user 0.017s sys 0.017s
I20250109 08:57:30.674495 21704 consensus_queue.cc:260] T ddd14cdbefd64936960e9ffb5b38b5f2 P 787ddcc824c2490a808fc5a53452dc5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.677117 21740 raft_consensus.cc:2798] T e38ec0ef84af4b798073eafd4d210c35 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:57:30.680621 21740 raft_consensus.cc:695] T e38ec0ef84af4b798073eafd4d210c35 P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Becoming Leader. State: Replica: 8bf6e26e98b14566899578a990b27d47, State: Running, Role: LEADER
I20250109 08:57:30.681402 21740 consensus_queue.cc:237] T e38ec0ef84af4b798073eafd4d210c35 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.682622 21586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e38ec0ef84af4b798073eafd4d210c35" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3"
W20250109 08:57:30.689709 21478 leader_election.cc:343] T e38ec0ef84af4b798073eafd4d210c35 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:30.684118 21514 raft_consensus.cc:2463] T 306c4892bbff4dbfbe5a0645294aa627 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 1.
I20250109 08:57:30.694789 21626 leader_election.cc:304] T 306c4892bbff4dbfbe5a0645294aa627 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, 8bf6e26e98b14566899578a990b27d47; no voters:
I20250109 08:57:30.695636 21732 raft_consensus.cc:2798] T 306c4892bbff4dbfbe5a0645294aa627 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:57:30.701866 21704 ts_tablet_manager.cc:1428] T ddd14cdbefd64936960e9ffb5b38b5f2 P 787ddcc824c2490a808fc5a53452dc5e: Time spent starting tablet: real 0.043s user 0.003s sys 0.002s
I20250109 08:57:30.704100 21842 raft_consensus.cc:491] T ddd14cdbefd64936960e9ffb5b38b5f2 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:30.704560 21842 raft_consensus.cc:513] T ddd14cdbefd64936960e9ffb5b38b5f2 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.706737 21842 leader_election.cc:290] T ddd14cdbefd64936960e9ffb5b38b5f2 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:30.707142 21732 raft_consensus.cc:695] T 306c4892bbff4dbfbe5a0645294aa627 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Becoming Leader. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Running, Role: LEADER
I20250109 08:57:30.704710 21742 consensus_queue.cc:1035] T 180b4d539a144a38905bf340ecb24ef4 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:57:30.693363 21702 raft_consensus.cc:357] T e38ec0ef84af4b798073eafd4d210c35 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.708715 21702 raft_consensus.cc:383] T e38ec0ef84af4b798073eafd4d210c35 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:30.709798 21702 raft_consensus.cc:738] T e38ec0ef84af4b798073eafd4d210c35 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2b1a692435241ee988e0bd1661cd4c3, State: Initialized, Role: FOLLOWER
I20250109 08:57:30.710296 21703 ts_tablet_manager.cc:1428] T ddd14cdbefd64936960e9ffb5b38b5f2 P 8bf6e26e98b14566899578a990b27d47: Time spent starting tablet: real 0.047s user 0.004s sys 0.001s
I20250109 08:57:30.708981 21732 consensus_queue.cc:237] T 306c4892bbff4dbfbe5a0645294aa627 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.711153 21702 consensus_queue.cc:260] T e38ec0ef84af4b798073eafd4d210c35 P a2b1a692435241ee988e0bd1661cd4c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.712109 21703 tablet_bootstrap.cc:492] T 75edb5d04fa14201ad711c1d95c81f64 P 8bf6e26e98b14566899578a990b27d47: Bootstrap starting.
I20250109 08:57:30.714921 21702 ts_tablet_manager.cc:1428] T e38ec0ef84af4b798073eafd4d210c35 P a2b1a692435241ee988e0bd1661cd4c3: Time spent starting tablet: real 0.036s user 0.005s sys 0.001s
I20250109 08:57:30.716641 21702 tablet_bootstrap.cc:492] T 75edb5d04fa14201ad711c1d95c81f64 P a2b1a692435241ee988e0bd1661cd4c3: Bootstrap starting.
I20250109 08:57:30.717022 21663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ddd14cdbefd64936960e9ffb5b38b5f2" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e" is_pre_election: true
I20250109 08:57:30.718159 21663 raft_consensus.cc:2463] T ddd14cdbefd64936960e9ffb5b38b5f2 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8bf6e26e98b14566899578a990b27d47 in term 0.
I20250109 08:57:30.709205 21742 consensus_queue.cc:1035] T 180b4d539a144a38905bf340ecb24ef4 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:57:30.719784 21703 tablet_bootstrap.cc:654] T 75edb5d04fa14201ad711c1d95c81f64 P 8bf6e26e98b14566899578a990b27d47: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:30.709868 21704 tablet_bootstrap.cc:492] T 0e06021cd6624e50af2c914399ebd6c4 P 787ddcc824c2490a808fc5a53452dc5e: Bootstrap starting.
I20250109 08:57:30.707154 21588 raft_consensus.cc:2463] T 306c4892bbff4dbfbe5a0645294aa627 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 1.
I20250109 08:57:30.727656 21704 tablet_bootstrap.cc:654] T 0e06021cd6624e50af2c914399ebd6c4 P 787ddcc824c2490a808fc5a53452dc5e: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:30.728091 21412 catalog_manager.cc:5526] T 306c4892bbff4dbfbe5a0645294aa627 P 787ddcc824c2490a808fc5a53452dc5e reported cstate change: term changed from 0 to 1, leader changed from <none> to 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195). New cstate: current_term: 1 leader_uuid: "787ddcc824c2490a808fc5a53452dc5e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: HEALTHY } } }
I20250109 08:57:30.729614 21702 tablet_bootstrap.cc:654] T 75edb5d04fa14201ad711c1d95c81f64 P a2b1a692435241ee988e0bd1661cd4c3: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:30.734467 21477 leader_election.cc:304] T ddd14cdbefd64936960e9ffb5b38b5f2 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, 8bf6e26e98b14566899578a990b27d47; no voters:
I20250109 08:57:30.735193 21842 raft_consensus.cc:2798] T ddd14cdbefd64936960e9ffb5b38b5f2 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:57:30.735567 21842 raft_consensus.cc:491] T ddd14cdbefd64936960e9ffb5b38b5f2 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:57:30.735669 21704 tablet_bootstrap.cc:492] T 0e06021cd6624e50af2c914399ebd6c4 P 787ddcc824c2490a808fc5a53452dc5e: No bootstrap required, opened a new log
I20250109 08:57:30.736040 21842 raft_consensus.cc:3054] T ddd14cdbefd64936960e9ffb5b38b5f2 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:30.736194 21704 ts_tablet_manager.cc:1397] T 0e06021cd6624e50af2c914399ebd6c4 P 787ddcc824c2490a808fc5a53452dc5e: Time spent bootstrapping tablet: real 0.027s user 0.004s sys 0.008s
I20250109 08:57:30.743856 21704 raft_consensus.cc:357] T 0e06021cd6624e50af2c914399ebd6c4 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.744252 21586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ddd14cdbefd64936960e9ffb5b38b5f2" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3" is_pre_election: true
I20250109 08:57:30.744568 21704 raft_consensus.cc:383] T 0e06021cd6624e50af2c914399ebd6c4 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:30.744838 21704 raft_consensus.cc:738] T 0e06021cd6624e50af2c914399ebd6c4 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Initialized, Role: FOLLOWER
W20250109 08:57:30.745409 21478 leader_election.cc:343] T ddd14cdbefd64936960e9ffb5b38b5f2 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:30.745492 21704 consensus_queue.cc:260] T 0e06021cd6624e50af2c914399ebd6c4 P 787ddcc824c2490a808fc5a53452dc5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.749153 21704 ts_tablet_manager.cc:1428] T 0e06021cd6624e50af2c914399ebd6c4 P 787ddcc824c2490a808fc5a53452dc5e: Time spent starting tablet: real 0.012s user 0.004s sys 0.003s
I20250109 08:57:30.749785 21704 tablet_bootstrap.cc:492] T 0cc4fcda876242c396b804edebebb01f P 787ddcc824c2490a808fc5a53452dc5e: Bootstrap starting.
I20250109 08:57:30.750918 21842 raft_consensus.cc:513] T ddd14cdbefd64936960e9ffb5b38b5f2 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.753945 21704 tablet_bootstrap.cc:654] T 0cc4fcda876242c396b804edebebb01f P 787ddcc824c2490a808fc5a53452dc5e: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:30.758693 21842 leader_election.cc:290] T ddd14cdbefd64936960e9ffb5b38b5f2 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 election: Requested vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:30.761967 21662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ddd14cdbefd64936960e9ffb5b38b5f2" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e"
I20250109 08:57:30.762617 21662 raft_consensus.cc:3054] T ddd14cdbefd64936960e9ffb5b38b5f2 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:30.768896 21820 raft_consensus.cc:491] T e38ec0ef84af4b798073eafd4d210c35 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:30.769405 21820 raft_consensus.cc:513] T e38ec0ef84af4b798073eafd4d210c35 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.771827 21820 leader_election.cc:290] T e38ec0ef84af4b798073eafd4d210c35 P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:30.771394 21411 catalog_manager.cc:5526] T e38ec0ef84af4b798073eafd4d210c35 P 8bf6e26e98b14566899578a990b27d47 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8bf6e26e98b14566899578a990b27d47 (127.19.226.193). New cstate: current_term: 1 leader_uuid: "8bf6e26e98b14566899578a990b27d47" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: UNKNOWN } } }
I20250109 08:57:30.773051 21512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e38ec0ef84af4b798073eafd4d210c35" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47" is_pre_election: true
I20250109 08:57:30.773396 21586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ddd14cdbefd64936960e9ffb5b38b5f2" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3"
W20250109 08:57:30.774755 21478 leader_election.cc:343] T ddd14cdbefd64936960e9ffb5b38b5f2 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:30.774935 21661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e38ec0ef84af4b798073eafd4d210c35" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e" is_pre_election: true
I20250109 08:57:30.775681 21661 raft_consensus.cc:2388] T e38ec0ef84af4b798073eafd4d210c35 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2b1a692435241ee988e0bd1661cd4c3 in current term 1: Already voted for candidate 8bf6e26e98b14566899578a990b27d47 in this term.
I20250109 08:57:30.777607 21662 raft_consensus.cc:2463] T ddd14cdbefd64936960e9ffb5b38b5f2 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8bf6e26e98b14566899578a990b27d47 in term 1.
I20250109 08:57:30.778733 21477 leader_election.cc:304] T ddd14cdbefd64936960e9ffb5b38b5f2 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, 8bf6e26e98b14566899578a990b27d47; no voters: a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:57:30.779992 21740 raft_consensus.cc:2798] T ddd14cdbefd64936960e9ffb5b38b5f2 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:57:30.780448 21740 raft_consensus.cc:695] T ddd14cdbefd64936960e9ffb5b38b5f2 P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Becoming Leader. State: Replica: 8bf6e26e98b14566899578a990b27d47, State: Running, Role: LEADER
I20250109 08:57:30.777631 21552 leader_election.cc:304] T e38ec0ef84af4b798073eafd4d210c35 P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2b1a692435241ee988e0bd1661cd4c3; no voters: 787ddcc824c2490a808fc5a53452dc5e, 8bf6e26e98b14566899578a990b27d47
I20250109 08:57:30.781205 21740 consensus_queue.cc:237] T ddd14cdbefd64936960e9ffb5b38b5f2 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.783006 21820 raft_consensus.cc:3054] T e38ec0ef84af4b798073eafd4d210c35 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:30.797360 21842 raft_consensus.cc:491] T d355d166de534fb4a50c9f48f2df7469 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:30.798378 21703 tablet_bootstrap.cc:492] T 75edb5d04fa14201ad711c1d95c81f64 P 8bf6e26e98b14566899578a990b27d47: No bootstrap required, opened a new log
I20250109 08:57:30.798249 21842 raft_consensus.cc:513] T d355d166de534fb4a50c9f48f2df7469 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.798833 21703 ts_tablet_manager.cc:1397] T 75edb5d04fa14201ad711c1d95c81f64 P 8bf6e26e98b14566899578a990b27d47: Time spent bootstrapping tablet: real 0.088s user 0.012s sys 0.010s
I20250109 08:57:30.799340 21820 raft_consensus.cc:2743] T e38ec0ef84af4b798073eafd4d210c35 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:57:30.800644 21781 raft_consensus.cc:491] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:30.801020 21781 raft_consensus.cc:513] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.802989 21781 leader_election.cc:290] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:30.803189 21703 raft_consensus.cc:357] T 75edb5d04fa14201ad711c1d95c81f64 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.803855 21703 raft_consensus.cc:383] T 75edb5d04fa14201ad711c1d95c81f64 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:30.804100 21703 raft_consensus.cc:738] T 75edb5d04fa14201ad711c1d95c81f64 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bf6e26e98b14566899578a990b27d47, State: Initialized, Role: FOLLOWER
I20250109 08:57:30.804693 21587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d355d166de534fb4a50c9f48f2df7469" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3" is_pre_election: true
I20250109 08:57:30.804682 21703 consensus_queue.cc:260] T 75edb5d04fa14201ad711c1d95c81f64 P 8bf6e26e98b14566899578a990b27d47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.806396 21704 tablet_bootstrap.cc:492] T 0cc4fcda876242c396b804edebebb01f P 787ddcc824c2490a808fc5a53452dc5e: No bootstrap required, opened a new log
I20250109 08:57:30.806529 21587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b54d6381a2cc41dfb59b1f6b99c0c5d4" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3" is_pre_election: true
I20250109 08:57:30.806828 21704 ts_tablet_manager.cc:1397] T 0cc4fcda876242c396b804edebebb01f P 787ddcc824c2490a808fc5a53452dc5e: Time spent bootstrapping tablet: real 0.057s user 0.006s sys 0.026s
I20250109 08:57:30.806985 21703 ts_tablet_manager.cc:1428] T 75edb5d04fa14201ad711c1d95c81f64 P 8bf6e26e98b14566899578a990b27d47: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
W20250109 08:57:30.807319 21478 leader_election.cc:343] T d355d166de534fb4a50c9f48f2df7469 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:30.807857 21703 tablet_bootstrap.cc:492] T 0cc4fcda876242c396b804edebebb01f P 8bf6e26e98b14566899578a990b27d47: Bootstrap starting.
W20250109 08:57:30.809484 21478 leader_election.cc:343] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:30.811807 21704 raft_consensus.cc:357] T 0cc4fcda876242c396b804edebebb01f P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.812516 21704 raft_consensus.cc:383] T 0cc4fcda876242c396b804edebebb01f P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:30.812759 21704 raft_consensus.cc:738] T 0cc4fcda876242c396b804edebebb01f P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Initialized, Role: FOLLOWER
I20250109 08:57:30.813856 21703 tablet_bootstrap.cc:654] T 0cc4fcda876242c396b804edebebb01f P 8bf6e26e98b14566899578a990b27d47: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:30.813341 21704 consensus_queue.cc:260] T 0cc4fcda876242c396b804edebebb01f P 787ddcc824c2490a808fc5a53452dc5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.814100 21662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b54d6381a2cc41dfb59b1f6b99c0c5d4" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e" is_pre_election: true
W20250109 08:57:30.815248 21477 leader_election.cc:343] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:30.815698 21704 ts_tablet_manager.cc:1428] T 0cc4fcda876242c396b804edebebb01f P 787ddcc824c2490a808fc5a53452dc5e: Time spent starting tablet: real 0.009s user 0.006s sys 0.002s
I20250109 08:57:30.815654 21477 leader_election.cc:304] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8bf6e26e98b14566899578a990b27d47; no voters: 787ddcc824c2490a808fc5a53452dc5e, a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:57:30.816673 21704 tablet_bootstrap.cc:492] T d355d166de534fb4a50c9f48f2df7469 P 787ddcc824c2490a808fc5a53452dc5e: Bootstrap starting.
I20250109 08:57:30.816787 21781 raft_consensus.cc:2743] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:57:30.818749 21662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d355d166de534fb4a50c9f48f2df7469" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e" is_pre_election: true
W20250109 08:57:30.819803 21477 leader_election.cc:343] T d355d166de534fb4a50c9f48f2df7469 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:30.820196 21477 leader_election.cc:304] T d355d166de534fb4a50c9f48f2df7469 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8bf6e26e98b14566899578a990b27d47; no voters: 787ddcc824c2490a808fc5a53452dc5e, a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:57:30.820986 21781 raft_consensus.cc:2743] T d355d166de534fb4a50c9f48f2df7469 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:57:30.821559 21842 leader_election.cc:290] T d355d166de534fb4a50c9f48f2df7469 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:30.822678 21702 tablet_bootstrap.cc:492] T 75edb5d04fa14201ad711c1d95c81f64 P a2b1a692435241ee988e0bd1661cd4c3: No bootstrap required, opened a new log
I20250109 08:57:30.823228 21702 ts_tablet_manager.cc:1397] T 75edb5d04fa14201ad711c1d95c81f64 P a2b1a692435241ee988e0bd1661cd4c3: Time spent bootstrapping tablet: real 0.107s user 0.017s sys 0.019s
I20250109 08:57:30.823475 21704 tablet_bootstrap.cc:654] T d355d166de534fb4a50c9f48f2df7469 P 787ddcc824c2490a808fc5a53452dc5e: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:30.825619 21702 raft_consensus.cc:357] T 75edb5d04fa14201ad711c1d95c81f64 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.826324 21702 raft_consensus.cc:383] T 75edb5d04fa14201ad711c1d95c81f64 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:30.826565 21702 raft_consensus.cc:738] T 75edb5d04fa14201ad711c1d95c81f64 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2b1a692435241ee988e0bd1661cd4c3, State: Initialized, Role: FOLLOWER
I20250109 08:57:30.827145 21702 consensus_queue.cc:260] T 75edb5d04fa14201ad711c1d95c81f64 P a2b1a692435241ee988e0bd1661cd4c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.830701 21586 raft_consensus.cc:3054] T 81d74e7d4472497aac5463a4b47e4fb9 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:30.834875 21702 ts_tablet_manager.cc:1428] T 75edb5d04fa14201ad711c1d95c81f64 P a2b1a692435241ee988e0bd1661cd4c3: Time spent starting tablet: real 0.011s user 0.006s sys 0.000s
I20250109 08:57:30.835826 21702 tablet_bootstrap.cc:492] T d355d166de534fb4a50c9f48f2df7469 P a2b1a692435241ee988e0bd1661cd4c3: Bootstrap starting.
I20250109 08:57:30.837416 21411 catalog_manager.cc:5526] T ddd14cdbefd64936960e9ffb5b38b5f2 P 8bf6e26e98b14566899578a990b27d47 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8bf6e26e98b14566899578a990b27d47 (127.19.226.193). New cstate: current_term: 1 leader_uuid: "8bf6e26e98b14566899578a990b27d47" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: UNKNOWN } } }
I20250109 08:57:30.841861 21702 tablet_bootstrap.cc:654] T d355d166de534fb4a50c9f48f2df7469 P a2b1a692435241ee988e0bd1661cd4c3: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:30.852859 21703 tablet_bootstrap.cc:492] T 0cc4fcda876242c396b804edebebb01f P 8bf6e26e98b14566899578a990b27d47: No bootstrap required, opened a new log
I20250109 08:57:30.853391 21703 ts_tablet_manager.cc:1397] T 0cc4fcda876242c396b804edebebb01f P 8bf6e26e98b14566899578a990b27d47: Time spent bootstrapping tablet: real 0.046s user 0.008s sys 0.007s
I20250109 08:57:30.854672 21704 tablet_bootstrap.cc:492] T d355d166de534fb4a50c9f48f2df7469 P 787ddcc824c2490a808fc5a53452dc5e: No bootstrap required, opened a new log
I20250109 08:57:30.855104 21704 ts_tablet_manager.cc:1397] T d355d166de534fb4a50c9f48f2df7469 P 787ddcc824c2490a808fc5a53452dc5e: Time spent bootstrapping tablet: real 0.039s user 0.018s sys 0.017s
I20250109 08:57:30.855790 21703 raft_consensus.cc:357] T 0cc4fcda876242c396b804edebebb01f P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.856635 21703 raft_consensus.cc:383] T 0cc4fcda876242c396b804edebebb01f P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:30.856909 21703 raft_consensus.cc:738] T 0cc4fcda876242c396b804edebebb01f P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bf6e26e98b14566899578a990b27d47, State: Initialized, Role: FOLLOWER
I20250109 08:57:30.857322 21702 tablet_bootstrap.cc:492] T d355d166de534fb4a50c9f48f2df7469 P a2b1a692435241ee988e0bd1661cd4c3: No bootstrap required, opened a new log
I20250109 08:57:30.858819 21781 raft_consensus.cc:491] T 89e1eeb24fea483eb63bd98af66e0b6b P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:30.859254 21781 raft_consensus.cc:513] T 89e1eeb24fea483eb63bd98af66e0b6b P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.861820 21587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89e1eeb24fea483eb63bd98af66e0b6b" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3" is_pre_election: true
I20250109 08:57:30.862421 21587 raft_consensus.cc:2463] T 89e1eeb24fea483eb63bd98af66e0b6b P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8bf6e26e98b14566899578a990b27d47 in term 0.
I20250109 08:57:30.863051 21732 raft_consensus.cc:491] T 89e1eeb24fea483eb63bd98af66e0b6b P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:30.863433 21478 leader_election.cc:304] T 89e1eeb24fea483eb63bd98af66e0b6b P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3; no voters:
I20250109 08:57:30.864179 21842 raft_consensus.cc:2798] T 89e1eeb24fea483eb63bd98af66e0b6b P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:57:30.864564 21842 raft_consensus.cc:491] T 89e1eeb24fea483eb63bd98af66e0b6b P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:57:30.864874 21842 raft_consensus.cc:3054] T 89e1eeb24fea483eb63bd98af66e0b6b P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:30.863468 21732 raft_consensus.cc:513] T 89e1eeb24fea483eb63bd98af66e0b6b P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.871199 21732 leader_election.cc:290] T 89e1eeb24fea483eb63bd98af66e0b6b P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
I20250109 08:57:30.872004 21512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89e1eeb24fea483eb63bd98af66e0b6b" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47" is_pre_election: true
I20250109 08:57:30.873323 21587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89e1eeb24fea483eb63bd98af66e0b6b" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3" is_pre_election: true
I20250109 08:57:30.874017 21587 raft_consensus.cc:2463] T 89e1eeb24fea483eb63bd98af66e0b6b P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 0.
I20250109 08:57:30.875185 21627 leader_election.cc:304] T 89e1eeb24fea483eb63bd98af66e0b6b P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, a2b1a692435241ee988e0bd1661cd4c3; no voters:
I20250109 08:57:30.876009 21732 raft_consensus.cc:2798] T 89e1eeb24fea483eb63bd98af66e0b6b P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:57:30.876348 21732 raft_consensus.cc:491] T 89e1eeb24fea483eb63bd98af66e0b6b P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:57:30.876756 21732 raft_consensus.cc:3054] T 89e1eeb24fea483eb63bd98af66e0b6b P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:30.876500 21704 raft_consensus.cc:357] T d355d166de534fb4a50c9f48f2df7469 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.877204 21704 raft_consensus.cc:383] T d355d166de534fb4a50c9f48f2df7469 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:30.877465 21704 raft_consensus.cc:738] T d355d166de534fb4a50c9f48f2df7469 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Initialized, Role: FOLLOWER
I20250109 08:57:30.878026 21740 consensus_queue.cc:1035] T d6f988a231a74f6fa112da9a527d4a0b P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:57:30.878329 21704 consensus_queue.cc:260] T d355d166de534fb4a50c9f48f2df7469 P 787ddcc824c2490a808fc5a53452dc5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.875186 21703 consensus_queue.cc:260] T 0cc4fcda876242c396b804edebebb01f P 8bf6e26e98b14566899578a990b27d47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.884613 21732 raft_consensus.cc:513] T 89e1eeb24fea483eb63bd98af66e0b6b P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.886492 21758 raft_consensus.cc:491] T d6f988a231a74f6fa112da9a527d4a0b P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:30.886632 21781 leader_election.cc:290] T 89e1eeb24fea483eb63bd98af66e0b6b P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:30.887115 21514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89e1eeb24fea483eb63bd98af66e0b6b" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47"
I20250109 08:57:30.886979 21758 raft_consensus.cc:513] T d6f988a231a74f6fa112da9a527d4a0b P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.888105 21662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89e1eeb24fea483eb63bd98af66e0b6b" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e" is_pre_election: true
I20250109 08:57:30.888389 21703 ts_tablet_manager.cc:1428] T 0cc4fcda876242c396b804edebebb01f P 8bf6e26e98b14566899578a990b27d47: Time spent starting tablet: real 0.035s user 0.018s sys 0.004s
I20250109 08:57:30.888825 21662 raft_consensus.cc:2388] T 89e1eeb24fea483eb63bd98af66e0b6b P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8bf6e26e98b14566899578a990b27d47 in current term 1: Already voted for candidate 787ddcc824c2490a808fc5a53452dc5e in this term.
I20250109 08:57:30.889022 21732 leader_election.cc:290] T 89e1eeb24fea483eb63bd98af66e0b6b P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 election: Requested vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
I20250109 08:57:30.890270 21703 tablet_bootstrap.cc:492] T df9715570a6844d9a2686307db16eff1 P 8bf6e26e98b14566899578a990b27d47: Bootstrap starting.
I20250109 08:57:30.888859 21758 leader_election.cc:290] T d6f988a231a74f6fa112da9a527d4a0b P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
I20250109 08:57:30.891878 21588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6f988a231a74f6fa112da9a527d4a0b" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3" is_pre_election: true
I20250109 08:57:30.892765 21704 ts_tablet_manager.cc:1428] T d355d166de534fb4a50c9f48f2df7469 P 787ddcc824c2490a808fc5a53452dc5e: Time spent starting tablet: real 0.037s user 0.007s sys 0.001s
I20250109 08:57:30.892820 21514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6f988a231a74f6fa112da9a527d4a0b" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47" is_pre_election: true
I20250109 08:57:30.893105 21589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89e1eeb24fea483eb63bd98af66e0b6b" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3"
I20250109 08:57:30.893627 21589 raft_consensus.cc:3054] T 89e1eeb24fea483eb63bd98af66e0b6b P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:30.895856 21703 tablet_bootstrap.cc:654] T df9715570a6844d9a2686307db16eff1 P 8bf6e26e98b14566899578a990b27d47: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:30.897141 21627 leader_election.cc:304] T d6f988a231a74f6fa112da9a527d4a0b P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e; no voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:57:30.899866 21758 raft_consensus.cc:2743] T d6f988a231a74f6fa112da9a527d4a0b P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:57:30.900939 21704 tablet_bootstrap.cc:492] T 75edb5d04fa14201ad711c1d95c81f64 P 787ddcc824c2490a808fc5a53452dc5e: Bootstrap starting.
I20250109 08:57:30.904960 21662 raft_consensus.cc:3054] T d6f988a231a74f6fa112da9a527d4a0b P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:30.910899 21704 tablet_bootstrap.cc:654] T 75edb5d04fa14201ad711c1d95c81f64 P 787ddcc824c2490a808fc5a53452dc5e: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:30.912595 21740 consensus_queue.cc:1035] T d6f988a231a74f6fa112da9a527d4a0b P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:57:30.904268 21589 raft_consensus.cc:2463] T 89e1eeb24fea483eb63bd98af66e0b6b P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 1.
I20250109 08:57:30.926723 21627 leader_election.cc:304] T 89e1eeb24fea483eb63bd98af66e0b6b P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, a2b1a692435241ee988e0bd1661cd4c3; no voters: 8bf6e26e98b14566899578a990b27d47
I20250109 08:57:30.928098 21758 raft_consensus.cc:2798] T 89e1eeb24fea483eb63bd98af66e0b6b P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:57:30.928659 21758 raft_consensus.cc:695] T 89e1eeb24fea483eb63bd98af66e0b6b P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Becoming Leader. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Running, Role: LEADER
I20250109 08:57:30.929476 21758 consensus_queue.cc:237] T 89e1eeb24fea483eb63bd98af66e0b6b P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.931404 21842 raft_consensus.cc:513] T 89e1eeb24fea483eb63bd98af66e0b6b P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.932628 21512 raft_consensus.cc:2388] T 89e1eeb24fea483eb63bd98af66e0b6b P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 787ddcc824c2490a808fc5a53452dc5e in current term 1: Already voted for candidate 8bf6e26e98b14566899578a990b27d47 in this term.
I20250109 08:57:30.938557 21704 tablet_bootstrap.cc:492] T 75edb5d04fa14201ad711c1d95c81f64 P 787ddcc824c2490a808fc5a53452dc5e: No bootstrap required, opened a new log
I20250109 08:57:30.938992 21704 ts_tablet_manager.cc:1397] T 75edb5d04fa14201ad711c1d95c81f64 P 787ddcc824c2490a808fc5a53452dc5e: Time spent bootstrapping tablet: real 0.038s user 0.012s sys 0.006s
I20250109 08:57:30.940116 21703 tablet_bootstrap.cc:492] T df9715570a6844d9a2686307db16eff1 P 8bf6e26e98b14566899578a990b27d47: No bootstrap required, opened a new log
I20250109 08:57:30.940531 21703 ts_tablet_manager.cc:1397] T df9715570a6844d9a2686307db16eff1 P 8bf6e26e98b14566899578a990b27d47: Time spent bootstrapping tablet: real 0.050s user 0.019s sys 0.023s
I20250109 08:57:30.942790 21703 raft_consensus.cc:357] T df9715570a6844d9a2686307db16eff1 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.943429 21703 raft_consensus.cc:383] T df9715570a6844d9a2686307db16eff1 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:30.943686 21703 raft_consensus.cc:738] T df9715570a6844d9a2686307db16eff1 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bf6e26e98b14566899578a990b27d47, State: Initialized, Role: FOLLOWER
I20250109 08:57:30.944268 21703 consensus_queue.cc:260] T df9715570a6844d9a2686307db16eff1 P 8bf6e26e98b14566899578a990b27d47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.945431 21412 catalog_manager.cc:5526] T 89e1eeb24fea483eb63bd98af66e0b6b P 787ddcc824c2490a808fc5a53452dc5e reported cstate change: term changed from 0 to 1, leader changed from <none> to 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195). New cstate: current_term: 1 leader_uuid: "787ddcc824c2490a808fc5a53452dc5e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: HEALTHY } } }
I20250109 08:57:30.947916 21704 raft_consensus.cc:357] T 75edb5d04fa14201ad711c1d95c81f64 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.948608 21704 raft_consensus.cc:383] T 75edb5d04fa14201ad711c1d95c81f64 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:30.949802 21703 ts_tablet_manager.cc:1428] T df9715570a6844d9a2686307db16eff1 P 8bf6e26e98b14566899578a990b27d47: Time spent starting tablet: real 0.009s user 0.005s sys 0.000s
I20250109 08:57:30.948859 21704 raft_consensus.cc:738] T 75edb5d04fa14201ad711c1d95c81f64 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Initialized, Role: FOLLOWER
I20250109 08:57:30.950595 21703 tablet_bootstrap.cc:492] T c18c1d46a6ca47598d7b161ed70a4e7a P 8bf6e26e98b14566899578a990b27d47: Bootstrap starting.
I20250109 08:57:30.951174 21704 consensus_queue.cc:260] T 75edb5d04fa14201ad711c1d95c81f64 P 787ddcc824c2490a808fc5a53452dc5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.953117 21704 ts_tablet_manager.cc:1428] T 75edb5d04fa14201ad711c1d95c81f64 P 787ddcc824c2490a808fc5a53452dc5e: Time spent starting tablet: real 0.014s user 0.006s sys 0.000s
I20250109 08:57:30.953971 21704 tablet_bootstrap.cc:492] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P 787ddcc824c2490a808fc5a53452dc5e: Bootstrap starting.
I20250109 08:57:30.957235 21703 tablet_bootstrap.cc:654] T c18c1d46a6ca47598d7b161ed70a4e7a P 8bf6e26e98b14566899578a990b27d47: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:30.964555 21704 tablet_bootstrap.cc:654] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P 787ddcc824c2490a808fc5a53452dc5e: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:30.965469 21587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89e1eeb24fea483eb63bd98af66e0b6b" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3"
I20250109 08:57:30.966285 21587 raft_consensus.cc:2388] T 89e1eeb24fea483eb63bd98af66e0b6b P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 8bf6e26e98b14566899578a990b27d47 in current term 1: Already voted for candidate 787ddcc824c2490a808fc5a53452dc5e in this term.
I20250109 08:57:30.967326 21709 consensus_queue.cc:1035] T 3b5049c00b0b4dd89c49fa5d13db169e P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:57:30.971410 21703 tablet_bootstrap.cc:492] T c18c1d46a6ca47598d7b161ed70a4e7a P 8bf6e26e98b14566899578a990b27d47: No bootstrap required, opened a new log
I20250109 08:57:30.971894 21703 ts_tablet_manager.cc:1397] T c18c1d46a6ca47598d7b161ed70a4e7a P 8bf6e26e98b14566899578a990b27d47: Time spent bootstrapping tablet: real 0.021s user 0.012s sys 0.004s
I20250109 08:57:30.974610 21703 raft_consensus.cc:357] T c18c1d46a6ca47598d7b161ed70a4e7a P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.975317 21703 raft_consensus.cc:383] T c18c1d46a6ca47598d7b161ed70a4e7a P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:30.975569 21703 raft_consensus.cc:738] T c18c1d46a6ca47598d7b161ed70a4e7a P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bf6e26e98b14566899578a990b27d47, State: Initialized, Role: FOLLOWER
I20250109 08:57:30.975646 21702 ts_tablet_manager.cc:1397] T d355d166de534fb4a50c9f48f2df7469 P a2b1a692435241ee988e0bd1661cd4c3: Time spent bootstrapping tablet: real 0.140s user 0.010s sys 0.007s
I20250109 08:57:30.976171 21703 consensus_queue.cc:260] T c18c1d46a6ca47598d7b161ed70a4e7a P 8bf6e26e98b14566899578a990b27d47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.980340 21662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89e1eeb24fea483eb63bd98af66e0b6b" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e"
I20250109 08:57:30.980901 21842 leader_election.cc:290] T 89e1eeb24fea483eb63bd98af66e0b6b P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 election: Requested vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:30.984555 21477 leader_election.cc:304] T 89e1eeb24fea483eb63bd98af66e0b6b P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8bf6e26e98b14566899578a990b27d47; no voters: 787ddcc824c2490a808fc5a53452dc5e, a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:57:30.985608 21842 raft_consensus.cc:2743] T 89e1eeb24fea483eb63bd98af66e0b6b P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20250109 08:57:30.978163 21702 raft_consensus.cc:357] T d355d166de534fb4a50c9f48f2df7469 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.982954 21703 ts_tablet_manager.cc:1428] T c18c1d46a6ca47598d7b161ed70a4e7a P 8bf6e26e98b14566899578a990b27d47: Time spent starting tablet: real 0.011s user 0.006s sys 0.002s
I20250109 08:57:30.987027 21703 tablet_bootstrap.cc:492] T 126a163a89c448be89ccb9490e9f0168 P 8bf6e26e98b14566899578a990b27d47: Bootstrap starting.
I20250109 08:57:30.986187 21702 raft_consensus.cc:383] T d355d166de534fb4a50c9f48f2df7469 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:30.992439 21702 raft_consensus.cc:738] T d355d166de534fb4a50c9f48f2df7469 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2b1a692435241ee988e0bd1661cd4c3, State: Initialized, Role: FOLLOWER
I20250109 08:57:30.993237 21702 consensus_queue.cc:260] T d355d166de534fb4a50c9f48f2df7469 P a2b1a692435241ee988e0bd1661cd4c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:30.995537 21702 ts_tablet_manager.cc:1428] T d355d166de534fb4a50c9f48f2df7469 P a2b1a692435241ee988e0bd1661cd4c3: Time spent starting tablet: real 0.020s user 0.003s sys 0.002s
I20250109 08:57:30.996413 21702 tablet_bootstrap.cc:492] T 90a4a2381cbe40f0b204be325ee9fea1 P a2b1a692435241ee988e0bd1661cd4c3: Bootstrap starting.
I20250109 08:57:31.001873 21703 tablet_bootstrap.cc:654] T 126a163a89c448be89ccb9490e9f0168 P 8bf6e26e98b14566899578a990b27d47: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:31.005966 21758 raft_consensus.cc:491] T 0e06021cd6624e50af2c914399ebd6c4 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:31.006155 21702 tablet_bootstrap.cc:654] T 90a4a2381cbe40f0b204be325ee9fea1 P a2b1a692435241ee988e0bd1661cd4c3: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:31.006443 21758 raft_consensus.cc:513] T 0e06021cd6624e50af2c914399ebd6c4 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.008421 21758 leader_election.cc:290] T 0e06021cd6624e50af2c914399ebd6c4 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
I20250109 08:57:31.009799 21512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e06021cd6624e50af2c914399ebd6c4" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47" is_pre_election: true
I20250109 08:57:31.010246 21589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e06021cd6624e50af2c914399ebd6c4" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3" is_pre_election: true
I20250109 08:57:31.012187 21627 leader_election.cc:304] T 0e06021cd6624e50af2c914399ebd6c4 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e; no voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:57:31.013269 21758 raft_consensus.cc:2743] T 0e06021cd6624e50af2c914399ebd6c4 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:57:31.016351 21703 tablet_bootstrap.cc:492] T 126a163a89c448be89ccb9490e9f0168 P 8bf6e26e98b14566899578a990b27d47: No bootstrap required, opened a new log
I20250109 08:57:31.017000 21703 ts_tablet_manager.cc:1397] T 126a163a89c448be89ccb9490e9f0168 P 8bf6e26e98b14566899578a990b27d47: Time spent bootstrapping tablet: real 0.030s user 0.012s sys 0.002s
I20250109 08:57:31.018402 21704 tablet_bootstrap.cc:492] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P 787ddcc824c2490a808fc5a53452dc5e: No bootstrap required, opened a new log
I20250109 08:57:31.019207 21704 ts_tablet_manager.cc:1397] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P 787ddcc824c2490a808fc5a53452dc5e: Time spent bootstrapping tablet: real 0.065s user 0.003s sys 0.031s
I20250109 08:57:31.020215 21759 consensus_queue.cc:1035] T 3b5049c00b0b4dd89c49fa5d13db169e P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:57:31.021230 21702 tablet_bootstrap.cc:492] T 90a4a2381cbe40f0b204be325ee9fea1 P a2b1a692435241ee988e0bd1661cd4c3: No bootstrap required, opened a new log
I20250109 08:57:31.021709 21702 ts_tablet_manager.cc:1397] T 90a4a2381cbe40f0b204be325ee9fea1 P a2b1a692435241ee988e0bd1661cd4c3: Time spent bootstrapping tablet: real 0.025s user 0.012s sys 0.000s
I20250109 08:57:31.025398 21704 raft_consensus.cc:357] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.026407 21704 raft_consensus.cc:383] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:31.026681 21704 raft_consensus.cc:738] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Initialized, Role: FOLLOWER
I20250109 08:57:31.027330 21704 consensus_queue.cc:260] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P 787ddcc824c2490a808fc5a53452dc5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.030241 21704 ts_tablet_manager.cc:1428] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P 787ddcc824c2490a808fc5a53452dc5e: Time spent starting tablet: real 0.011s user 0.008s sys 0.000s
I20250109 08:57:31.035291 21702 raft_consensus.cc:357] T 90a4a2381cbe40f0b204be325ee9fea1 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.036053 21702 raft_consensus.cc:383] T 90a4a2381cbe40f0b204be325ee9fea1 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:31.036315 21702 raft_consensus.cc:738] T 90a4a2381cbe40f0b204be325ee9fea1 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2b1a692435241ee988e0bd1661cd4c3, State: Initialized, Role: FOLLOWER
I20250109 08:57:31.036954 21702 consensus_queue.cc:260] T 90a4a2381cbe40f0b204be325ee9fea1 P a2b1a692435241ee988e0bd1661cd4c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.039147 21702 ts_tablet_manager.cc:1428] T 90a4a2381cbe40f0b204be325ee9fea1 P a2b1a692435241ee988e0bd1661cd4c3: Time spent starting tablet: real 0.017s user 0.005s sys 0.000s
I20250109 08:57:31.036896 21704 tablet_bootstrap.cc:492] T 0f3bde1a1ccd4c359a2e80feca968453 P 787ddcc824c2490a808fc5a53452dc5e: Bootstrap starting.
I20250109 08:57:31.039973 21702 tablet_bootstrap.cc:492] T e8ff570914d74df8b87296010e9b864f P a2b1a692435241ee988e0bd1661cd4c3: Bootstrap starting.
I20250109 08:57:31.045346 21703 raft_consensus.cc:357] T 126a163a89c448be89ccb9490e9f0168 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.045771 21704 tablet_bootstrap.cc:654] T 0f3bde1a1ccd4c359a2e80feca968453 P 787ddcc824c2490a808fc5a53452dc5e: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:31.046042 21703 raft_consensus.cc:383] T 126a163a89c448be89ccb9490e9f0168 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:31.046325 21703 raft_consensus.cc:738] T 126a163a89c448be89ccb9490e9f0168 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bf6e26e98b14566899578a990b27d47, State: Initialized, Role: FOLLOWER
I20250109 08:57:31.046931 21703 consensus_queue.cc:260] T 126a163a89c448be89ccb9490e9f0168 P 8bf6e26e98b14566899578a990b27d47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.048511 21703 ts_tablet_manager.cc:1428] T 126a163a89c448be89ccb9490e9f0168 P 8bf6e26e98b14566899578a990b27d47: Time spent starting tablet: real 0.031s user 0.005s sys 0.001s
I20250109 08:57:31.049041 21758 raft_consensus.cc:491] T 75edb5d04fa14201ad711c1d95c81f64 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20250109 08:57:31.049260 21460 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20250109 08:57:31.049342 21703 tablet_bootstrap.cc:492] T 90978f185a374feba3bc19b54f9d97f5 P 8bf6e26e98b14566899578a990b27d47: Bootstrap starting.
I20250109 08:57:31.049489 21758 raft_consensus.cc:513] T 75edb5d04fa14201ad711c1d95c81f64 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.051396 21704 tablet_bootstrap.cc:492] T 0f3bde1a1ccd4c359a2e80feca968453 P 787ddcc824c2490a808fc5a53452dc5e: No bootstrap required, opened a new log
I20250109 08:57:31.051803 21704 ts_tablet_manager.cc:1397] T 0f3bde1a1ccd4c359a2e80feca968453 P 787ddcc824c2490a808fc5a53452dc5e: Time spent bootstrapping tablet: real 0.015s user 0.010s sys 0.000s
I20250109 08:57:31.051817 21702 tablet_bootstrap.cc:654] T e8ff570914d74df8b87296010e9b864f P a2b1a692435241ee988e0bd1661cd4c3: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:31.052076 21758 leader_election.cc:290] T 75edb5d04fa14201ad711c1d95c81f64 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
I20250109 08:57:31.053702 21589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75edb5d04fa14201ad711c1d95c81f64" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3" is_pre_election: true
I20250109 08:57:31.054963 21589 raft_consensus.cc:2463] T 75edb5d04fa14201ad711c1d95c81f64 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 0.
I20250109 08:57:31.056514 21512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75edb5d04fa14201ad711c1d95c81f64" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47" is_pre_election: true
I20250109 08:57:31.056200 21704 raft_consensus.cc:357] T 0f3bde1a1ccd4c359a2e80feca968453 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.057214 21704 raft_consensus.cc:383] T 0f3bde1a1ccd4c359a2e80feca968453 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:31.057088 21512 raft_consensus.cc:2463] T 75edb5d04fa14201ad711c1d95c81f64 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 0.
I20250109 08:57:31.057157 21627 leader_election.cc:304] T 75edb5d04fa14201ad711c1d95c81f64 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, a2b1a692435241ee988e0bd1661cd4c3; no voters:
I20250109 08:57:31.057598 21704 raft_consensus.cc:738] T 0f3bde1a1ccd4c359a2e80feca968453 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Initialized, Role: FOLLOWER
I20250109 08:57:31.058223 21704 consensus_queue.cc:260] T 0f3bde1a1ccd4c359a2e80feca968453 P 787ddcc824c2490a808fc5a53452dc5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.058988 21758 raft_consensus.cc:2798] T 75edb5d04fa14201ad711c1d95c81f64 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:57:31.060424 21758 raft_consensus.cc:491] T 75edb5d04fa14201ad711c1d95c81f64 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:57:31.060484 21842 raft_consensus.cc:491] T c18c1d46a6ca47598d7b161ed70a4e7a P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:31.060820 21758 raft_consensus.cc:3054] T 75edb5d04fa14201ad711c1d95c81f64 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:31.061002 21842 raft_consensus.cc:513] T c18c1d46a6ca47598d7b161ed70a4e7a P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.063199 21663 raft_consensus.cc:3054] T 0e06021cd6624e50af2c914399ebd6c4 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:31.064158 21589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c18c1d46a6ca47598d7b161ed70a4e7a" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3" is_pre_election: true
I20250109 08:57:31.064698 21662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c18c1d46a6ca47598d7b161ed70a4e7a" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e" is_pre_election: true
W20250109 08:57:31.065486 21478 leader_election.cc:343] T c18c1d46a6ca47598d7b161ed70a4e7a P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:31.066006 21842 leader_election.cc:290] T c18c1d46a6ca47598d7b161ed70a4e7a P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:31.060487 21704 ts_tablet_manager.cc:1428] T 0f3bde1a1ccd4c359a2e80feca968453 P 787ddcc824c2490a808fc5a53452dc5e: Time spent starting tablet: real 0.008s user 0.005s sys 0.002s
W20250109 08:57:31.066989 21477 leader_election.cc:343] T c18c1d46a6ca47598d7b161ed70a4e7a P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:31.067224 21704 tablet_bootstrap.cc:492] T 90a4a2381cbe40f0b204be325ee9fea1 P 787ddcc824c2490a808fc5a53452dc5e: Bootstrap starting.
I20250109 08:57:31.067936 21758 raft_consensus.cc:513] T 75edb5d04fa14201ad711c1d95c81f64 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.068820 21477 leader_election.cc:304] T c18c1d46a6ca47598d7b161ed70a4e7a P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8bf6e26e98b14566899578a990b27d47; no voters: 787ddcc824c2490a808fc5a53452dc5e, a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:57:31.069833 21710 consensus_queue.cc:1035] T d8493108cba04514bd2236192cef1079 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:57:31.071866 21842 raft_consensus.cc:2743] T c18c1d46a6ca47598d7b161ed70a4e7a P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:57:31.074334 21512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75edb5d04fa14201ad711c1d95c81f64" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47"
I20250109 08:57:31.074913 21512 raft_consensus.cc:3054] T 75edb5d04fa14201ad711c1d95c81f64 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:31.081023 21758 leader_election.cc:290] T 75edb5d04fa14201ad711c1d95c81f64 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 election: Requested vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
I20250109 08:57:31.081590 21587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75edb5d04fa14201ad711c1d95c81f64" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3"
I20250109 08:57:31.082175 21587 raft_consensus.cc:3054] T 75edb5d04fa14201ad711c1d95c81f64 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:31.083673 21702 tablet_bootstrap.cc:492] T e8ff570914d74df8b87296010e9b864f P a2b1a692435241ee988e0bd1661cd4c3: No bootstrap required, opened a new log
I20250109 08:57:31.084110 21702 ts_tablet_manager.cc:1397] T e8ff570914d74df8b87296010e9b864f P a2b1a692435241ee988e0bd1661cd4c3: Time spent bootstrapping tablet: real 0.044s user 0.012s sys 0.004s
I20250109 08:57:31.084976 21512 raft_consensus.cc:2463] T 75edb5d04fa14201ad711c1d95c81f64 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 1.
I20250109 08:57:31.087278 21703 tablet_bootstrap.cc:654] T 90978f185a374feba3bc19b54f9d97f5 P 8bf6e26e98b14566899578a990b27d47: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:31.087155 21702 raft_consensus.cc:357] T e8ff570914d74df8b87296010e9b864f P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.087826 21626 leader_election.cc:304] T 75edb5d04fa14201ad711c1d95c81f64 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, 8bf6e26e98b14566899578a990b27d47; no voters:
I20250109 08:57:31.088128 21702 raft_consensus.cc:383] T e8ff570914d74df8b87296010e9b864f P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:31.088609 21702 raft_consensus.cc:738] T e8ff570914d74df8b87296010e9b864f P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2b1a692435241ee988e0bd1661cd4c3, State: Initialized, Role: FOLLOWER
I20250109 08:57:31.089227 21702 consensus_queue.cc:260] T e8ff570914d74df8b87296010e9b864f P a2b1a692435241ee988e0bd1661cd4c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.091017 21702 ts_tablet_manager.cc:1428] T e8ff570914d74df8b87296010e9b864f P a2b1a692435241ee988e0bd1661cd4c3: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20250109 08:57:31.091823 21702 tablet_bootstrap.cc:492] T ddd14cdbefd64936960e9ffb5b38b5f2 P a2b1a692435241ee988e0bd1661cd4c3: Bootstrap starting.
I20250109 08:57:31.092027 21842 consensus_queue.cc:1035] T e38ec0ef84af4b798073eafd4d210c35 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:57:31.096851 21732 raft_consensus.cc:2798] T 75edb5d04fa14201ad711c1d95c81f64 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:57:31.097302 21732 raft_consensus.cc:695] T 75edb5d04fa14201ad711c1d95c81f64 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Becoming Leader. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Running, Role: LEADER
I20250109 08:57:31.097775 21702 tablet_bootstrap.cc:654] T ddd14cdbefd64936960e9ffb5b38b5f2 P a2b1a692435241ee988e0bd1661cd4c3: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:31.088160 21587 raft_consensus.cc:2463] T 75edb5d04fa14201ad711c1d95c81f64 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 1.
I20250109 08:57:31.100770 21732 consensus_queue.cc:237] T 75edb5d04fa14201ad711c1d95c81f64 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.105558 21710 consensus_queue.cc:1035] T 306c4892bbff4dbfbe5a0645294aa627 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:57:31.109257 21842 raft_consensus.cc:491] T 0cc4fcda876242c396b804edebebb01f P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:31.109760 21842 raft_consensus.cc:513] T 0cc4fcda876242c396b804edebebb01f P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.110208 21704 tablet_bootstrap.cc:654] T 90a4a2381cbe40f0b204be325ee9fea1 P 787ddcc824c2490a808fc5a53452dc5e: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:31.112504 21588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cc4fcda876242c396b804edebebb01f" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3" is_pre_election: true
I20250109 08:57:31.113323 21702 tablet_bootstrap.cc:492] T ddd14cdbefd64936960e9ffb5b38b5f2 P a2b1a692435241ee988e0bd1661cd4c3: No bootstrap required, opened a new log
I20250109 08:57:31.113842 21702 ts_tablet_manager.cc:1397] T ddd14cdbefd64936960e9ffb5b38b5f2 P a2b1a692435241ee988e0bd1661cd4c3: Time spent bootstrapping tablet: real 0.022s user 0.010s sys 0.002s
W20250109 08:57:31.114053 21478 leader_election.cc:343] T 0cc4fcda876242c396b804edebebb01f P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:31.115963 21661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cc4fcda876242c396b804edebebb01f" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e" is_pre_election: true
I20250109 08:57:31.116539 21661 raft_consensus.cc:2463] T 0cc4fcda876242c396b804edebebb01f P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8bf6e26e98b14566899578a990b27d47 in term 0.
I20250109 08:57:31.117197 21842 leader_election.cc:290] T 0cc4fcda876242c396b804edebebb01f P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:31.117661 21477 leader_election.cc:304] T 0cc4fcda876242c396b804edebebb01f P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, 8bf6e26e98b14566899578a990b27d47; no voters: a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:57:31.117627 21702 raft_consensus.cc:357] T ddd14cdbefd64936960e9ffb5b38b5f2 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.118860 21781 raft_consensus.cc:2798] T 0cc4fcda876242c396b804edebebb01f P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:57:31.119228 21781 raft_consensus.cc:491] T 0cc4fcda876242c396b804edebebb01f P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:57:31.119537 21781 raft_consensus.cc:3054] T 0cc4fcda876242c396b804edebebb01f P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:31.120997 21703 tablet_bootstrap.cc:492] T 90978f185a374feba3bc19b54f9d97f5 P 8bf6e26e98b14566899578a990b27d47: No bootstrap required, opened a new log
I20250109 08:57:31.121467 21703 ts_tablet_manager.cc:1397] T 90978f185a374feba3bc19b54f9d97f5 P 8bf6e26e98b14566899578a990b27d47: Time spent bootstrapping tablet: real 0.072s user 0.016s sys 0.005s
I20250109 08:57:31.123989 21703 raft_consensus.cc:357] T 90978f185a374feba3bc19b54f9d97f5 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.124706 21703 raft_consensus.cc:383] T 90978f185a374feba3bc19b54f9d97f5 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:31.124963 21703 raft_consensus.cc:738] T 90978f185a374feba3bc19b54f9d97f5 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bf6e26e98b14566899578a990b27d47, State: Initialized, Role: FOLLOWER
I20250109 08:57:31.125625 21703 consensus_queue.cc:260] T 90978f185a374feba3bc19b54f9d97f5 P 8bf6e26e98b14566899578a990b27d47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.128072 21703 ts_tablet_manager.cc:1428] T 90978f185a374feba3bc19b54f9d97f5 P 8bf6e26e98b14566899578a990b27d47: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20250109 08:57:31.128954 21703 tablet_bootstrap.cc:492] T af273bab41cc4a9da780147eebfd52ec P 8bf6e26e98b14566899578a990b27d47: Bootstrap starting.
I20250109 08:57:31.130843 21710 consensus_queue.cc:1035] T d8493108cba04514bd2236192cef1079 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:57:31.130820 21704 tablet_bootstrap.cc:492] T 90a4a2381cbe40f0b204be325ee9fea1 P 787ddcc824c2490a808fc5a53452dc5e: No bootstrap required, opened a new log
I20250109 08:57:31.133167 21704 ts_tablet_manager.cc:1397] T 90a4a2381cbe40f0b204be325ee9fea1 P 787ddcc824c2490a808fc5a53452dc5e: Time spent bootstrapping tablet: real 0.066s user 0.009s sys 0.008s
I20250109 08:57:31.134074 21759 raft_consensus.cc:491] T e8ff570914d74df8b87296010e9b864f P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:31.134503 21759 raft_consensus.cc:513] T e8ff570914d74df8b87296010e9b864f P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.136535 21759 leader_election.cc:290] T e8ff570914d74df8b87296010e9b864f P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:31.135535 21704 raft_consensus.cc:357] T 90a4a2381cbe40f0b204be325ee9fea1 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.137303 21704 raft_consensus.cc:383] T 90a4a2381cbe40f0b204be325ee9fea1 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:31.137576 21704 raft_consensus.cc:738] T 90a4a2381cbe40f0b204be325ee9fea1 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Initialized, Role: FOLLOWER
I20250109 08:57:31.138469 21661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8ff570914d74df8b87296010e9b864f" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e" is_pre_election: true
I20250109 08:57:31.138166 21704 consensus_queue.cc:260] T 90a4a2381cbe40f0b204be325ee9fea1 P 787ddcc824c2490a808fc5a53452dc5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.138262 21781 raft_consensus.cc:513] T 0cc4fcda876242c396b804edebebb01f P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
W20250109 08:57:31.139722 21552 leader_election.cc:343] T e8ff570914d74df8b87296010e9b864f P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:31.140573 21704 ts_tablet_manager.cc:1428] T 90a4a2381cbe40f0b204be325ee9fea1 P 787ddcc824c2490a808fc5a53452dc5e: Time spent starting tablet: real 0.007s user 0.004s sys 0.002s
I20250109 08:57:31.141413 21704 tablet_bootstrap.cc:492] T e03846e461974d0d8944c344e715b86a P 787ddcc824c2490a808fc5a53452dc5e: Bootstrap starting.
I20250109 08:57:31.141446 21842 consensus_queue.cc:1035] T e38ec0ef84af4b798073eafd4d210c35 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:57:31.142445 21589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cc4fcda876242c396b804edebebb01f" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3"
W20250109 08:57:31.144065 21478 leader_election.cc:343] T 0cc4fcda876242c396b804edebebb01f P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:31.148001 21513 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8ff570914d74df8b87296010e9b864f" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47" is_pre_election: true
I20250109 08:57:31.118583 21702 raft_consensus.cc:383] T ddd14cdbefd64936960e9ffb5b38b5f2 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:31.147873 21758 consensus_queue.cc:1035] T 306c4892bbff4dbfbe5a0645294aa627 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20250109 08:57:31.149207 21552 leader_election.cc:343] T e8ff570914d74df8b87296010e9b864f P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:31.149643 21552 leader_election.cc:304] T e8ff570914d74df8b87296010e9b864f P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2b1a692435241ee988e0bd1661cd4c3; no voters: 787ddcc824c2490a808fc5a53452dc5e, 8bf6e26e98b14566899578a990b27d47
I20250109 08:57:31.149035 21702 raft_consensus.cc:738] T ddd14cdbefd64936960e9ffb5b38b5f2 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2b1a692435241ee988e0bd1661cd4c3, State: Initialized, Role: FOLLOWER
I20250109 08:57:31.150974 21759 raft_consensus.cc:2743] T e8ff570914d74df8b87296010e9b864f P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:57:31.153290 21781 leader_election.cc:290] T 0cc4fcda876242c396b804edebebb01f P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 election: Requested vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:31.152724 21661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cc4fcda876242c396b804edebebb01f" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e"
I20250109 08:57:31.155320 21704 tablet_bootstrap.cc:654] T e03846e461974d0d8944c344e715b86a P 787ddcc824c2490a808fc5a53452dc5e: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:31.155369 21661 raft_consensus.cc:3054] T 0cc4fcda876242c396b804edebebb01f P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:31.151189 21703 tablet_bootstrap.cc:654] T af273bab41cc4a9da780147eebfd52ec P 8bf6e26e98b14566899578a990b27d47: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:31.153996 21702 consensus_queue.cc:260] T ddd14cdbefd64936960e9ffb5b38b5f2 P a2b1a692435241ee988e0bd1661cd4c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.165660 21702 ts_tablet_manager.cc:1428] T ddd14cdbefd64936960e9ffb5b38b5f2 P a2b1a692435241ee988e0bd1661cd4c3: Time spent starting tablet: real 0.051s user 0.003s sys 0.003s
I20250109 08:57:31.171751 21702 tablet_bootstrap.cc:492] T af273bab41cc4a9da780147eebfd52ec P a2b1a692435241ee988e0bd1661cd4c3: Bootstrap starting.
I20250109 08:57:31.162326 21661 raft_consensus.cc:2463] T 0cc4fcda876242c396b804edebebb01f P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8bf6e26e98b14566899578a990b27d47 in term 1.
I20250109 08:57:31.179472 21758 raft_consensus.cc:491] T 90a4a2381cbe40f0b204be325ee9fea1 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:31.194090 21758 raft_consensus.cc:513] T 90a4a2381cbe40f0b204be325ee9fea1 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.196215 21758 leader_election.cc:290] T 90a4a2381cbe40f0b204be325ee9fea1 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
I20250109 08:57:31.197973 21512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90a4a2381cbe40f0b204be325ee9fea1" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47" is_pre_election: true
I20250109 08:57:31.190338 21703 tablet_bootstrap.cc:492] T af273bab41cc4a9da780147eebfd52ec P 8bf6e26e98b14566899578a990b27d47: No bootstrap required, opened a new log
I20250109 08:57:31.199154 21703 ts_tablet_manager.cc:1397] T af273bab41cc4a9da780147eebfd52ec P 8bf6e26e98b14566899578a990b27d47: Time spent bootstrapping tablet: real 0.070s user 0.011s sys 0.000s
W20250109 08:57:31.201946 21626 leader_election.cc:343] T 90a4a2381cbe40f0b204be325ee9fea1 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:31.202694 21589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90a4a2381cbe40f0b204be325ee9fea1" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3" is_pre_election: true
I20250109 08:57:31.203361 21589 raft_consensus.cc:2463] T 90a4a2381cbe40f0b204be325ee9fea1 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 0.
I20250109 08:57:31.207074 21703 raft_consensus.cc:357] T af273bab41cc4a9da780147eebfd52ec P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.207779 21703 raft_consensus.cc:383] T af273bab41cc4a9da780147eebfd52ec P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:31.208042 21703 raft_consensus.cc:738] T af273bab41cc4a9da780147eebfd52ec P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bf6e26e98b14566899578a990b27d47, State: Initialized, Role: FOLLOWER
I20250109 08:57:31.209270 21704 tablet_bootstrap.cc:492] T e03846e461974d0d8944c344e715b86a P 787ddcc824c2490a808fc5a53452dc5e: No bootstrap required, opened a new log
I20250109 08:57:31.213089 21704 ts_tablet_manager.cc:1397] T e03846e461974d0d8944c344e715b86a P 787ddcc824c2490a808fc5a53452dc5e: Time spent bootstrapping tablet: real 0.072s user 0.021s sys 0.030s
I20250109 08:57:31.213027 21477 leader_election.cc:304] T 0cc4fcda876242c396b804edebebb01f P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, 8bf6e26e98b14566899578a990b27d47; no voters: a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:57:31.214386 21703 consensus_queue.cc:260] T af273bab41cc4a9da780147eebfd52ec P 8bf6e26e98b14566899578a990b27d47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.216020 21731 raft_consensus.cc:2798] T 0cc4fcda876242c396b804edebebb01f P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:57:31.216506 21731 raft_consensus.cc:695] T 0cc4fcda876242c396b804edebebb01f P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Becoming Leader. State: Replica: 8bf6e26e98b14566899578a990b27d47, State: Running, Role: LEADER
I20250109 08:57:31.217260 21731 consensus_queue.cc:237] T 0cc4fcda876242c396b804edebebb01f P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.221719 21627 leader_election.cc:304] T 90a4a2381cbe40f0b204be325ee9fea1 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, a2b1a692435241ee988e0bd1661cd4c3; no voters: 8bf6e26e98b14566899578a990b27d47
I20250109 08:57:31.222376 21732 raft_consensus.cc:2798] T 90a4a2381cbe40f0b204be325ee9fea1 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:57:31.222754 21732 raft_consensus.cc:491] T 90a4a2381cbe40f0b204be325ee9fea1 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:57:31.223156 21732 raft_consensus.cc:3054] T 90a4a2381cbe40f0b204be325ee9fea1 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:31.225279 21702 tablet_bootstrap.cc:654] T af273bab41cc4a9da780147eebfd52ec P a2b1a692435241ee988e0bd1661cd4c3: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:31.233564 21703 ts_tablet_manager.cc:1428] T af273bab41cc4a9da780147eebfd52ec P 8bf6e26e98b14566899578a990b27d47: Time spent starting tablet: real 0.034s user 0.006s sys 0.000s
I20250109 08:57:31.234510 21703 tablet_bootstrap.cc:492] T 0f3bde1a1ccd4c359a2e80feca968453 P 8bf6e26e98b14566899578a990b27d47: Bootstrap starting.
I20250109 08:57:31.234959 21704 raft_consensus.cc:357] T e03846e461974d0d8944c344e715b86a P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.235664 21704 raft_consensus.cc:383] T e03846e461974d0d8944c344e715b86a P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:31.235939 21704 raft_consensus.cc:738] T e03846e461974d0d8944c344e715b86a P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Initialized, Role: FOLLOWER
I20250109 08:57:31.236541 21704 consensus_queue.cc:260] T e03846e461974d0d8944c344e715b86a P 787ddcc824c2490a808fc5a53452dc5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.239336 21704 ts_tablet_manager.cc:1428] T e03846e461974d0d8944c344e715b86a P 787ddcc824c2490a808fc5a53452dc5e: Time spent starting tablet: real 0.026s user 0.006s sys 0.000s
I20250109 08:57:31.239039 21411 catalog_manager.cc:5526] T 75edb5d04fa14201ad711c1d95c81f64 P 787ddcc824c2490a808fc5a53452dc5e reported cstate change: term changed from 0 to 1, leader changed from <none> to 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195). New cstate: current_term: 1 leader_uuid: "787ddcc824c2490a808fc5a53452dc5e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: HEALTHY } } }
I20250109 08:57:31.240581 21704 tablet_bootstrap.cc:492] T af273bab41cc4a9da780147eebfd52ec P 787ddcc824c2490a808fc5a53452dc5e: Bootstrap starting.
I20250109 08:57:31.243358 21703 tablet_bootstrap.cc:654] T 0f3bde1a1ccd4c359a2e80feca968453 P 8bf6e26e98b14566899578a990b27d47: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:31.247016 21702 tablet_bootstrap.cc:492] T af273bab41cc4a9da780147eebfd52ec P a2b1a692435241ee988e0bd1661cd4c3: No bootstrap required, opened a new log
I20250109 08:57:31.247105 21704 tablet_bootstrap.cc:654] T af273bab41cc4a9da780147eebfd52ec P 787ddcc824c2490a808fc5a53452dc5e: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:31.247509 21702 ts_tablet_manager.cc:1397] T af273bab41cc4a9da780147eebfd52ec P a2b1a692435241ee988e0bd1661cd4c3: Time spent bootstrapping tablet: real 0.076s user 0.007s sys 0.003s
I20250109 08:57:31.250036 21702 raft_consensus.cc:357] T af273bab41cc4a9da780147eebfd52ec P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.250689 21702 raft_consensus.cc:383] T af273bab41cc4a9da780147eebfd52ec P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:31.250945 21702 raft_consensus.cc:738] T af273bab41cc4a9da780147eebfd52ec P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2b1a692435241ee988e0bd1661cd4c3, State: Initialized, Role: FOLLOWER
I20250109 08:57:31.251611 21702 consensus_queue.cc:260] T af273bab41cc4a9da780147eebfd52ec P a2b1a692435241ee988e0bd1661cd4c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.252187 21710 raft_consensus.cc:491] T d355d166de534fb4a50c9f48f2df7469 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:31.252682 21710 raft_consensus.cc:513] T d355d166de534fb4a50c9f48f2df7469 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.254168 21702 ts_tablet_manager.cc:1428] T af273bab41cc4a9da780147eebfd52ec P a2b1a692435241ee988e0bd1661cd4c3: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
I20250109 08:57:31.254581 21710 leader_election.cc:290] T d355d166de534fb4a50c9f48f2df7469 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
I20250109 08:57:31.255089 21702 tablet_bootstrap.cc:492] T 0cc4fcda876242c396b804edebebb01f P a2b1a692435241ee988e0bd1661cd4c3: Bootstrap starting.
I20250109 08:57:31.258392 21512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d355d166de534fb4a50c9f48f2df7469" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47" is_pre_election: true
I20250109 08:57:31.259425 21512 raft_consensus.cc:2463] T d355d166de534fb4a50c9f48f2df7469 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 0.
I20250109 08:57:31.262105 21412 catalog_manager.cc:5526] T 0cc4fcda876242c396b804edebebb01f P 8bf6e26e98b14566899578a990b27d47 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8bf6e26e98b14566899578a990b27d47 (127.19.226.193). New cstate: current_term: 1 leader_uuid: "8bf6e26e98b14566899578a990b27d47" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: UNKNOWN } } }
I20250109 08:57:31.263234 21731 raft_consensus.cc:491] T 90978f185a374feba3bc19b54f9d97f5 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:31.261312 21587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d355d166de534fb4a50c9f48f2df7469" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3" is_pre_election: true
I20250109 08:57:31.264901 21731 raft_consensus.cc:513] T 90978f185a374feba3bc19b54f9d97f5 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.267689 21732 raft_consensus.cc:513] T 90a4a2381cbe40f0b204be325ee9fea1 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.264566 21626 leader_election.cc:304] T d355d166de534fb4a50c9f48f2df7469 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, 8bf6e26e98b14566899578a990b27d47; no voters:
I20250109 08:57:31.271890 21710 raft_consensus.cc:2798] T d355d166de534fb4a50c9f48f2df7469 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:57:31.272388 21732 leader_election.cc:290] T 90a4a2381cbe40f0b204be325ee9fea1 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 election: Requested vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
I20250109 08:57:31.272241 21710 raft_consensus.cc:491] T d355d166de534fb4a50c9f48f2df7469 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:57:31.273411 21710 raft_consensus.cc:3054] T d355d166de534fb4a50c9f48f2df7469 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:31.275061 21589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90a4a2381cbe40f0b204be325ee9fea1" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3"
I20250109 08:57:31.275684 21589 raft_consensus.cc:3054] T 90a4a2381cbe40f0b204be325ee9fea1 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:31.269117 21588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90978f185a374feba3bc19b54f9d97f5" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3" is_pre_election: true
I20250109 08:57:31.269744 21587 raft_consensus.cc:2463] T d355d166de534fb4a50c9f48f2df7469 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 0.
I20250109 08:57:31.270018 21731 leader_election.cc:290] T 90978f185a374feba3bc19b54f9d97f5 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:31.270715 21702 tablet_bootstrap.cc:654] T 0cc4fcda876242c396b804edebebb01f P a2b1a692435241ee988e0bd1661cd4c3: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:31.283334 21742 raft_consensus.cc:491] T df9715570a6844d9a2686307db16eff1 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:31.284123 21704 tablet_bootstrap.cc:492] T af273bab41cc4a9da780147eebfd52ec P 787ddcc824c2490a808fc5a53452dc5e: No bootstrap required, opened a new log
I20250109 08:57:31.284034 21742 raft_consensus.cc:513] T df9715570a6844d9a2686307db16eff1 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.284566 21704 ts_tablet_manager.cc:1397] T af273bab41cc4a9da780147eebfd52ec P 787ddcc824c2490a808fc5a53452dc5e: Time spent bootstrapping tablet: real 0.044s user 0.010s sys 0.006s
I20250109 08:57:31.288223 21512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90a4a2381cbe40f0b204be325ee9fea1" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47"
I20250109 08:57:31.288981 21704 raft_consensus.cc:357] T af273bab41cc4a9da780147eebfd52ec P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
W20250109 08:57:31.290467 21626 leader_election.cc:343] T 90a4a2381cbe40f0b204be325ee9fea1 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:31.290683 21704 raft_consensus.cc:383] T af273bab41cc4a9da780147eebfd52ec P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:31.291030 21704 raft_consensus.cc:738] T af273bab41cc4a9da780147eebfd52ec P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Initialized, Role: FOLLOWER
I20250109 08:57:31.291648 21704 consensus_queue.cc:260] T af273bab41cc4a9da780147eebfd52ec P 787ddcc824c2490a808fc5a53452dc5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
W20250109 08:57:31.289168 21478 leader_election.cc:343] T 90978f185a374feba3bc19b54f9d97f5 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:31.288558 21742 leader_election.cc:290] T df9715570a6844d9a2686307db16eff1 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:31.289283 21661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90978f185a374feba3bc19b54f9d97f5" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e" is_pre_election: true
I20250109 08:57:31.299388 21589 raft_consensus.cc:2463] T 90a4a2381cbe40f0b204be325ee9fea1 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 1.
I20250109 08:57:31.301117 21661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df9715570a6844d9a2686307db16eff1" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e" is_pre_election: true
I20250109 08:57:31.302026 21627 leader_election.cc:304] T 90a4a2381cbe40f0b204be325ee9fea1 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, a2b1a692435241ee988e0bd1661cd4c3; no voters: 8bf6e26e98b14566899578a990b27d47
W20250109 08:57:31.302264 21477 leader_election.cc:343] T 90978f185a374feba3bc19b54f9d97f5 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:31.302760 21477 leader_election.cc:304] T 90978f185a374feba3bc19b54f9d97f5 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8bf6e26e98b14566899578a990b27d47; no voters: 787ddcc824c2490a808fc5a53452dc5e, a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:57:31.309259 21588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df9715570a6844d9a2686307db16eff1" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3" is_pre_election: true
I20250109 08:57:31.309561 21731 raft_consensus.cc:2743] T 90978f185a374feba3bc19b54f9d97f5 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:57:31.302906 21732 raft_consensus.cc:2798] T 90a4a2381cbe40f0b204be325ee9fea1 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:57:31.312674 21704 ts_tablet_manager.cc:1428] T af273bab41cc4a9da780147eebfd52ec P 787ddcc824c2490a808fc5a53452dc5e: Time spent starting tablet: real 0.028s user 0.006s sys 0.000s
W20250109 08:57:31.312881 21477 leader_election.cc:343] T df9715570a6844d9a2686307db16eff1 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731): Illegal state: must be running to vote when last-logged opid is not known
W20250109 08:57:31.315392 21478 leader_election.cc:343] T df9715570a6844d9a2686307db16eff1 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:31.319084 21901 raft_consensus.cc:491] T e03846e461974d0d8944c344e715b86a P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:31.314236 21732 raft_consensus.cc:695] T 90a4a2381cbe40f0b204be325ee9fea1 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Becoming Leader. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Running, Role: LEADER
I20250109 08:57:31.320194 21478 leader_election.cc:304] T df9715570a6844d9a2686307db16eff1 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8bf6e26e98b14566899578a990b27d47; no voters: 787ddcc824c2490a808fc5a53452dc5e, a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:57:31.323035 21731 raft_consensus.cc:2743] T df9715570a6844d9a2686307db16eff1 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:57:31.320060 21901 raft_consensus.cc:513] T e03846e461974d0d8944c344e715b86a P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.324570 21704 tablet_bootstrap.cc:492] T e8ff570914d74df8b87296010e9b864f P 787ddcc824c2490a808fc5a53452dc5e: Bootstrap starting.
I20250109 08:57:31.323144 21710 raft_consensus.cc:513] T d355d166de534fb4a50c9f48f2df7469 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.325752 21732 consensus_queue.cc:237] T 90a4a2381cbe40f0b204be325ee9fea1 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.326709 21759 raft_consensus.cc:491] T ddd14cdbefd64936960e9ffb5b38b5f2 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:31.327214 21759 raft_consensus.cc:513] T ddd14cdbefd64936960e9ffb5b38b5f2 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.327795 21512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e03846e461974d0d8944c344e715b86a" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47" is_pre_election: true
I20250109 08:57:31.328943 21703 tablet_bootstrap.cc:492] T 0f3bde1a1ccd4c359a2e80feca968453 P 8bf6e26e98b14566899578a990b27d47: No bootstrap required, opened a new log
W20250109 08:57:31.329195 21626 leader_election.cc:343] T e03846e461974d0d8944c344e715b86a P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:31.329387 21703 ts_tablet_manager.cc:1397] T 0f3bde1a1ccd4c359a2e80feca968453 P 8bf6e26e98b14566899578a990b27d47: Time spent bootstrapping tablet: real 0.095s user 0.026s sys 0.030s
I20250109 08:57:31.329596 21759 leader_election.cc:290] T ddd14cdbefd64936960e9ffb5b38b5f2 P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:31.330822 21512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ddd14cdbefd64936960e9ffb5b38b5f2" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47" is_pre_election: true
I20250109 08:57:31.333112 21661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ddd14cdbefd64936960e9ffb5b38b5f2" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e" is_pre_election: true
I20250109 08:57:31.333997 21661 raft_consensus.cc:2388] T ddd14cdbefd64936960e9ffb5b38b5f2 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2b1a692435241ee988e0bd1661cd4c3 in current term 1: Already voted for candidate 8bf6e26e98b14566899578a990b27d47 in this term.
I20250109 08:57:31.338017 21552 leader_election.cc:304] T ddd14cdbefd64936960e9ffb5b38b5f2 P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2b1a692435241ee988e0bd1661cd4c3; no voters: 787ddcc824c2490a808fc5a53452dc5e, 8bf6e26e98b14566899578a990b27d47
I20250109 08:57:31.339010 21819 raft_consensus.cc:3054] T ddd14cdbefd64936960e9ffb5b38b5f2 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:31.339783 21901 leader_election.cc:290] T e03846e461974d0d8944c344e715b86a P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
I20250109 08:57:31.339943 21704 tablet_bootstrap.cc:654] T e8ff570914d74df8b87296010e9b864f P 787ddcc824c2490a808fc5a53452dc5e: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:31.343418 21703 raft_consensus.cc:357] T 0f3bde1a1ccd4c359a2e80feca968453 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.344233 21703 raft_consensus.cc:383] T 0f3bde1a1ccd4c359a2e80feca968453 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:31.344566 21703 raft_consensus.cc:738] T 0f3bde1a1ccd4c359a2e80feca968453 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bf6e26e98b14566899578a990b27d47, State: Initialized, Role: FOLLOWER
I20250109 08:57:31.345229 21703 consensus_queue.cc:260] T 0f3bde1a1ccd4c359a2e80feca968453 P 8bf6e26e98b14566899578a990b27d47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.351651 21819 raft_consensus.cc:2743] T ddd14cdbefd64936960e9ffb5b38b5f2 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:57:31.353425 21589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e03846e461974d0d8944c344e715b86a" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3" is_pre_election: true
I20250109 08:57:31.354238 21703 ts_tablet_manager.cc:1428] T 0f3bde1a1ccd4c359a2e80feca968453 P 8bf6e26e98b14566899578a990b27d47: Time spent starting tablet: real 0.024s user 0.005s sys 0.001s
I20250109 08:57:31.353605 21758 consensus_queue.cc:1035] T 89e1eeb24fea483eb63bd98af66e0b6b P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20250109 08:57:31.355120 21703 tablet_bootstrap.cc:492] T e8ff570914d74df8b87296010e9b864f P 8bf6e26e98b14566899578a990b27d47: Bootstrap starting.
W20250109 08:57:31.357462 21627 leader_election.cc:343] T e03846e461974d0d8944c344e715b86a P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:31.357887 21627 leader_election.cc:304] T e03846e461974d0d8944c344e715b86a P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e; no voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:57:31.358611 21758 raft_consensus.cc:2743] T e03846e461974d0d8944c344e715b86a P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:57:31.360323 21514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d355d166de534fb4a50c9f48f2df7469" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47"
I20250109 08:57:31.361039 21514 raft_consensus.cc:3054] T d355d166de534fb4a50c9f48f2df7469 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:31.372299 21742 consensus_queue.cc:1035] T ddd14cdbefd64936960e9ffb5b38b5f2 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:57:31.382839 21732 consensus_queue.cc:1035] T 89e1eeb24fea483eb63bd98af66e0b6b P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:57:31.381759 21408 catalog_manager.cc:5526] T 90a4a2381cbe40f0b204be325ee9fea1 P 787ddcc824c2490a808fc5a53452dc5e reported cstate change: term changed from 0 to 1, leader changed from <none> to 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195). New cstate: current_term: 1 leader_uuid: "787ddcc824c2490a808fc5a53452dc5e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: HEALTHY } } }
I20250109 08:57:31.385849 21703 tablet_bootstrap.cc:654] T e8ff570914d74df8b87296010e9b864f P 8bf6e26e98b14566899578a990b27d47: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:31.387961 21514 raft_consensus.cc:2463] T d355d166de534fb4a50c9f48f2df7469 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 1.
I20250109 08:57:31.389220 21626 leader_election.cc:304] T d355d166de534fb4a50c9f48f2df7469 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, 8bf6e26e98b14566899578a990b27d47; no voters:
I20250109 08:57:31.390089 21732 raft_consensus.cc:2798] T d355d166de534fb4a50c9f48f2df7469 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:57:31.390552 21732 raft_consensus.cc:695] T d355d166de534fb4a50c9f48f2df7469 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Becoming Leader. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Running, Role: LEADER
I20250109 08:57:31.391861 21732 consensus_queue.cc:237] T d355d166de534fb4a50c9f48f2df7469 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.395174 21710 leader_election.cc:290] T d355d166de534fb4a50c9f48f2df7469 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 election: Requested vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
I20250109 08:57:31.396169 21587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d355d166de534fb4a50c9f48f2df7469" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3"
I20250109 08:57:31.396811 21587 raft_consensus.cc:3054] T d355d166de534fb4a50c9f48f2df7469 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:31.402491 21702 tablet_bootstrap.cc:492] T 0cc4fcda876242c396b804edebebb01f P a2b1a692435241ee988e0bd1661cd4c3: No bootstrap required, opened a new log
I20250109 08:57:31.402974 21702 ts_tablet_manager.cc:1397] T 0cc4fcda876242c396b804edebebb01f P a2b1a692435241ee988e0bd1661cd4c3: Time spent bootstrapping tablet: real 0.148s user 0.007s sys 0.017s
I20250109 08:57:31.406148 21842 consensus_queue.cc:1035] T ddd14cdbefd64936960e9ffb5b38b5f2 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:57:31.406740 21587 raft_consensus.cc:2463] T d355d166de534fb4a50c9f48f2df7469 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 1.
I20250109 08:57:31.416918 21702 raft_consensus.cc:357] T 0cc4fcda876242c396b804edebebb01f P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.417623 21702 raft_consensus.cc:383] T 0cc4fcda876242c396b804edebebb01f P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:31.417886 21702 raft_consensus.cc:738] T 0cc4fcda876242c396b804edebebb01f P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2b1a692435241ee988e0bd1661cd4c3, State: Initialized, Role: FOLLOWER
I20250109 08:57:31.418502 21702 consensus_queue.cc:260] T 0cc4fcda876242c396b804edebebb01f P a2b1a692435241ee988e0bd1661cd4c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.427891 21702 ts_tablet_manager.cc:1428] T 0cc4fcda876242c396b804edebebb01f P a2b1a692435241ee988e0bd1661cd4c3: Time spent starting tablet: real 0.025s user 0.005s sys 0.002s
I20250109 08:57:31.428773 21702 tablet_bootstrap.cc:492] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P a2b1a692435241ee988e0bd1661cd4c3: Bootstrap starting.
I20250109 08:57:31.438699 21704 tablet_bootstrap.cc:492] T e8ff570914d74df8b87296010e9b864f P 787ddcc824c2490a808fc5a53452dc5e: No bootstrap required, opened a new log
I20250109 08:57:31.443439 21704 ts_tablet_manager.cc:1397] T e8ff570914d74df8b87296010e9b864f P 787ddcc824c2490a808fc5a53452dc5e: Time spent bootstrapping tablet: real 0.119s user 0.020s sys 0.057s
I20250109 08:57:31.442685 21702 tablet_bootstrap.cc:654] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P a2b1a692435241ee988e0bd1661cd4c3: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:31.440024 21412 catalog_manager.cc:5526] T d355d166de534fb4a50c9f48f2df7469 P 787ddcc824c2490a808fc5a53452dc5e reported cstate change: term changed from 0 to 1, leader changed from <none> to 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195). New cstate: current_term: 1 leader_uuid: "787ddcc824c2490a808fc5a53452dc5e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: HEALTHY } } }
I20250109 08:57:31.446669 21703 tablet_bootstrap.cc:492] T e8ff570914d74df8b87296010e9b864f P 8bf6e26e98b14566899578a990b27d47: No bootstrap required, opened a new log
I20250109 08:57:31.454514 21703 ts_tablet_manager.cc:1397] T e8ff570914d74df8b87296010e9b864f P 8bf6e26e98b14566899578a990b27d47: Time spent bootstrapping tablet: real 0.100s user 0.006s sys 0.013s
I20250109 08:57:31.457043 21703 raft_consensus.cc:357] T e8ff570914d74df8b87296010e9b864f P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.457751 21819 raft_consensus.cc:491] T 0cc4fcda876242c396b804edebebb01f P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:31.447635 21704 raft_consensus.cc:357] T e8ff570914d74df8b87296010e9b864f P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.458555 21704 raft_consensus.cc:383] T e8ff570914d74df8b87296010e9b864f P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:31.458309 21819 raft_consensus.cc:513] T 0cc4fcda876242c396b804edebebb01f P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.458827 21704 raft_consensus.cc:738] T e8ff570914d74df8b87296010e9b864f P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Initialized, Role: FOLLOWER
I20250109 08:57:31.461647 21704 consensus_queue.cc:260] T e8ff570914d74df8b87296010e9b864f P 787ddcc824c2490a808fc5a53452dc5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.463924 21704 ts_tablet_manager.cc:1428] T e8ff570914d74df8b87296010e9b864f P 787ddcc824c2490a808fc5a53452dc5e: Time spent starting tablet: real 0.020s user 0.005s sys 0.001s
I20250109 08:57:31.457720 21703 raft_consensus.cc:383] T e8ff570914d74df8b87296010e9b864f P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:31.464463 21703 raft_consensus.cc:738] T e8ff570914d74df8b87296010e9b864f P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bf6e26e98b14566899578a990b27d47, State: Initialized, Role: FOLLOWER
I20250109 08:57:31.465062 21703 consensus_queue.cc:260] T e8ff570914d74df8b87296010e9b864f P 8bf6e26e98b14566899578a990b27d47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.466830 21662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cc4fcda876242c396b804edebebb01f" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e" is_pre_election: true
I20250109 08:57:31.467327 21703 ts_tablet_manager.cc:1428] T e8ff570914d74df8b87296010e9b864f P 8bf6e26e98b14566899578a990b27d47: Time spent starting tablet: real 0.012s user 0.004s sys 0.000s
I20250109 08:57:31.467617 21662 raft_consensus.cc:2388] T 0cc4fcda876242c396b804edebebb01f P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2b1a692435241ee988e0bd1661cd4c3 in current term 1: Already voted for candidate 8bf6e26e98b14566899578a990b27d47 in this term.
I20250109 08:57:31.466847 21514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cc4fcda876242c396b804edebebb01f" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47" is_pre_election: true
I20250109 08:57:31.468570 21703 tablet_bootstrap.cc:492] T c99aacce09d44f889be5480854a22760 P 8bf6e26e98b14566899578a990b27d47: Bootstrap starting.
I20250109 08:57:31.468510 21907 raft_consensus.cc:491] T 0f3bde1a1ccd4c359a2e80feca968453 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:31.469067 21907 raft_consensus.cc:513] T 0f3bde1a1ccd4c359a2e80feca968453 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.471383 21907 leader_election.cc:290] T 0f3bde1a1ccd4c359a2e80feca968453 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
I20250109 08:57:31.472661 21514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f3bde1a1ccd4c359a2e80feca968453" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47" is_pre_election: true
I20250109 08:57:31.472976 21589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f3bde1a1ccd4c359a2e80feca968453" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3" is_pre_election: true
I20250109 08:57:31.473405 21514 raft_consensus.cc:2463] T 0f3bde1a1ccd4c359a2e80feca968453 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 0.
W20250109 08:57:31.474149 21627 leader_election.cc:343] T 0f3bde1a1ccd4c359a2e80feca968453 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:31.474474 21552 leader_election.cc:304] T 0cc4fcda876242c396b804edebebb01f P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2b1a692435241ee988e0bd1661cd4c3; no voters: 787ddcc824c2490a808fc5a53452dc5e, 8bf6e26e98b14566899578a990b27d47
I20250109 08:57:31.475246 21709 raft_consensus.cc:3054] T 0cc4fcda876242c396b804edebebb01f P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:31.475328 21703 tablet_bootstrap.cc:654] T c99aacce09d44f889be5480854a22760 P 8bf6e26e98b14566899578a990b27d47: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:31.479429 21626 leader_election.cc:304] T 0f3bde1a1ccd4c359a2e80feca968453 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, 8bf6e26e98b14566899578a990b27d47; no voters: a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:57:31.480741 21907 raft_consensus.cc:2798] T 0f3bde1a1ccd4c359a2e80feca968453 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:57:31.481139 21907 raft_consensus.cc:491] T 0f3bde1a1ccd4c359a2e80feca968453 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:57:31.481467 21907 raft_consensus.cc:3054] T 0f3bde1a1ccd4c359a2e80feca968453 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:31.460110 21819 leader_election.cc:290] T 0cc4fcda876242c396b804edebebb01f P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:31.484478 21702 tablet_bootstrap.cc:492] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P a2b1a692435241ee988e0bd1661cd4c3: No bootstrap required, opened a new log
I20250109 08:57:31.485431 21702 ts_tablet_manager.cc:1397] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P a2b1a692435241ee988e0bd1661cd4c3: Time spent bootstrapping tablet: real 0.057s user 0.005s sys 0.008s
I20250109 08:57:31.448689 21732 raft_consensus.cc:491] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:31.486994 21732 raft_consensus.cc:513] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.484988 21704 tablet_bootstrap.cc:492] T df9715570a6844d9a2686307db16eff1 P 787ddcc824c2490a808fc5a53452dc5e: Bootstrap starting.
I20250109 08:57:31.491500 21709 raft_consensus.cc:2743] T 0cc4fcda876242c396b804edebebb01f P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:57:31.493155 21514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b54d6381a2cc41dfb59b1f6b99c0c5d4" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47" is_pre_election: true
I20250109 08:57:31.493953 21732 leader_election.cc:290] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
I20250109 08:57:31.497273 21589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b54d6381a2cc41dfb59b1f6b99c0c5d4" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3" is_pre_election: true
I20250109 08:57:31.500888 21704 tablet_bootstrap.cc:654] T df9715570a6844d9a2686307db16eff1 P 787ddcc824c2490a808fc5a53452dc5e: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:31.506994 21702 raft_consensus.cc:357] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.507719 21702 raft_consensus.cc:383] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:31.501415 21514 raft_consensus.cc:2463] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 0.
I20250109 08:57:31.508038 21702 raft_consensus.cc:738] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2b1a692435241ee988e0bd1661cd4c3, State: Initialized, Role: FOLLOWER
I20250109 08:57:31.508641 21702 consensus_queue.cc:260] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P a2b1a692435241ee988e0bd1661cd4c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.512137 21703 tablet_bootstrap.cc:492] T c99aacce09d44f889be5480854a22760 P 8bf6e26e98b14566899578a990b27d47: No bootstrap required, opened a new log
I20250109 08:57:31.512609 21703 ts_tablet_manager.cc:1397] T c99aacce09d44f889be5480854a22760 P 8bf6e26e98b14566899578a990b27d47: Time spent bootstrapping tablet: real 0.044s user 0.028s sys 0.006s
I20250109 08:57:31.514830 21703 raft_consensus.cc:357] T c99aacce09d44f889be5480854a22760 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.515555 21703 raft_consensus.cc:383] T c99aacce09d44f889be5480854a22760 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:31.515887 21703 raft_consensus.cc:738] T c99aacce09d44f889be5480854a22760 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bf6e26e98b14566899578a990b27d47, State: Initialized, Role: FOLLOWER
I20250109 08:57:31.509225 21626 leader_election.cc:304] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, 8bf6e26e98b14566899578a990b27d47; no voters:
I20250109 08:57:31.511276 21589 raft_consensus.cc:2463] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 0.
I20250109 08:57:31.516587 21703 consensus_queue.cc:260] T c99aacce09d44f889be5480854a22760 P 8bf6e26e98b14566899578a990b27d47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.517099 21732 raft_consensus.cc:2798] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:57:31.517616 21732 raft_consensus.cc:491] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:57:31.517997 21732 raft_consensus.cc:3054] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:31.519682 21702 ts_tablet_manager.cc:1428] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P a2b1a692435241ee988e0bd1661cd4c3: Time spent starting tablet: real 0.034s user 0.003s sys 0.003s
I20250109 08:57:31.522105 21702 tablet_bootstrap.cc:492] T df9715570a6844d9a2686307db16eff1 P a2b1a692435241ee988e0bd1661cd4c3: Bootstrap starting.
I20250109 08:57:31.524547 21907 raft_consensus.cc:513] T 0f3bde1a1ccd4c359a2e80feca968453 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.527721 21514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f3bde1a1ccd4c359a2e80feca968453" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47"
I20250109 08:57:31.528314 21514 raft_consensus.cc:3054] T 0f3bde1a1ccd4c359a2e80feca968453 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:31.528888 21732 raft_consensus.cc:513] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.531134 21732 leader_election.cc:290] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 election: Requested vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
I20250109 08:57:31.518687 21703 ts_tablet_manager.cc:1428] T c99aacce09d44f889be5480854a22760 P 8bf6e26e98b14566899578a990b27d47: Time spent starting tablet: real 0.006s user 0.003s sys 0.001s
I20250109 08:57:31.532138 21589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b54d6381a2cc41dfb59b1f6b99c0c5d4" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3"
I20250109 08:57:31.532770 21589 raft_consensus.cc:3054] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:31.532655 21513 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b54d6381a2cc41dfb59b1f6b99c0c5d4" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47"
I20250109 08:57:31.533401 21907 leader_election.cc:290] T 0f3bde1a1ccd4c359a2e80feca968453 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 election: Requested vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
I20250109 08:57:31.532435 21703 tablet_bootstrap.cc:492] T af330dcd4767423ea4c179be7f88d0c4 P 8bf6e26e98b14566899578a990b27d47: Bootstrap starting.
I20250109 08:57:31.533318 21513 raft_consensus.cc:3054] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:31.537170 21588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f3bde1a1ccd4c359a2e80feca968453" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3"
W20250109 08:57:31.539017 21627 leader_election.cc:343] T 0f3bde1a1ccd4c359a2e80feca968453 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:31.541188 21704 tablet_bootstrap.cc:492] T df9715570a6844d9a2686307db16eff1 P 787ddcc824c2490a808fc5a53452dc5e: No bootstrap required, opened a new log
I20250109 08:57:31.541319 21702 tablet_bootstrap.cc:654] T df9715570a6844d9a2686307db16eff1 P a2b1a692435241ee988e0bd1661cd4c3: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:31.541643 21704 ts_tablet_manager.cc:1397] T df9715570a6844d9a2686307db16eff1 P 787ddcc824c2490a808fc5a53452dc5e: Time spent bootstrapping tablet: real 0.057s user 0.007s sys 0.004s
I20250109 08:57:31.543941 21589 raft_consensus.cc:2463] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 1.
I20250109 08:57:31.544517 21704 raft_consensus.cc:357] T df9715570a6844d9a2686307db16eff1 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.544997 21627 leader_election.cc:304] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, a2b1a692435241ee988e0bd1661cd4c3; no voters:
I20250109 08:57:31.545199 21704 raft_consensus.cc:383] T df9715570a6844d9a2686307db16eff1 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:31.545496 21704 raft_consensus.cc:738] T df9715570a6844d9a2686307db16eff1 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Initialized, Role: FOLLOWER
I20250109 08:57:31.546049 21711 raft_consensus.cc:491] T 126a163a89c448be89ccb9490e9f0168 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:31.546146 21704 consensus_queue.cc:260] T df9715570a6844d9a2686307db16eff1 P 787ddcc824c2490a808fc5a53452dc5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.546528 21711 raft_consensus.cc:513] T 126a163a89c448be89ccb9490e9f0168 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.547886 21709 raft_consensus.cc:491] T af273bab41cc4a9da780147eebfd52ec P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:31.548045 21514 raft_consensus.cc:2463] T 0f3bde1a1ccd4c359a2e80feca968453 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 1.
I20250109 08:57:31.548583 21703 tablet_bootstrap.cc:654] T af330dcd4767423ea4c179be7f88d0c4 P 8bf6e26e98b14566899578a990b27d47: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:31.548317 21709 raft_consensus.cc:513] T af273bab41cc4a9da780147eebfd52ec P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.548985 21626 leader_election.cc:304] T 0f3bde1a1ccd4c359a2e80feca968453 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, 8bf6e26e98b14566899578a990b27d47; no voters: a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:57:31.549860 21710 raft_consensus.cc:2798] T 0f3bde1a1ccd4c359a2e80feca968453 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:57:31.550308 21710 raft_consensus.cc:695] T 0f3bde1a1ccd4c359a2e80feca968453 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Becoming Leader. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Running, Role: LEADER
I20250109 08:57:31.550706 21709 leader_election.cc:290] T af273bab41cc4a9da780147eebfd52ec P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:31.550921 21661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "126a163a89c448be89ccb9490e9f0168" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e" is_pre_election: true
I20250109 08:57:31.551086 21710 consensus_queue.cc:237] T 0f3bde1a1ccd4c359a2e80feca968453 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.552241 21661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af273bab41cc4a9da780147eebfd52ec" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e" is_pre_election: true
I20250109 08:57:31.553416 21661 raft_consensus.cc:2463] T af273bab41cc4a9da780147eebfd52ec P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2b1a692435241ee988e0bd1661cd4c3 in term 0.
I20250109 08:57:31.554534 21552 leader_election.cc:304] T af273bab41cc4a9da780147eebfd52ec P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, a2b1a692435241ee988e0bd1661cd4c3; no voters:
I20250109 08:57:31.554776 21731 raft_consensus.cc:491] T af273bab41cc4a9da780147eebfd52ec P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:31.555197 21731 raft_consensus.cc:513] T af273bab41cc4a9da780147eebfd52ec P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.548457 21711 leader_election.cc:290] T 126a163a89c448be89ccb9490e9f0168 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
W20250109 08:57:31.552508 21477 leader_election.cc:343] T 126a163a89c448be89ccb9490e9f0168 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:31.557037 21907 raft_consensus.cc:2798] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:57:31.557557 21731 leader_election.cc:290] T af273bab41cc4a9da780147eebfd52ec P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:31.559011 21512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af273bab41cc4a9da780147eebfd52ec" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47" is_pre_election: true
I20250109 08:57:31.559394 21588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af273bab41cc4a9da780147eebfd52ec" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3" is_pre_election: true
I20250109 08:57:31.559608 21512 raft_consensus.cc:2463] T af273bab41cc4a9da780147eebfd52ec P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2b1a692435241ee988e0bd1661cd4c3 in term 0.
I20250109 08:57:31.556015 21819 raft_consensus.cc:2798] T af273bab41cc4a9da780147eebfd52ec P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:57:31.561511 21819 raft_consensus.cc:491] T af273bab41cc4a9da780147eebfd52ec P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:57:31.543835 21513 raft_consensus.cc:2463] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 1.
I20250109 08:57:31.561867 21819 raft_consensus.cc:3054] T af273bab41cc4a9da780147eebfd52ec P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:31.565366 21661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af273bab41cc4a9da780147eebfd52ec" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e" is_pre_election: true
I20250109 08:57:31.566011 21661 raft_consensus.cc:2463] T af273bab41cc4a9da780147eebfd52ec P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8bf6e26e98b14566899578a990b27d47 in term 0.
I20250109 08:57:31.570210 21704 ts_tablet_manager.cc:1428] T df9715570a6844d9a2686307db16eff1 P 787ddcc824c2490a808fc5a53452dc5e: Time spent starting tablet: real 0.028s user 0.004s sys 0.003s
I20250109 08:57:31.567304 21477 leader_election.cc:304] T af273bab41cc4a9da780147eebfd52ec P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, 8bf6e26e98b14566899578a990b27d47; no voters:
I20250109 08:57:31.571630 21704 tablet_bootstrap.cc:492] T af330dcd4767423ea4c179be7f88d0c4 P 787ddcc824c2490a808fc5a53452dc5e: Bootstrap starting.
I20250109 08:57:31.571964 21711 raft_consensus.cc:2798] T af273bab41cc4a9da780147eebfd52ec P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:57:31.572351 21711 raft_consensus.cc:491] T af273bab41cc4a9da780147eebfd52ec P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:57:31.573220 21711 raft_consensus.cc:3054] T af273bab41cc4a9da780147eebfd52ec P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:31.570115 21819 raft_consensus.cc:513] T af273bab41cc4a9da780147eebfd52ec P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.576009 21588 raft_consensus.cc:2388] T af273bab41cc4a9da780147eebfd52ec P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8bf6e26e98b14566899578a990b27d47 in current term 1: Already voted for candidate a2b1a692435241ee988e0bd1661cd4c3 in this term.
I20250109 08:57:31.577422 21514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af273bab41cc4a9da780147eebfd52ec" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47"
I20250109 08:57:31.576287 21819 leader_election.cc:290] T af273bab41cc4a9da780147eebfd52ec P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 election: Requested vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:31.558506 21907 raft_consensus.cc:695] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Becoming Leader. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Running, Role: LEADER
I20250109 08:57:31.581112 21907 consensus_queue.cc:237] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.602880 21589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "126a163a89c448be89ccb9490e9f0168" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3" is_pre_election: true
I20250109 08:57:31.604938 21711 raft_consensus.cc:513] T af273bab41cc4a9da780147eebfd52ec P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.618219 21514 raft_consensus.cc:2388] T af273bab41cc4a9da780147eebfd52ec P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate a2b1a692435241ee988e0bd1661cd4c3 in current term 1: Already voted for candidate 8bf6e26e98b14566899578a990b27d47 in this term.
I20250109 08:57:31.618973 21711 leader_election.cc:290] T af273bab41cc4a9da780147eebfd52ec P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 election: Requested vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:31.619740 21588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af273bab41cc4a9da780147eebfd52ec" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3"
I20250109 08:57:31.620980 21588 raft_consensus.cc:2388] T af273bab41cc4a9da780147eebfd52ec P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 8bf6e26e98b14566899578a990b27d47 in current term 1: Already voted for candidate a2b1a692435241ee988e0bd1661cd4c3 in this term.
I20250109 08:57:31.621150 21662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af273bab41cc4a9da780147eebfd52ec" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e"
I20250109 08:57:31.621707 21662 raft_consensus.cc:3054] T af273bab41cc4a9da780147eebfd52ec P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:31.622225 21702 tablet_bootstrap.cc:492] T df9715570a6844d9a2686307db16eff1 P a2b1a692435241ee988e0bd1661cd4c3: No bootstrap required, opened a new log
I20250109 08:57:31.622618 21703 tablet_bootstrap.cc:492] T af330dcd4767423ea4c179be7f88d0c4 P 8bf6e26e98b14566899578a990b27d47: No bootstrap required, opened a new log
I20250109 08:57:31.622632 21702 ts_tablet_manager.cc:1397] T df9715570a6844d9a2686307db16eff1 P a2b1a692435241ee988e0bd1661cd4c3: Time spent bootstrapping tablet: real 0.101s user 0.017s sys 0.029s
I20250109 08:57:31.623097 21703 ts_tablet_manager.cc:1397] T af330dcd4767423ea4c179be7f88d0c4 P 8bf6e26e98b14566899578a990b27d47: Time spent bootstrapping tablet: real 0.091s user 0.013s sys 0.022s
I20250109 08:57:31.624722 21661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af273bab41cc4a9da780147eebfd52ec" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e"
I20250109 08:57:31.626475 21703 raft_consensus.cc:357] T af330dcd4767423ea4c179be7f88d0c4 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.627136 21703 raft_consensus.cc:383] T af330dcd4767423ea4c179be7f88d0c4 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:31.627377 21703 raft_consensus.cc:738] T af330dcd4767423ea4c179be7f88d0c4 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bf6e26e98b14566899578a990b27d47, State: Initialized, Role: FOLLOWER
I20250109 08:57:31.627954 21703 consensus_queue.cc:260] T af330dcd4767423ea4c179be7f88d0c4 P 8bf6e26e98b14566899578a990b27d47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.628548 21711 raft_consensus.cc:491] T e8ff570914d74df8b87296010e9b864f P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20250109 08:57:31.628979 21478 leader_election.cc:343] T 126a163a89c448be89ccb9490e9f0168 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:31.629418 21478 leader_election.cc:304] T 126a163a89c448be89ccb9490e9f0168 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8bf6e26e98b14566899578a990b27d47; no voters: 787ddcc824c2490a808fc5a53452dc5e, a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:57:31.630342 21552 leader_election.cc:304] T af273bab41cc4a9da780147eebfd52ec P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2b1a692435241ee988e0bd1661cd4c3; no voters: 787ddcc824c2490a808fc5a53452dc5e, 8bf6e26e98b14566899578a990b27d47
I20250109 08:57:31.628855 21411 catalog_manager.cc:5526] T 0f3bde1a1ccd4c359a2e80feca968453 P 787ddcc824c2490a808fc5a53452dc5e reported cstate change: term changed from 0 to 1, leader changed from <none> to 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195). New cstate: current_term: 1 leader_uuid: "787ddcc824c2490a808fc5a53452dc5e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: HEALTHY } } }
I20250109 08:57:31.633462 21731 raft_consensus.cc:2743] T 126a163a89c448be89ccb9490e9f0168 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:57:31.628988 21711 raft_consensus.cc:513] T e8ff570914d74df8b87296010e9b864f P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.637168 21711 leader_election.cc:290] T e8ff570914d74df8b87296010e9b864f P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:31.633595 21662 raft_consensus.cc:2463] T af273bab41cc4a9da780147eebfd52ec P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8bf6e26e98b14566899578a990b27d47 in term 1.
I20250109 08:57:31.632503 21819 raft_consensus.cc:2743] T af273bab41cc4a9da780147eebfd52ec P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20250109 08:57:31.638447 21477 leader_election.cc:304] T af273bab41cc4a9da780147eebfd52ec P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, 8bf6e26e98b14566899578a990b27d47; no voters: a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:57:31.639745 21711 raft_consensus.cc:2798] T af273bab41cc4a9da780147eebfd52ec P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:57:31.639542 21589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8ff570914d74df8b87296010e9b864f" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3" is_pre_election: true
I20250109 08:57:31.640561 21711 raft_consensus.cc:695] T af273bab41cc4a9da780147eebfd52ec P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Becoming Leader. State: Replica: 8bf6e26e98b14566899578a990b27d47, State: Running, Role: LEADER
I20250109 08:57:31.640619 21589 raft_consensus.cc:2463] T e8ff570914d74df8b87296010e9b864f P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8bf6e26e98b14566899578a990b27d47 in term 0.
I20250109 08:57:31.641135 21704 tablet_bootstrap.cc:654] T af330dcd4767423ea4c179be7f88d0c4 P 787ddcc824c2490a808fc5a53452dc5e: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:31.641691 21478 leader_election.cc:304] T e8ff570914d74df8b87296010e9b864f P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3; no voters:
I20250109 08:57:31.642681 21731 raft_consensus.cc:2798] T e8ff570914d74df8b87296010e9b864f P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:57:31.643013 21731 raft_consensus.cc:491] T e8ff570914d74df8b87296010e9b864f P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:57:31.643460 21731 raft_consensus.cc:3054] T e8ff570914d74df8b87296010e9b864f P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:31.643035 21702 raft_consensus.cc:357] T df9715570a6844d9a2686307db16eff1 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.644510 21702 raft_consensus.cc:383] T df9715570a6844d9a2686307db16eff1 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:31.644788 21702 raft_consensus.cc:738] T df9715570a6844d9a2686307db16eff1 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2b1a692435241ee988e0bd1661cd4c3, State: Initialized, Role: FOLLOWER
I20250109 08:57:31.645344 21702 consensus_queue.cc:260] T df9715570a6844d9a2686307db16eff1 P a2b1a692435241ee988e0bd1661cd4c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.643994 21661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8ff570914d74df8b87296010e9b864f" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e" is_pre_election: true
I20250109 08:57:31.646908 21661 raft_consensus.cc:2463] T e8ff570914d74df8b87296010e9b864f P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8bf6e26e98b14566899578a990b27d47 in term 0.
I20250109 08:57:31.649494 21702 ts_tablet_manager.cc:1428] T df9715570a6844d9a2686307db16eff1 P a2b1a692435241ee988e0bd1661cd4c3: Time spent starting tablet: real 0.026s user 0.004s sys 0.001s
I20250109 08:57:31.650231 21842 consensus_queue.cc:1035] T 0cc4fcda876242c396b804edebebb01f P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:57:31.652897 21703 ts_tablet_manager.cc:1428] T af330dcd4767423ea4c179be7f88d0c4 P 8bf6e26e98b14566899578a990b27d47: Time spent starting tablet: real 0.030s user 0.005s sys 0.001s
I20250109 08:57:31.653081 21731 raft_consensus.cc:513] T e8ff570914d74df8b87296010e9b864f P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.653825 21703 tablet_bootstrap.cc:492] T e03846e461974d0d8944c344e715b86a P 8bf6e26e98b14566899578a990b27d47: Bootstrap starting.
I20250109 08:57:31.658438 21731 leader_election.cc:290] T e8ff570914d74df8b87296010e9b864f P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 election: Requested vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:31.659718 21732 consensus_queue.cc:1035] T 75edb5d04fa14201ad711c1d95c81f64 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:57:31.660776 21587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8ff570914d74df8b87296010e9b864f" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3"
I20250109 08:57:31.661342 21587 raft_consensus.cc:3054] T e8ff570914d74df8b87296010e9b864f P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:31.659739 21408 catalog_manager.cc:5526] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P 787ddcc824c2490a808fc5a53452dc5e reported cstate change: term changed from 0 to 1, leader changed from <none> to 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195). New cstate: current_term: 1 leader_uuid: "787ddcc824c2490a808fc5a53452dc5e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: HEALTHY } } }
I20250109 08:57:31.665952 21702 tablet_bootstrap.cc:492] T 90978f185a374feba3bc19b54f9d97f5 P a2b1a692435241ee988e0bd1661cd4c3: Bootstrap starting.
I20250109 08:57:31.668741 21587 raft_consensus.cc:2463] T e8ff570914d74df8b87296010e9b864f P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8bf6e26e98b14566899578a990b27d47 in term 1.
I20250109 08:57:31.669843 21478 leader_election.cc:304] T e8ff570914d74df8b87296010e9b864f P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3; no voters:
I20250109 08:57:31.671160 21731 raft_consensus.cc:2798] T e8ff570914d74df8b87296010e9b864f P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:57:31.671890 21731 raft_consensus.cc:695] T e8ff570914d74df8b87296010e9b864f P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Becoming Leader. State: Replica: 8bf6e26e98b14566899578a990b27d47, State: Running, Role: LEADER
I20250109 08:57:31.673166 21702 tablet_bootstrap.cc:654] T 90978f185a374feba3bc19b54f9d97f5 P a2b1a692435241ee988e0bd1661cd4c3: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:31.672684 21731 consensus_queue.cc:237] T e8ff570914d74df8b87296010e9b864f P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.661229 21661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8ff570914d74df8b87296010e9b864f" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e"
I20250109 08:57:31.675109 21661 raft_consensus.cc:3054] T e8ff570914d74df8b87296010e9b864f P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:31.641552 21711 consensus_queue.cc:237] T af273bab41cc4a9da780147eebfd52ec P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.684566 21732 consensus_queue.cc:1035] T 75edb5d04fa14201ad711c1d95c81f64 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250109 08:57:31.696090 21661 raft_consensus.cc:2463] T e8ff570914d74df8b87296010e9b864f P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8bf6e26e98b14566899578a990b27d47 in term 1.
I20250109 08:57:31.716809 21742 consensus_queue.cc:1035] T 0cc4fcda876242c396b804edebebb01f P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.022s
I20250109 08:57:31.738855 21703 tablet_bootstrap.cc:654] T e03846e461974d0d8944c344e715b86a P 8bf6e26e98b14566899578a990b27d47: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:31.739805 21704 tablet_bootstrap.cc:492] T af330dcd4767423ea4c179be7f88d0c4 P 787ddcc824c2490a808fc5a53452dc5e: No bootstrap required, opened a new log
I20250109 08:57:31.740315 21704 ts_tablet_manager.cc:1397] T af330dcd4767423ea4c179be7f88d0c4 P 787ddcc824c2490a808fc5a53452dc5e: Time spent bootstrapping tablet: real 0.169s user 0.031s sys 0.030s
I20250109 08:57:31.719002 21702 tablet_bootstrap.cc:492] T 90978f185a374feba3bc19b54f9d97f5 P a2b1a692435241ee988e0bd1661cd4c3: No bootstrap required, opened a new log
I20250109 08:57:31.741068 21412 catalog_manager.cc:5526] T af273bab41cc4a9da780147eebfd52ec P 8bf6e26e98b14566899578a990b27d47 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8bf6e26e98b14566899578a990b27d47 (127.19.226.193). New cstate: current_term: 1 leader_uuid: "8bf6e26e98b14566899578a990b27d47" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: UNKNOWN } } }
I20250109 08:57:31.741981 21702 ts_tablet_manager.cc:1397] T 90978f185a374feba3bc19b54f9d97f5 P a2b1a692435241ee988e0bd1661cd4c3: Time spent bootstrapping tablet: real 0.076s user 0.009s sys 0.004s
I20250109 08:57:31.743320 21704 raft_consensus.cc:357] T af330dcd4767423ea4c179be7f88d0c4 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.744091 21704 raft_consensus.cc:383] T af330dcd4767423ea4c179be7f88d0c4 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:31.744413 21704 raft_consensus.cc:738] T af330dcd4767423ea4c179be7f88d0c4 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Initialized, Role: FOLLOWER
I20250109 08:57:31.744944 21702 raft_consensus.cc:357] T 90978f185a374feba3bc19b54f9d97f5 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.745667 21702 raft_consensus.cc:383] T 90978f185a374feba3bc19b54f9d97f5 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:31.747030 21702 raft_consensus.cc:738] T 90978f185a374feba3bc19b54f9d97f5 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2b1a692435241ee988e0bd1661cd4c3, State: Initialized, Role: FOLLOWER
I20250109 08:57:31.747725 21702 consensus_queue.cc:260] T 90978f185a374feba3bc19b54f9d97f5 P a2b1a692435241ee988e0bd1661cd4c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
W20250109 08:57:31.749370 21626 consensus_peers.cc:487] T 90a4a2381cbe40f0b204be325ee9fea1 P 787ddcc824c2490a808fc5a53452dc5e -> Peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Couldn't send request to peer 8bf6e26e98b14566899578a990b27d47. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:57:31.749853 21702 ts_tablet_manager.cc:1428] T 90978f185a374feba3bc19b54f9d97f5 P a2b1a692435241ee988e0bd1661cd4c3: Time spent starting tablet: real 0.008s user 0.002s sys 0.003s
I20250109 08:57:31.745178 21704 consensus_queue.cc:260] T af330dcd4767423ea4c179be7f88d0c4 P 787ddcc824c2490a808fc5a53452dc5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.710757 21901 raft_consensus.cc:491] T df9715570a6844d9a2686307db16eff1 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:31.750551 21901 raft_consensus.cc:513] T df9715570a6844d9a2686307db16eff1 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.752151 21704 ts_tablet_manager.cc:1428] T af330dcd4767423ea4c179be7f88d0c4 P 787ddcc824c2490a808fc5a53452dc5e: Time spent starting tablet: real 0.011s user 0.005s sys 0.000s
I20250109 08:57:31.753018 21514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df9715570a6844d9a2686307db16eff1" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47" is_pre_election: true
I20250109 08:57:31.753333 21702 tablet_bootstrap.cc:492] T e03846e461974d0d8944c344e715b86a P a2b1a692435241ee988e0bd1661cd4c3: Bootstrap starting.
I20250109 08:57:31.753635 21514 raft_consensus.cc:2463] T df9715570a6844d9a2686307db16eff1 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 0.
I20250109 08:57:31.754102 21704 tablet_bootstrap.cc:492] T 239ca74a4c0a455a9db2652d7debb989 P 787ddcc824c2490a808fc5a53452dc5e: Bootstrap starting.
I20250109 08:57:31.760419 21626 leader_election.cc:304] T df9715570a6844d9a2686307db16eff1 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, 8bf6e26e98b14566899578a990b27d47; no voters:
I20250109 08:57:31.761314 21732 raft_consensus.cc:2798] T df9715570a6844d9a2686307db16eff1 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:57:31.761606 21732 raft_consensus.cc:491] T df9715570a6844d9a2686307db16eff1 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:57:31.761934 21732 raft_consensus.cc:3054] T df9715570a6844d9a2686307db16eff1 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:31.766031 21412 catalog_manager.cc:5526] T e8ff570914d74df8b87296010e9b864f P 8bf6e26e98b14566899578a990b27d47 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8bf6e26e98b14566899578a990b27d47 (127.19.226.193). New cstate: current_term: 1 leader_uuid: "8bf6e26e98b14566899578a990b27d47" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: UNKNOWN } } }
I20250109 08:57:31.769816 21732 raft_consensus.cc:513] T df9715570a6844d9a2686307db16eff1 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.772815 21732 leader_election.cc:290] T df9715570a6844d9a2686307db16eff1 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 election: Requested vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
I20250109 08:57:31.776413 21587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df9715570a6844d9a2686307db16eff1" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3" is_pre_election: true
I20250109 08:57:31.776650 21589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df9715570a6844d9a2686307db16eff1" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3"
I20250109 08:57:31.777069 21587 raft_consensus.cc:2463] T df9715570a6844d9a2686307db16eff1 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 0.
I20250109 08:57:31.775084 21901 leader_election.cc:290] T df9715570a6844d9a2686307db16eff1 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
I20250109 08:57:31.780918 21514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df9715570a6844d9a2686307db16eff1" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47"
I20250109 08:57:31.781713 21514 raft_consensus.cc:3054] T df9715570a6844d9a2686307db16eff1 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:31.784531 21702 tablet_bootstrap.cc:654] T e03846e461974d0d8944c344e715b86a P a2b1a692435241ee988e0bd1661cd4c3: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:31.790086 21514 raft_consensus.cc:2463] T df9715570a6844d9a2686307db16eff1 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 1.
I20250109 08:57:31.791569 21626 leader_election.cc:304] T df9715570a6844d9a2686307db16eff1 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, 8bf6e26e98b14566899578a990b27d47; no voters: a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:57:31.794684 21703 tablet_bootstrap.cc:492] T e03846e461974d0d8944c344e715b86a P 8bf6e26e98b14566899578a990b27d47: No bootstrap required, opened a new log
I20250109 08:57:31.794973 21704 tablet_bootstrap.cc:654] T 239ca74a4c0a455a9db2652d7debb989 P 787ddcc824c2490a808fc5a53452dc5e: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:31.795130 21703 ts_tablet_manager.cc:1397] T e03846e461974d0d8944c344e715b86a P 8bf6e26e98b14566899578a990b27d47: Time spent bootstrapping tablet: real 0.141s user 0.019s sys 0.002s
I20250109 08:57:31.797608 21703 raft_consensus.cc:357] T e03846e461974d0d8944c344e715b86a P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.798290 21703 raft_consensus.cc:383] T e03846e461974d0d8944c344e715b86a P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:31.798556 21703 raft_consensus.cc:738] T e03846e461974d0d8944c344e715b86a P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bf6e26e98b14566899578a990b27d47, State: Initialized, Role: FOLLOWER
I20250109 08:57:31.799125 21703 consensus_queue.cc:260] T e03846e461974d0d8944c344e715b86a P 8bf6e26e98b14566899578a990b27d47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.801226 21703 ts_tablet_manager.cc:1428] T e03846e461974d0d8944c344e715b86a P 8bf6e26e98b14566899578a990b27d47: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20250109 08:57:31.800231 21901 raft_consensus.cc:2798] T df9715570a6844d9a2686307db16eff1 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:57:31.802429 21901 raft_consensus.cc:695] T df9715570a6844d9a2686307db16eff1 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Becoming Leader. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Running, Role: LEADER
I20250109 08:57:31.803238 21702 tablet_bootstrap.cc:492] T e03846e461974d0d8944c344e715b86a P a2b1a692435241ee988e0bd1661cd4c3: No bootstrap required, opened a new log
I20250109 08:57:31.803797 21702 ts_tablet_manager.cc:1397] T e03846e461974d0d8944c344e715b86a P a2b1a692435241ee988e0bd1661cd4c3: Time spent bootstrapping tablet: real 0.051s user 0.002s sys 0.010s
I20250109 08:57:31.803237 21901 consensus_queue.cc:237] T df9715570a6844d9a2686307db16eff1 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.806351 21702 raft_consensus.cc:357] T e03846e461974d0d8944c344e715b86a P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.807017 21702 raft_consensus.cc:383] T e03846e461974d0d8944c344e715b86a P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:31.807260 21702 raft_consensus.cc:738] T e03846e461974d0d8944c344e715b86a P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2b1a692435241ee988e0bd1661cd4c3, State: Initialized, Role: FOLLOWER
I20250109 08:57:31.807484 21732 consensus_queue.cc:1035] T 90a4a2381cbe40f0b204be325ee9fea1 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:57:31.807884 21702 consensus_queue.cc:260] T e03846e461974d0d8944c344e715b86a P a2b1a692435241ee988e0bd1661cd4c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.810063 21702 ts_tablet_manager.cc:1428] T e03846e461974d0d8944c344e715b86a P a2b1a692435241ee988e0bd1661cd4c3: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20250109 08:57:31.810940 21702 tablet_bootstrap.cc:492] T b00671c87aac405a81c6543f4defd722 P a2b1a692435241ee988e0bd1661cd4c3: Bootstrap starting.
I20250109 08:57:31.818297 21703 tablet_bootstrap.cc:492] T 239ca74a4c0a455a9db2652d7debb989 P 8bf6e26e98b14566899578a990b27d47: Bootstrap starting.
I20250109 08:57:31.818152 21907 consensus_queue.cc:1035] T d355d166de534fb4a50c9f48f2df7469 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:57:31.828811 21704 tablet_bootstrap.cc:492] T 239ca74a4c0a455a9db2652d7debb989 P 787ddcc824c2490a808fc5a53452dc5e: No bootstrap required, opened a new log
I20250109 08:57:31.829309 21704 ts_tablet_manager.cc:1397] T 239ca74a4c0a455a9db2652d7debb989 P 787ddcc824c2490a808fc5a53452dc5e: Time spent bootstrapping tablet: real 0.075s user 0.009s sys 0.011s
I20250109 08:57:31.824349 21703 tablet_bootstrap.cc:654] T 239ca74a4c0a455a9db2652d7debb989 P 8bf6e26e98b14566899578a990b27d47: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:31.831691 21704 raft_consensus.cc:357] T 239ca74a4c0a455a9db2652d7debb989 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.837005 21704 raft_consensus.cc:383] T 239ca74a4c0a455a9db2652d7debb989 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:31.837316 21704 raft_consensus.cc:738] T 239ca74a4c0a455a9db2652d7debb989 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Initialized, Role: FOLLOWER
I20250109 08:57:31.838666 21702 tablet_bootstrap.cc:654] T b00671c87aac405a81c6543f4defd722 P a2b1a692435241ee988e0bd1661cd4c3: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:31.851480 21819 raft_consensus.cc:491] T 90978f185a374feba3bc19b54f9d97f5 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:31.851984 21819 raft_consensus.cc:513] T 90978f185a374feba3bc19b54f9d97f5 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.853825 21819 leader_election.cc:290] T 90978f185a374feba3bc19b54f9d97f5 P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:31.855427 21663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90978f185a374feba3bc19b54f9d97f5" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e" is_pre_election: true
I20250109 08:57:31.855110 21513 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90978f185a374feba3bc19b54f9d97f5" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47" is_pre_election: true
I20250109 08:57:31.856237 21513 raft_consensus.cc:2463] T 90978f185a374feba3bc19b54f9d97f5 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2b1a692435241ee988e0bd1661cd4c3 in term 0.
I20250109 08:57:31.857712 21552 leader_election.cc:304] T 90978f185a374feba3bc19b54f9d97f5 P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3; no voters:
W20250109 08:57:31.858395 21552 leader_election.cc:343] T 90978f185a374feba3bc19b54f9d97f5 P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:31.859018 21709 raft_consensus.cc:2798] T 90978f185a374feba3bc19b54f9d97f5 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:57:31.837942 21412 catalog_manager.cc:5526] T df9715570a6844d9a2686307db16eff1 P 787ddcc824c2490a808fc5a53452dc5e reported cstate change: term changed from 0 to 1, leader changed from <none> to 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195). New cstate: current_term: 1 leader_uuid: "787ddcc824c2490a808fc5a53452dc5e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: HEALTHY } } }
I20250109 08:57:31.859400 21709 raft_consensus.cc:491] T 90978f185a374feba3bc19b54f9d97f5 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:57:31.838798 21704 consensus_queue.cc:260] T 239ca74a4c0a455a9db2652d7debb989 P 787ddcc824c2490a808fc5a53452dc5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.843434 21742 raft_consensus.cc:491] T c99aacce09d44f889be5480854a22760 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:31.859896 21709 raft_consensus.cc:3054] T 90978f185a374feba3bc19b54f9d97f5 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:31.860193 21742 raft_consensus.cc:513] T c99aacce09d44f889be5480854a22760 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.861486 21704 ts_tablet_manager.cc:1428] T 239ca74a4c0a455a9db2652d7debb989 P 787ddcc824c2490a808fc5a53452dc5e: Time spent starting tablet: real 0.032s user 0.006s sys 0.000s
I20250109 08:57:31.862098 21742 leader_election.cc:290] T c99aacce09d44f889be5480854a22760 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:31.862294 21704 tablet_bootstrap.cc:492] T c18c1d46a6ca47598d7b161ed70a4e7a P 787ddcc824c2490a808fc5a53452dc5e: Bootstrap starting.
I20250109 08:57:31.845688 21703 tablet_bootstrap.cc:492] T 239ca74a4c0a455a9db2652d7debb989 P 8bf6e26e98b14566899578a990b27d47: No bootstrap required, opened a new log
I20250109 08:57:31.864642 21703 ts_tablet_manager.cc:1397] T 239ca74a4c0a455a9db2652d7debb989 P 8bf6e26e98b14566899578a990b27d47: Time spent bootstrapping tablet: real 0.047s user 0.012s sys 0.000s
I20250109 08:57:31.868048 21663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c99aacce09d44f889be5480854a22760" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e" is_pre_election: true
I20250109 08:57:31.869128 21758 consensus_queue.cc:1035] T d355d166de534fb4a50c9f48f2df7469 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20250109 08:57:31.871392 21477 leader_election.cc:343] T c99aacce09d44f889be5480854a22760 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:31.864110 21589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c99aacce09d44f889be5480854a22760" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3" is_pre_election: true
W20250109 08:57:31.873215 21478 leader_election.cc:343] T c99aacce09d44f889be5480854a22760 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:31.874701 21478 leader_election.cc:304] T c99aacce09d44f889be5480854a22760 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8bf6e26e98b14566899578a990b27d47; no voters: 787ddcc824c2490a808fc5a53452dc5e, a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:57:31.875036 21709 raft_consensus.cc:513] T 90978f185a374feba3bc19b54f9d97f5 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.876307 21742 raft_consensus.cc:2743] T c99aacce09d44f889be5480854a22760 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:57:31.874666 21702 tablet_bootstrap.cc:492] T b00671c87aac405a81c6543f4defd722 P a2b1a692435241ee988e0bd1661cd4c3: No bootstrap required, opened a new log
I20250109 08:57:31.880239 21702 ts_tablet_manager.cc:1397] T b00671c87aac405a81c6543f4defd722 P a2b1a692435241ee988e0bd1661cd4c3: Time spent bootstrapping tablet: real 0.069s user 0.009s sys 0.008s
I20250109 08:57:31.881685 21514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90978f185a374feba3bc19b54f9d97f5" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47"
I20250109 08:57:31.882346 21514 raft_consensus.cc:3054] T 90978f185a374feba3bc19b54f9d97f5 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:31.884141 21702 raft_consensus.cc:357] T b00671c87aac405a81c6543f4defd722 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.884832 21702 raft_consensus.cc:383] T b00671c87aac405a81c6543f4defd722 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:31.885097 21702 raft_consensus.cc:738] T b00671c87aac405a81c6543f4defd722 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2b1a692435241ee988e0bd1661cd4c3, State: Initialized, Role: FOLLOWER
I20250109 08:57:31.885825 21702 consensus_queue.cc:260] T b00671c87aac405a81c6543f4defd722 P a2b1a692435241ee988e0bd1661cd4c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.887830 21703 raft_consensus.cc:357] T 239ca74a4c0a455a9db2652d7debb989 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.890413 21703 raft_consensus.cc:383] T 239ca74a4c0a455a9db2652d7debb989 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:31.890677 21703 raft_consensus.cc:738] T 239ca74a4c0a455a9db2652d7debb989 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bf6e26e98b14566899578a990b27d47, State: Initialized, Role: FOLLOWER
I20250109 08:57:31.891067 21663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90978f185a374feba3bc19b54f9d97f5" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e"
I20250109 08:57:31.891290 21703 consensus_queue.cc:260] T 239ca74a4c0a455a9db2652d7debb989 P 8bf6e26e98b14566899578a990b27d47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
W20250109 08:57:31.892643 21552 leader_election.cc:343] T 90978f185a374feba3bc19b54f9d97f5 P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:31.893412 21703 ts_tablet_manager.cc:1428] T 239ca74a4c0a455a9db2652d7debb989 P 8bf6e26e98b14566899578a990b27d47: Time spent starting tablet: real 0.028s user 0.004s sys 0.002s
I20250109 08:57:31.893702 21514 raft_consensus.cc:2463] T 90978f185a374feba3bc19b54f9d97f5 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2b1a692435241ee988e0bd1661cd4c3 in term 1.
I20250109 08:57:31.894296 21703 tablet_bootstrap.cc:492] T 8c0459d2eee64f0a83bd05ad5974f506 P 8bf6e26e98b14566899578a990b27d47: Bootstrap starting.
I20250109 08:57:31.895013 21552 leader_election.cc:304] T 90978f185a374feba3bc19b54f9d97f5 P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3; no voters: 787ddcc824c2490a808fc5a53452dc5e
I20250109 08:57:31.895738 21702 ts_tablet_manager.cc:1428] T b00671c87aac405a81c6543f4defd722 P a2b1a692435241ee988e0bd1661cd4c3: Time spent starting tablet: real 0.015s user 0.006s sys 0.000s
I20250109 08:57:31.896600 21702 tablet_bootstrap.cc:492] T 239ca74a4c0a455a9db2652d7debb989 P a2b1a692435241ee988e0bd1661cd4c3: Bootstrap starting.
I20250109 08:57:31.900319 21819 raft_consensus.cc:2798] T 90978f185a374feba3bc19b54f9d97f5 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:57:31.900866 21819 raft_consensus.cc:695] T 90978f185a374feba3bc19b54f9d97f5 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 LEADER]: Becoming Leader. State: Replica: a2b1a692435241ee988e0bd1661cd4c3, State: Running, Role: LEADER
I20250109 08:57:31.901640 21819 consensus_queue.cc:237] T 90978f185a374feba3bc19b54f9d97f5 P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.902333 21702 tablet_bootstrap.cc:654] T 239ca74a4c0a455a9db2652d7debb989 P a2b1a692435241ee988e0bd1661cd4c3: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:31.895045 21709 leader_election.cc:290] T 90978f185a374feba3bc19b54f9d97f5 P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 election: Requested vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:31.912999 21703 tablet_bootstrap.cc:654] T 8c0459d2eee64f0a83bd05ad5974f506 P 8bf6e26e98b14566899578a990b27d47: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:31.925490 21704 tablet_bootstrap.cc:654] T c18c1d46a6ca47598d7b161ed70a4e7a P 787ddcc824c2490a808fc5a53452dc5e: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:31.929873 21702 tablet_bootstrap.cc:492] T 239ca74a4c0a455a9db2652d7debb989 P a2b1a692435241ee988e0bd1661cd4c3: No bootstrap required, opened a new log
I20250109 08:57:31.930369 21702 ts_tablet_manager.cc:1397] T 239ca74a4c0a455a9db2652d7debb989 P a2b1a692435241ee988e0bd1661cd4c3: Time spent bootstrapping tablet: real 0.034s user 0.019s sys 0.007s
W20250109 08:57:31.932538 21627 consensus_peers.cc:487] T 0f3bde1a1ccd4c359a2e80feca968453 P 787ddcc824c2490a808fc5a53452dc5e -> Peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Couldn't send request to peer a2b1a692435241ee988e0bd1661cd4c3. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:57:31.933213 21703 tablet_bootstrap.cc:492] T 8c0459d2eee64f0a83bd05ad5974f506 P 8bf6e26e98b14566899578a990b27d47: No bootstrap required, opened a new log
I20250109 08:57:31.932873 21702 raft_consensus.cc:357] T 239ca74a4c0a455a9db2652d7debb989 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.933681 21702 raft_consensus.cc:383] T 239ca74a4c0a455a9db2652d7debb989 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:31.933837 21703 ts_tablet_manager.cc:1397] T 8c0459d2eee64f0a83bd05ad5974f506 P 8bf6e26e98b14566899578a990b27d47: Time spent bootstrapping tablet: real 0.040s user 0.011s sys 0.003s
I20250109 08:57:31.933979 21702 raft_consensus.cc:738] T 239ca74a4c0a455a9db2652d7debb989 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2b1a692435241ee988e0bd1661cd4c3, State: Initialized, Role: FOLLOWER
I20250109 08:57:31.934720 21702 consensus_queue.cc:260] T 239ca74a4c0a455a9db2652d7debb989 P a2b1a692435241ee988e0bd1661cd4c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.937008 21702 ts_tablet_manager.cc:1428] T 239ca74a4c0a455a9db2652d7debb989 P a2b1a692435241ee988e0bd1661cd4c3: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20250109 08:57:31.944098 21702 tablet_bootstrap.cc:492] T 0f3bde1a1ccd4c359a2e80feca968453 P a2b1a692435241ee988e0bd1661cd4c3: Bootstrap starting.
I20250109 08:57:31.948774 21411 catalog_manager.cc:5526] T 90978f185a374feba3bc19b54f9d97f5 P a2b1a692435241ee988e0bd1661cd4c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194). New cstate: current_term: 1 leader_uuid: "a2b1a692435241ee988e0bd1661cd4c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: UNKNOWN } } }
I20250109 08:57:31.950520 21703 raft_consensus.cc:357] T 8c0459d2eee64f0a83bd05ad5974f506 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.951155 21702 tablet_bootstrap.cc:654] T 0f3bde1a1ccd4c359a2e80feca968453 P a2b1a692435241ee988e0bd1661cd4c3: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:31.951280 21703 raft_consensus.cc:383] T 8c0459d2eee64f0a83bd05ad5974f506 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:31.951707 21703 raft_consensus.cc:738] T 8c0459d2eee64f0a83bd05ad5974f506 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bf6e26e98b14566899578a990b27d47, State: Initialized, Role: FOLLOWER
I20250109 08:57:31.952314 21703 consensus_queue.cc:260] T 8c0459d2eee64f0a83bd05ad5974f506 P 8bf6e26e98b14566899578a990b27d47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.955297 21703 ts_tablet_manager.cc:1428] T 8c0459d2eee64f0a83bd05ad5974f506 P 8bf6e26e98b14566899578a990b27d47: Time spent starting tablet: real 0.021s user 0.006s sys 0.000s
I20250109 08:57:31.956910 21704 tablet_bootstrap.cc:492] T c18c1d46a6ca47598d7b161ed70a4e7a P 787ddcc824c2490a808fc5a53452dc5e: No bootstrap required, opened a new log
I20250109 08:57:31.957345 21704 ts_tablet_manager.cc:1397] T c18c1d46a6ca47598d7b161ed70a4e7a P 787ddcc824c2490a808fc5a53452dc5e: Time spent bootstrapping tablet: real 0.095s user 0.024s sys 0.008s
I20250109 08:57:31.957549 21732 raft_consensus.cc:491] T 239ca74a4c0a455a9db2652d7debb989 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:31.958066 21732 raft_consensus.cc:513] T 239ca74a4c0a455a9db2652d7debb989 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.960201 21732 leader_election.cc:290] T 239ca74a4c0a455a9db2652d7debb989 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
I20250109 08:57:31.960350 21704 raft_consensus.cc:357] T c18c1d46a6ca47598d7b161ed70a4e7a P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.961071 21704 raft_consensus.cc:383] T c18c1d46a6ca47598d7b161ed70a4e7a P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:31.961325 21704 raft_consensus.cc:738] T c18c1d46a6ca47598d7b161ed70a4e7a P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Initialized, Role: FOLLOWER
I20250109 08:57:31.961982 21704 consensus_queue.cc:260] T c18c1d46a6ca47598d7b161ed70a4e7a P 787ddcc824c2490a808fc5a53452dc5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.962708 21703 tablet_bootstrap.cc:492] T f5f85ad7c7c94acfaf9d550d55fce48c P 8bf6e26e98b14566899578a990b27d47: Bootstrap starting.
I20250109 08:57:31.964905 21514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "239ca74a4c0a455a9db2652d7debb989" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47" is_pre_election: true
I20250109 08:57:31.965569 21514 raft_consensus.cc:2463] T 239ca74a4c0a455a9db2652d7debb989 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 0.
I20250109 08:57:31.967182 21626 leader_election.cc:304] T 239ca74a4c0a455a9db2652d7debb989 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, 8bf6e26e98b14566899578a990b27d47; no voters:
I20250109 08:57:31.967964 21901 raft_consensus.cc:2798] T 239ca74a4c0a455a9db2652d7debb989 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:57:31.968303 21901 raft_consensus.cc:491] T 239ca74a4c0a455a9db2652d7debb989 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:57:31.968643 21901 raft_consensus.cc:3054] T 239ca74a4c0a455a9db2652d7debb989 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:31.970032 21587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "239ca74a4c0a455a9db2652d7debb989" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3" is_pre_election: true
I20250109 08:57:31.970716 21587 raft_consensus.cc:2463] T 239ca74a4c0a455a9db2652d7debb989 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 0.
I20250109 08:57:31.973096 21704 ts_tablet_manager.cc:1428] T c18c1d46a6ca47598d7b161ed70a4e7a P 787ddcc824c2490a808fc5a53452dc5e: Time spent starting tablet: real 0.015s user 0.005s sys 0.003s
I20250109 08:57:31.980584 21901 raft_consensus.cc:513] T 239ca74a4c0a455a9db2652d7debb989 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.981626 21732 consensus_queue.cc:1035] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:57:31.982755 21703 tablet_bootstrap.cc:654] T f5f85ad7c7c94acfaf9d550d55fce48c P 8bf6e26e98b14566899578a990b27d47: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:31.986766 21704 tablet_bootstrap.cc:492] T 90978f185a374feba3bc19b54f9d97f5 P 787ddcc824c2490a808fc5a53452dc5e: Bootstrap starting.
I20250109 08:57:31.989420 21702 tablet_bootstrap.cc:492] T 0f3bde1a1ccd4c359a2e80feca968453 P a2b1a692435241ee988e0bd1661cd4c3: No bootstrap required, opened a new log
I20250109 08:57:31.989964 21702 ts_tablet_manager.cc:1397] T 0f3bde1a1ccd4c359a2e80feca968453 P a2b1a692435241ee988e0bd1661cd4c3: Time spent bootstrapping tablet: real 0.046s user 0.019s sys 0.018s
I20250109 08:57:31.994160 21704 tablet_bootstrap.cc:654] T 90978f185a374feba3bc19b54f9d97f5 P 787ddcc824c2490a808fc5a53452dc5e: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:31.993520 21702 raft_consensus.cc:357] T 0f3bde1a1ccd4c359a2e80feca968453 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.994921 21702 raft_consensus.cc:383] T 0f3bde1a1ccd4c359a2e80feca968453 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:31.995246 21702 raft_consensus.cc:738] T 0f3bde1a1ccd4c359a2e80feca968453 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2b1a692435241ee988e0bd1661cd4c3, State: Initialized, Role: FOLLOWER
I20250109 08:57:31.995960 21702 consensus_queue.cc:260] T 0f3bde1a1ccd4c359a2e80feca968453 P a2b1a692435241ee988e0bd1661cd4c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:31.998018 21702 ts_tablet_manager.cc:1428] T 0f3bde1a1ccd4c359a2e80feca968453 P a2b1a692435241ee988e0bd1661cd4c3: Time spent starting tablet: real 0.008s user 0.004s sys 0.000s
I20250109 08:57:31.998829 21702 tablet_bootstrap.cc:492] T c99aacce09d44f889be5480854a22760 P a2b1a692435241ee988e0bd1661cd4c3: Bootstrap starting.
I20250109 08:57:32.004132 21742 raft_consensus.cc:491] T af330dcd4767423ea4c179be7f88d0c4 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:32.004617 21742 raft_consensus.cc:513] T af330dcd4767423ea4c179be7f88d0c4 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:32.007154 21710 consensus_queue.cc:1035] T 0f3bde1a1ccd4c359a2e80feca968453 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:57:32.008427 21702 tablet_bootstrap.cc:654] T c99aacce09d44f889be5480854a22760 P a2b1a692435241ee988e0bd1661cd4c3: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:32.009886 21587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af330dcd4767423ea4c179be7f88d0c4" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3" is_pre_election: true
I20250109 08:57:32.010205 21704 tablet_bootstrap.cc:492] T 90978f185a374feba3bc19b54f9d97f5 P 787ddcc824c2490a808fc5a53452dc5e: No bootstrap required, opened a new log
I20250109 08:57:32.010680 21704 ts_tablet_manager.cc:1397] T 90978f185a374feba3bc19b54f9d97f5 P 787ddcc824c2490a808fc5a53452dc5e: Time spent bootstrapping tablet: real 0.034s user 0.016s sys 0.001s
W20250109 08:57:32.012342 21478 leader_election.cc:343] T af330dcd4767423ea4c179be7f88d0c4 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:32.013196 21663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af330dcd4767423ea4c179be7f88d0c4" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e" is_pre_election: true
I20250109 08:57:32.013077 21704 raft_consensus.cc:357] T 90978f185a374feba3bc19b54f9d97f5 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:32.013793 21704 raft_consensus.cc:383] T 90978f185a374feba3bc19b54f9d97f5 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:32.008518 21710 consensus_queue.cc:1035] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20250109 08:57:32.013839 21663 raft_consensus.cc:2463] T af330dcd4767423ea4c179be7f88d0c4 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8bf6e26e98b14566899578a990b27d47 in term 0.
I20250109 08:57:32.014261 21704 raft_consensus.cc:738] T 90978f185a374feba3bc19b54f9d97f5 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Initialized, Role: FOLLOWER
I20250109 08:57:32.014030 21742 leader_election.cc:290] T af330dcd4767423ea4c179be7f88d0c4 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:32.015923 21477 leader_election.cc:304] T af330dcd4767423ea4c179be7f88d0c4 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, 8bf6e26e98b14566899578a990b27d47; no voters: a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:57:32.016983 21742 raft_consensus.cc:2798] T af330dcd4767423ea4c179be7f88d0c4 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:57:32.017307 21742 raft_consensus.cc:491] T af330dcd4767423ea4c179be7f88d0c4 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:57:32.017650 21703 tablet_bootstrap.cc:492] T f5f85ad7c7c94acfaf9d550d55fce48c P 8bf6e26e98b14566899578a990b27d47: No bootstrap required, opened a new log
I20250109 08:57:32.017630 21742 raft_consensus.cc:3054] T af330dcd4767423ea4c179be7f88d0c4 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:32.018246 21703 ts_tablet_manager.cc:1397] T f5f85ad7c7c94acfaf9d550d55fce48c P 8bf6e26e98b14566899578a990b27d47: Time spent bootstrapping tablet: real 0.056s user 0.011s sys 0.005s
I20250109 08:57:32.023180 21703 raft_consensus.cc:357] T f5f85ad7c7c94acfaf9d550d55fce48c P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:32.023857 21703 raft_consensus.cc:383] T f5f85ad7c7c94acfaf9d550d55fce48c P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:32.024119 21703 raft_consensus.cc:738] T f5f85ad7c7c94acfaf9d550d55fce48c P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bf6e26e98b14566899578a990b27d47, State: Initialized, Role: FOLLOWER
I20250109 08:57:32.024835 21703 consensus_queue.cc:260] T f5f85ad7c7c94acfaf9d550d55fce48c P 8bf6e26e98b14566899578a990b27d47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:32.015712 21704 consensus_queue.cc:260] T 90978f185a374feba3bc19b54f9d97f5 P 787ddcc824c2490a808fc5a53452dc5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:32.029423 21704 ts_tablet_manager.cc:1428] T 90978f185a374feba3bc19b54f9d97f5 P 787ddcc824c2490a808fc5a53452dc5e: Time spent starting tablet: real 0.018s user 0.005s sys 0.002s
I20250109 08:57:32.029913 21703 ts_tablet_manager.cc:1428] T f5f85ad7c7c94acfaf9d550d55fce48c P 8bf6e26e98b14566899578a990b27d47: Time spent starting tablet: real 0.011s user 0.006s sys 0.000s
I20250109 08:57:32.030835 21703 tablet_bootstrap.cc:492] T eb4516b9a16041c2adf34be520768f3f P 8bf6e26e98b14566899578a990b27d47: Bootstrap starting.
I20250109 08:57:32.032630 21702 tablet_bootstrap.cc:492] T c99aacce09d44f889be5480854a22760 P a2b1a692435241ee988e0bd1661cd4c3: No bootstrap required, opened a new log
I20250109 08:57:32.033062 21702 ts_tablet_manager.cc:1397] T c99aacce09d44f889be5480854a22760 P a2b1a692435241ee988e0bd1661cd4c3: Time spent bootstrapping tablet: real 0.034s user 0.012s sys 0.010s
I20250109 08:57:32.033543 21514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "239ca74a4c0a455a9db2652d7debb989" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47"
I20250109 08:57:32.034113 21901 leader_election.cc:290] T 239ca74a4c0a455a9db2652d7debb989 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 election: Requested vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
I20250109 08:57:32.035504 21704 tablet_bootstrap.cc:492] T eb4516b9a16041c2adf34be520768f3f P 787ddcc824c2490a808fc5a53452dc5e: Bootstrap starting.
I20250109 08:57:32.035858 21587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "239ca74a4c0a455a9db2652d7debb989" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3"
I20250109 08:57:32.036473 21587 raft_consensus.cc:3054] T 239ca74a4c0a455a9db2652d7debb989 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:32.037531 21702 raft_consensus.cc:357] T c99aacce09d44f889be5480854a22760 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:32.038203 21702 raft_consensus.cc:383] T c99aacce09d44f889be5480854a22760 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:32.038442 21702 raft_consensus.cc:738] T c99aacce09d44f889be5480854a22760 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2b1a692435241ee988e0bd1661cd4c3, State: Initialized, Role: FOLLOWER
I20250109 08:57:32.039008 21702 consensus_queue.cc:260] T c99aacce09d44f889be5480854a22760 P a2b1a692435241ee988e0bd1661cd4c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:32.041183 21702 ts_tablet_manager.cc:1428] T c99aacce09d44f889be5480854a22760 P a2b1a692435241ee988e0bd1661cd4c3: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20250109 08:57:32.041942 21702 tablet_bootstrap.cc:492] T af330dcd4767423ea4c179be7f88d0c4 P a2b1a692435241ee988e0bd1661cd4c3: Bootstrap starting.
I20250109 08:57:32.045764 21704 tablet_bootstrap.cc:654] T eb4516b9a16041c2adf34be520768f3f P 787ddcc824c2490a808fc5a53452dc5e: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:32.046350 21703 tablet_bootstrap.cc:654] T eb4516b9a16041c2adf34be520768f3f P 8bf6e26e98b14566899578a990b27d47: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:32.047586 21702 tablet_bootstrap.cc:654] T af330dcd4767423ea4c179be7f88d0c4 P a2b1a692435241ee988e0bd1661cd4c3: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:32.053429 21709 raft_consensus.cc:491] T c99aacce09d44f889be5480854a22760 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:32.053920 21709 raft_consensus.cc:513] T c99aacce09d44f889be5480854a22760 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:32.034159 21514 raft_consensus.cc:3054] T 239ca74a4c0a455a9db2652d7debb989 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:32.056299 21709 leader_election.cc:290] T c99aacce09d44f889be5480854a22760 P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:32.059948 21587 raft_consensus.cc:2463] T 239ca74a4c0a455a9db2652d7debb989 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 1.
I20250109 08:57:32.066004 21514 raft_consensus.cc:2463] T 239ca74a4c0a455a9db2652d7debb989 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 1.
I20250109 08:57:32.069165 21627 leader_election.cc:304] T 239ca74a4c0a455a9db2652d7debb989 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, a2b1a692435241ee988e0bd1661cd4c3; no voters:
I20250109 08:57:32.078116 21514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c99aacce09d44f889be5480854a22760" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47" is_pre_election: true
I20250109 08:57:32.078785 21514 raft_consensus.cc:2463] T c99aacce09d44f889be5480854a22760 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2b1a692435241ee988e0bd1661cd4c3 in term 0.
I20250109 08:57:32.080029 21703 tablet_bootstrap.cc:492] T eb4516b9a16041c2adf34be520768f3f P 8bf6e26e98b14566899578a990b27d47: No bootstrap required, opened a new log
I20250109 08:57:32.080353 21703 ts_tablet_manager.cc:1397] T eb4516b9a16041c2adf34be520768f3f P 8bf6e26e98b14566899578a990b27d47: Time spent bootstrapping tablet: real 0.050s user 0.007s sys 0.006s
I20250109 08:57:32.081327 21907 raft_consensus.cc:2798] T 239ca74a4c0a455a9db2652d7debb989 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:57:32.082414 21907 raft_consensus.cc:695] T 239ca74a4c0a455a9db2652d7debb989 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Becoming Leader. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Running, Role: LEADER
I20250109 08:57:32.076433 21663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c99aacce09d44f889be5480854a22760" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e" is_pre_election: true
I20250109 08:57:32.083204 21907 consensus_queue.cc:237] T 239ca74a4c0a455a9db2652d7debb989 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
W20250109 08:57:32.084497 21552 leader_election.cc:343] T c99aacce09d44f889be5480854a22760 P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:32.089664 21704 tablet_bootstrap.cc:492] T eb4516b9a16041c2adf34be520768f3f P 787ddcc824c2490a808fc5a53452dc5e: No bootstrap required, opened a new log
I20250109 08:57:32.090144 21704 ts_tablet_manager.cc:1397] T eb4516b9a16041c2adf34be520768f3f P 787ddcc824c2490a808fc5a53452dc5e: Time spent bootstrapping tablet: real 0.055s user 0.011s sys 0.014s
I20250109 08:57:32.092502 21709 raft_consensus.cc:491] T b00671c87aac405a81c6543f4defd722 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:32.092898 21709 raft_consensus.cc:513] T b00671c87aac405a81c6543f4defd722 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:32.093987 21552 leader_election.cc:304] T c99aacce09d44f889be5480854a22760 P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3; no voters: 787ddcc824c2490a808fc5a53452dc5e
I20250109 08:57:32.094287 21709 leader_election.cc:290] T b00671c87aac405a81c6543f4defd722 P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:32.094766 21819 raft_consensus.cc:2798] T c99aacce09d44f889be5480854a22760 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:57:32.097141 21819 raft_consensus.cc:491] T c99aacce09d44f889be5480854a22760 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:57:32.097487 21819 raft_consensus.cc:3054] T c99aacce09d44f889be5480854a22760 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:32.099442 21514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b00671c87aac405a81c6543f4defd722" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47" is_pre_election: true
I20250109 08:57:32.102088 21704 raft_consensus.cc:357] T eb4516b9a16041c2adf34be520768f3f P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:32.102679 21704 raft_consensus.cc:383] T eb4516b9a16041c2adf34be520768f3f P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:32.096522 21663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b00671c87aac405a81c6543f4defd722" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e" is_pre_election: true
W20250109 08:57:32.104013 21552 leader_election.cc:343] T b00671c87aac405a81c6543f4defd722 P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:32.102948 21704 raft_consensus.cc:738] T eb4516b9a16041c2adf34be520768f3f P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Initialized, Role: FOLLOWER
W20250109 08:57:32.104725 21552 leader_election.cc:343] T b00671c87aac405a81c6543f4defd722 P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:32.104888 21704 consensus_queue.cc:260] T eb4516b9a16041c2adf34be520768f3f P 787ddcc824c2490a808fc5a53452dc5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:32.105180 21552 leader_election.cc:304] T b00671c87aac405a81c6543f4defd722 P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2b1a692435241ee988e0bd1661cd4c3; no voters: 787ddcc824c2490a808fc5a53452dc5e, 8bf6e26e98b14566899578a990b27d47
I20250109 08:57:32.105890 21819 raft_consensus.cc:513] T c99aacce09d44f889be5480854a22760 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:32.106207 21709 raft_consensus.cc:2743] T b00671c87aac405a81c6543f4defd722 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:57:32.106211 21731 consensus_queue.cc:1035] T e8ff570914d74df8b87296010e9b864f P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250109 08:57:32.107460 21702 tablet_bootstrap.cc:492] T af330dcd4767423ea4c179be7f88d0c4 P a2b1a692435241ee988e0bd1661cd4c3: No bootstrap required, opened a new log
I20250109 08:57:32.108564 21702 ts_tablet_manager.cc:1397] T af330dcd4767423ea4c179be7f88d0c4 P a2b1a692435241ee988e0bd1661cd4c3: Time spent bootstrapping tablet: real 0.067s user 0.010s sys 0.020s
I20250109 08:57:32.109503 21704 ts_tablet_manager.cc:1428] T eb4516b9a16041c2adf34be520768f3f P 787ddcc824c2490a808fc5a53452dc5e: Time spent starting tablet: real 0.019s user 0.007s sys 0.000s
I20250109 08:57:32.110337 21704 tablet_bootstrap.cc:492] T b00671c87aac405a81c6543f4defd722 P 787ddcc824c2490a808fc5a53452dc5e: Bootstrap starting.
I20250109 08:57:32.106213 21742 raft_consensus.cc:513] T af330dcd4767423ea4c179be7f88d0c4 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:32.112881 21702 raft_consensus.cc:357] T af330dcd4767423ea4c179be7f88d0c4 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:32.113314 21703 raft_consensus.cc:357] T eb4516b9a16041c2adf34be520768f3f P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:32.113916 21702 raft_consensus.cc:383] T af330dcd4767423ea4c179be7f88d0c4 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:32.113966 21703 raft_consensus.cc:383] T eb4516b9a16041c2adf34be520768f3f P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:32.114331 21702 raft_consensus.cc:738] T af330dcd4767423ea4c179be7f88d0c4 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2b1a692435241ee988e0bd1661cd4c3, State: Initialized, Role: FOLLOWER
I20250109 08:57:32.114420 21703 raft_consensus.cc:738] T eb4516b9a16041c2adf34be520768f3f P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bf6e26e98b14566899578a990b27d47, State: Initialized, Role: FOLLOWER
I20250109 08:57:32.114943 21702 consensus_queue.cc:260] T af330dcd4767423ea4c179be7f88d0c4 P a2b1a692435241ee988e0bd1661cd4c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:32.115859 21703 consensus_queue.cc:260] T eb4516b9a16041c2adf34be520768f3f P 8bf6e26e98b14566899578a990b27d47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:32.132508 21819 leader_election.cc:290] T c99aacce09d44f889be5480854a22760 P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 election: Requested vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:32.133620 21514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c99aacce09d44f889be5480854a22760" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47"
I20250109 08:57:32.134209 21514 raft_consensus.cc:3054] T c99aacce09d44f889be5480854a22760 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:32.140429 21702 ts_tablet_manager.cc:1428] T af330dcd4767423ea4c179be7f88d0c4 P a2b1a692435241ee988e0bd1661cd4c3: Time spent starting tablet: real 0.032s user 0.007s sys 0.000s
I20250109 08:57:32.141173 21702 tablet_bootstrap.cc:492] T c18c1d46a6ca47598d7b161ed70a4e7a P a2b1a692435241ee988e0bd1661cd4c3: Bootstrap starting.
I20250109 08:57:32.142892 21663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c99aacce09d44f889be5480854a22760" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e"
W20250109 08:57:32.146862 21552 leader_election.cc:343] T c99aacce09d44f889be5480854a22760 P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:32.145889 21514 raft_consensus.cc:2463] T c99aacce09d44f889be5480854a22760 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2b1a692435241ee988e0bd1661cd4c3 in term 1.
I20250109 08:57:32.151785 21907 raft_consensus.cc:491] T 90978f185a374feba3bc19b54f9d97f5 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:32.147588 21663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af330dcd4767423ea4c179be7f88d0c4" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e"
I20250109 08:57:32.153774 21703 ts_tablet_manager.cc:1428] T eb4516b9a16041c2adf34be520768f3f P 8bf6e26e98b14566899578a990b27d47: Time spent starting tablet: real 0.052s user 0.006s sys 0.000s
I20250109 08:57:32.154776 21552 leader_election.cc:304] T c99aacce09d44f889be5480854a22760 P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3; no voters: 787ddcc824c2490a808fc5a53452dc5e
I20250109 08:57:32.155910 21759 raft_consensus.cc:2798] T c99aacce09d44f889be5480854a22760 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:57:32.152272 21907 raft_consensus.cc:513] T 90978f185a374feba3bc19b54f9d97f5 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:32.156399 21759 raft_consensus.cc:695] T c99aacce09d44f889be5480854a22760 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 LEADER]: Becoming Leader. State: Replica: a2b1a692435241ee988e0bd1661cd4c3, State: Running, Role: LEADER
I20250109 08:57:32.156980 21703 tablet_bootstrap.cc:492] T f961d3f357944118a69f905621731552 P 8bf6e26e98b14566899578a990b27d47: Bootstrap starting.
I20250109 08:57:32.147796 21587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af330dcd4767423ea4c179be7f88d0c4" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3"
I20250109 08:57:32.158028 21663 raft_consensus.cc:3054] T af330dcd4767423ea4c179be7f88d0c4 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:32.158339 21759 consensus_queue.cc:237] T c99aacce09d44f889be5480854a22760 P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:32.160410 21408 catalog_manager.cc:5526] T 239ca74a4c0a455a9db2652d7debb989 P 787ddcc824c2490a808fc5a53452dc5e reported cstate change: term changed from 0 to 1, leader changed from <none> to 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195). New cstate: current_term: 1 leader_uuid: "787ddcc824c2490a808fc5a53452dc5e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: HEALTHY } } }
I20250109 08:57:32.148487 21742 leader_election.cc:290] T af330dcd4767423ea4c179be7f88d0c4 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 election: Requested vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:32.158298 21587 raft_consensus.cc:3054] T af330dcd4767423ea4c179be7f88d0c4 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:32.152407 21819 raft_consensus.cc:491] T 0f3bde1a1ccd4c359a2e80feca968453 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:32.166709 21819 raft_consensus.cc:513] T 0f3bde1a1ccd4c359a2e80feca968453 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:32.167063 21907 leader_election.cc:290] T 90978f185a374feba3bc19b54f9d97f5 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
I20250109 08:57:32.169152 21819 leader_election.cc:290] T 0f3bde1a1ccd4c359a2e80feca968453 P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:32.170233 21661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f3bde1a1ccd4c359a2e80feca968453" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e" is_pre_election: true
I20250109 08:57:32.170357 21513 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90978f185a374feba3bc19b54f9d97f5" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47" is_pre_election: true
I20250109 08:57:32.171155 21513 raft_consensus.cc:2388] T 90978f185a374feba3bc19b54f9d97f5 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 787ddcc824c2490a808fc5a53452dc5e in current term 1: Already voted for candidate a2b1a692435241ee988e0bd1661cd4c3 in this term.
I20250109 08:57:32.171746 21589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90978f185a374feba3bc19b54f9d97f5" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3" is_pre_election: true
I20250109 08:57:32.173789 21626 leader_election.cc:304] T 90978f185a374feba3bc19b54f9d97f5 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e; no voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:57:32.175659 21907 raft_consensus.cc:3054] T 90978f185a374feba3bc19b54f9d97f5 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:32.183355 21514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f3bde1a1ccd4c359a2e80feca968453" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47" is_pre_election: true
I20250109 08:57:32.187562 21552 leader_election.cc:304] T 0f3bde1a1ccd4c359a2e80feca968453 P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2b1a692435241ee988e0bd1661cd4c3; no voters: 787ddcc824c2490a808fc5a53452dc5e, 8bf6e26e98b14566899578a990b27d47
I20250109 08:57:32.188349 21819 raft_consensus.cc:2743] T 0f3bde1a1ccd4c359a2e80feca968453 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:57:32.188724 21711 consensus_queue.cc:1035] T e8ff570914d74df8b87296010e9b864f P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250109 08:57:32.190717 21663 raft_consensus.cc:2463] T af330dcd4767423ea4c179be7f88d0c4 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8bf6e26e98b14566899578a990b27d47 in term 1.
I20250109 08:57:32.193154 21477 leader_election.cc:304] T af330dcd4767423ea4c179be7f88d0c4 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, 8bf6e26e98b14566899578a990b27d47; no voters:
I20250109 08:57:32.195081 21742 raft_consensus.cc:2798] T af330dcd4767423ea4c179be7f88d0c4 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:57:32.195540 21742 raft_consensus.cc:695] T af330dcd4767423ea4c179be7f88d0c4 P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Becoming Leader. State: Replica: 8bf6e26e98b14566899578a990b27d47, State: Running, Role: LEADER
I20250109 08:57:32.209545 21842 raft_consensus.cc:491] T f5f85ad7c7c94acfaf9d550d55fce48c P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:32.210038 21842 raft_consensus.cc:513] T f5f85ad7c7c94acfaf9d550d55fce48c P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:32.212316 21702 tablet_bootstrap.cc:654] T c18c1d46a6ca47598d7b161ed70a4e7a P a2b1a692435241ee988e0bd1661cd4c3: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:32.215667 21704 tablet_bootstrap.cc:654] T b00671c87aac405a81c6543f4defd722 P 787ddcc824c2490a808fc5a53452dc5e: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:32.222177 21842 leader_election.cc:290] T f5f85ad7c7c94acfaf9d550d55fce48c P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:32.222863 21589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5f85ad7c7c94acfaf9d550d55fce48c" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3" is_pre_election: true
I20250109 08:57:32.223311 21662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5f85ad7c7c94acfaf9d550d55fce48c" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e" is_pre_election: true
W20250109 08:57:32.224457 21478 leader_election.cc:343] T f5f85ad7c7c94acfaf9d550d55fce48c P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Illegal state: must be running to vote when last-logged opid is not known
W20250109 08:57:32.225174 21477 leader_election.cc:343] T f5f85ad7c7c94acfaf9d550d55fce48c P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:32.225562 21477 leader_election.cc:304] T f5f85ad7c7c94acfaf9d550d55fce48c P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8bf6e26e98b14566899578a990b27d47; no voters: 787ddcc824c2490a808fc5a53452dc5e, a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:57:32.226735 21907 raft_consensus.cc:2743] T 90978f185a374feba3bc19b54f9d97f5 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:57:32.200850 21742 consensus_queue.cc:237] T af330dcd4767423ea4c179be7f88d0c4 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:32.228636 21731 consensus_queue.cc:1035] T af273bab41cc4a9da780147eebfd52ec P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:57:32.236282 21842 raft_consensus.cc:2743] T f5f85ad7c7c94acfaf9d550d55fce48c P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:57:32.247560 21731 raft_consensus.cc:491] T e03846e461974d0d8944c344e715b86a P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:32.248237 21731 raft_consensus.cc:513] T e03846e461974d0d8944c344e715b86a P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:32.250551 21410 catalog_manager.cc:5526] T c99aacce09d44f889be5480854a22760 P a2b1a692435241ee988e0bd1661cd4c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194). New cstate: current_term: 1 leader_uuid: "a2b1a692435241ee988e0bd1661cd4c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: UNKNOWN } } }
I20250109 08:57:32.252224 21587 raft_consensus.cc:2463] T af330dcd4767423ea4c179be7f88d0c4 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8bf6e26e98b14566899578a990b27d47 in term 1.
I20250109 08:57:32.252877 21703 tablet_bootstrap.cc:654] T f961d3f357944118a69f905621731552 P 8bf6e26e98b14566899578a990b27d47: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:32.268455 21819 raft_consensus.cc:491] T e03846e461974d0d8944c344e715b86a P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:32.269024 21819 raft_consensus.cc:513] T e03846e461974d0d8944c344e715b86a P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:32.281226 21731 leader_election.cc:290] T e03846e461974d0d8944c344e715b86a P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:32.282511 21819 leader_election.cc:290] T e03846e461974d0d8944c344e715b86a P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:32.290292 21702 tablet_bootstrap.cc:492] T c18c1d46a6ca47598d7b161ed70a4e7a P a2b1a692435241ee988e0bd1661cd4c3: No bootstrap required, opened a new log
I20250109 08:57:32.290851 21702 ts_tablet_manager.cc:1397] T c18c1d46a6ca47598d7b161ed70a4e7a P a2b1a692435241ee988e0bd1661cd4c3: Time spent bootstrapping tablet: real 0.150s user 0.018s sys 0.012s
I20250109 08:57:32.299206 21704 tablet_bootstrap.cc:492] T b00671c87aac405a81c6543f4defd722 P 787ddcc824c2490a808fc5a53452dc5e: No bootstrap required, opened a new log
I20250109 08:57:32.299664 21704 ts_tablet_manager.cc:1397] T b00671c87aac405a81c6543f4defd722 P 787ddcc824c2490a808fc5a53452dc5e: Time spent bootstrapping tablet: real 0.189s user 0.025s sys 0.006s
I20250109 08:57:32.291923 21907 raft_consensus.cc:491] T c18c1d46a6ca47598d7b161ed70a4e7a P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:32.300865 21907 raft_consensus.cc:513] T c18c1d46a6ca47598d7b161ed70a4e7a P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:32.302793 21907 leader_election.cc:290] T c18c1d46a6ca47598d7b161ed70a4e7a P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
I20250109 08:57:32.309151 21662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e03846e461974d0d8944c344e715b86a" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e" is_pre_election: true
I20250109 08:57:32.309868 21662 raft_consensus.cc:2463] T e03846e461974d0d8944c344e715b86a P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2b1a692435241ee988e0bd1661cd4c3 in term 0.
I20250109 08:57:32.310312 21587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e03846e461974d0d8944c344e715b86a" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3" is_pre_election: true
I20250109 08:57:32.338646 21552 leader_election.cc:304] T e03846e461974d0d8944c344e715b86a P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, a2b1a692435241ee988e0bd1661cd4c3; no voters:
I20250109 08:57:32.338745 21511 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c18c1d46a6ca47598d7b161ed70a4e7a" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47" is_pre_election: true
I20250109 08:57:32.339844 21514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e03846e461974d0d8944c344e715b86a" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47" is_pre_election: true
I20250109 08:57:32.340620 21514 raft_consensus.cc:2463] T e03846e461974d0d8944c344e715b86a P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2b1a692435241ee988e0bd1661cd4c3 in term 0.
I20250109 08:57:32.344360 21589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c18c1d46a6ca47598d7b161ed70a4e7a" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3" is_pre_election: true
I20250109 08:57:32.344252 21411 catalog_manager.cc:5526] T af330dcd4767423ea4c179be7f88d0c4 P 8bf6e26e98b14566899578a990b27d47 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8bf6e26e98b14566899578a990b27d47 (127.19.226.193). New cstate: current_term: 1 leader_uuid: "8bf6e26e98b14566899578a990b27d47" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: UNKNOWN } } }
I20250109 08:57:32.347462 21588 raft_consensus.cc:3054] T df9715570a6844d9a2686307db16eff1 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:32.351817 21662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e03846e461974d0d8944c344e715b86a" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e" is_pre_election: true
W20250109 08:57:32.352267 21460 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20250109 08:57:32.346874 21511 raft_consensus.cc:2463] T c18c1d46a6ca47598d7b161ed70a4e7a P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 0.
I20250109 08:57:32.352470 21662 raft_consensus.cc:2463] T e03846e461974d0d8944c344e715b86a P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8bf6e26e98b14566899578a990b27d47 in term 0.
I20250109 08:57:32.354502 21626 leader_election.cc:304] T c18c1d46a6ca47598d7b161ed70a4e7a P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, 8bf6e26e98b14566899578a990b27d47; no voters:
I20250109 08:57:32.355636 21758 raft_consensus.cc:2798] T c18c1d46a6ca47598d7b161ed70a4e7a P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:57:32.356024 21758 raft_consensus.cc:491] T c18c1d46a6ca47598d7b161ed70a4e7a P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:57:32.356338 21758 raft_consensus.cc:3054] T c18c1d46a6ca47598d7b161ed70a4e7a P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:32.359827 21585 raft_consensus.cc:3054] T 0f3bde1a1ccd4c359a2e80feca968453 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:32.362020 21477 leader_election.cc:304] T e03846e461974d0d8944c344e715b86a P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, 8bf6e26e98b14566899578a990b27d47; no voters:
I20250109 08:57:32.365778 21842 raft_consensus.cc:2798] T e03846e461974d0d8944c344e715b86a P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:57:32.366144 21842 raft_consensus.cc:491] T e03846e461974d0d8944c344e715b86a P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:57:32.366504 21842 raft_consensus.cc:3054] T e03846e461974d0d8944c344e715b86a P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:32.367444 21703 tablet_bootstrap.cc:492] T f961d3f357944118a69f905621731552 P 8bf6e26e98b14566899578a990b27d47: No bootstrap required, opened a new log
I20250109 08:57:32.344539 21587 raft_consensus.cc:2463] T e03846e461974d0d8944c344e715b86a P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8bf6e26e98b14566899578a990b27d47 in term 0.
I20250109 08:57:32.368055 21703 ts_tablet_manager.cc:1397] T f961d3f357944118a69f905621731552 P 8bf6e26e98b14566899578a990b27d47: Time spent bootstrapping tablet: real 0.213s user 0.013s sys 0.008s
I20250109 08:57:32.370328 21703 raft_consensus.cc:357] T f961d3f357944118a69f905621731552 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:32.370997 21703 raft_consensus.cc:383] T f961d3f357944118a69f905621731552 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:32.371258 21703 raft_consensus.cc:738] T f961d3f357944118a69f905621731552 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bf6e26e98b14566899578a990b27d47, State: Initialized, Role: FOLLOWER
I20250109 08:57:32.371980 21703 consensus_queue.cc:260] T f961d3f357944118a69f905621731552 P 8bf6e26e98b14566899578a990b27d47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:32.372476 21709 raft_consensus.cc:2798] T e03846e461974d0d8944c344e715b86a P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:57:32.374131 21703 ts_tablet_manager.cc:1428] T f961d3f357944118a69f905621731552 P 8bf6e26e98b14566899578a990b27d47: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20250109 08:57:32.374217 21709 raft_consensus.cc:491] T e03846e461974d0d8944c344e715b86a P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:57:32.374600 21709 raft_consensus.cc:3054] T e03846e461974d0d8944c344e715b86a P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:32.374971 21703 tablet_bootstrap.cc:492] T d015a10f95eb4f19b6d7c53cb201fa80 P 8bf6e26e98b14566899578a990b27d47: Bootstrap starting.
I20250109 08:57:32.379727 21702 raft_consensus.cc:357] T c18c1d46a6ca47598d7b161ed70a4e7a P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:32.380429 21702 raft_consensus.cc:383] T c18c1d46a6ca47598d7b161ed70a4e7a P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:32.380616 21702 raft_consensus.cc:738] T c18c1d46a6ca47598d7b161ed70a4e7a P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2b1a692435241ee988e0bd1661cd4c3, State: Initialized, Role: FOLLOWER
I20250109 08:57:32.385222 21702 consensus_queue.cc:260] T c18c1d46a6ca47598d7b161ed70a4e7a P a2b1a692435241ee988e0bd1661cd4c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:32.385921 21589 raft_consensus.cc:2463] T c18c1d46a6ca47598d7b161ed70a4e7a P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 0.
I20250109 08:57:32.386530 21732 raft_consensus.cc:491] T eb4516b9a16041c2adf34be520768f3f P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:32.387017 21732 raft_consensus.cc:513] T eb4516b9a16041c2adf34be520768f3f P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:32.388746 21732 leader_election.cc:290] T eb4516b9a16041c2adf34be520768f3f P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
I20250109 08:57:32.389292 21702 ts_tablet_manager.cc:1428] T c18c1d46a6ca47598d7b161ed70a4e7a P a2b1a692435241ee988e0bd1661cd4c3: Time spent starting tablet: real 0.098s user 0.005s sys 0.000s
I20250109 08:57:32.390128 21702 tablet_bootstrap.cc:492] T f961d3f357944118a69f905621731552 P a2b1a692435241ee988e0bd1661cd4c3: Bootstrap starting.
I20250109 08:57:32.388051 21704 raft_consensus.cc:357] T b00671c87aac405a81c6543f4defd722 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:32.393005 21704 raft_consensus.cc:383] T b00671c87aac405a81c6543f4defd722 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:32.393286 21704 raft_consensus.cc:738] T b00671c87aac405a81c6543f4defd722 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Initialized, Role: FOLLOWER
I20250109 08:57:32.395459 21514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb4516b9a16041c2adf34be520768f3f" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47" is_pre_election: true
I20250109 08:57:32.396071 21514 raft_consensus.cc:2463] T eb4516b9a16041c2adf34be520768f3f P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 0.
I20250109 08:57:32.397181 21758 raft_consensus.cc:513] T c18c1d46a6ca47598d7b161ed70a4e7a P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:32.397538 21842 raft_consensus.cc:513] T e03846e461974d0d8944c344e715b86a P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:32.399245 21842 leader_election.cc:290] T e03846e461974d0d8944c344e715b86a P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 election: Requested vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:32.399770 21758 leader_election.cc:290] T c18c1d46a6ca47598d7b161ed70a4e7a P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 election: Requested vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
I20250109 08:57:32.401029 21704 consensus_queue.cc:260] T b00671c87aac405a81c6543f4defd722 P 787ddcc824c2490a808fc5a53452dc5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:32.401927 21709 raft_consensus.cc:513] T e03846e461974d0d8944c344e715b86a P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:32.408936 21514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c18c1d46a6ca47598d7b161ed70a4e7a" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47"
I20250109 08:57:32.409523 21514 raft_consensus.cc:3054] T c18c1d46a6ca47598d7b161ed70a4e7a P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:32.400348 21626 leader_election.cc:304] T eb4516b9a16041c2adf34be520768f3f P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, 8bf6e26e98b14566899578a990b27d47; no voters:
I20250109 08:57:32.410974 21704 ts_tablet_manager.cc:1428] T b00671c87aac405a81c6543f4defd722 P 787ddcc824c2490a808fc5a53452dc5e: Time spent starting tablet: real 0.111s user 0.010s sys 0.027s
I20250109 08:57:32.411757 21709 leader_election.cc:290] T e03846e461974d0d8944c344e715b86a P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 election: Requested vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:32.413654 21758 raft_consensus.cc:2798] T eb4516b9a16041c2adf34be520768f3f P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:57:32.399430 21819 consensus_queue.cc:1035] T 90978f185a374feba3bc19b54f9d97f5 P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.015s
I20250109 08:57:32.414595 21758 raft_consensus.cc:491] T eb4516b9a16041c2adf34be520768f3f P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:57:32.415004 21758 raft_consensus.cc:3054] T eb4516b9a16041c2adf34be520768f3f P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:32.417680 21704 tablet_bootstrap.cc:492] T 7781c00473024307987096b596da12ac P 787ddcc824c2490a808fc5a53452dc5e: Bootstrap starting.
I20250109 08:57:32.418720 21659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e03846e461974d0d8944c344e715b86a" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e"
I20250109 08:57:32.419420 21659 raft_consensus.cc:3054] T e03846e461974d0d8944c344e715b86a P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:32.426082 21511 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e03846e461974d0d8944c344e715b86a" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47"
I20250109 08:57:32.423548 21514 raft_consensus.cc:2463] T c18c1d46a6ca47598d7b161ed70a4e7a P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 1.
I20250109 08:57:32.427754 21511 raft_consensus.cc:2388] T e03846e461974d0d8944c344e715b86a P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate a2b1a692435241ee988e0bd1661cd4c3 in current term 1: Already voted for candidate 8bf6e26e98b14566899578a990b27d47 in this term.
I20250109 08:57:32.418740 21842 raft_consensus.cc:491] T 8c0459d2eee64f0a83bd05ad5974f506 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:32.428723 21842 raft_consensus.cc:513] T 8c0459d2eee64f0a83bd05ad5974f506 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:32.430721 21710 consensus_queue.cc:1035] T df9715570a6844d9a2686307db16eff1 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20250109 08:57:32.438165 21626 leader_election.cc:304] T c18c1d46a6ca47598d7b161ed70a4e7a P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, 8bf6e26e98b14566899578a990b27d47; no voters:
I20250109 08:57:32.441426 21842 leader_election.cc:290] T 8c0459d2eee64f0a83bd05ad5974f506 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:32.442427 21588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb4516b9a16041c2adf34be520768f3f" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3" is_pre_election: true
I20250109 08:57:32.444526 21732 raft_consensus.cc:2798] T c18c1d46a6ca47598d7b161ed70a4e7a P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:57:32.444890 21732 raft_consensus.cc:695] T c18c1d46a6ca47598d7b161ed70a4e7a P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Becoming Leader. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Running, Role: LEADER
I20250109 08:57:32.445407 21732 consensus_queue.cc:237] T c18c1d46a6ca47598d7b161ed70a4e7a P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:32.434597 21731 consensus_queue.cc:1035] T af273bab41cc4a9da780147eebfd52ec P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20250109 08:57:32.452008 21589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c18c1d46a6ca47598d7b161ed70a4e7a" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3"
I20250109 08:57:32.452608 21589 raft_consensus.cc:3054] T c18c1d46a6ca47598d7b161ed70a4e7a P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:32.461880 21589 raft_consensus.cc:2463] T c18c1d46a6ca47598d7b161ed70a4e7a P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 1.
I20250109 08:57:32.465641 21663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e03846e461974d0d8944c344e715b86a" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e"
I20250109 08:57:32.468631 21588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c0459d2eee64f0a83bd05ad5974f506" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3" is_pre_election: true
I20250109 08:57:32.470448 21589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e03846e461974d0d8944c344e715b86a" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3"
I20250109 08:57:32.471282 21589 raft_consensus.cc:2388] T e03846e461974d0d8944c344e715b86a P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 8bf6e26e98b14566899578a990b27d47 in current term 1: Already voted for candidate a2b1a692435241ee988e0bd1661cd4c3 in this term.
W20250109 08:57:32.473730 21478 leader_election.cc:343] T 8c0459d2eee64f0a83bd05ad5974f506 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:32.475809 21662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c0459d2eee64f0a83bd05ad5974f506" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e" is_pre_election: true
I20250109 08:57:32.485147 21907 consensus_queue.cc:1035] T df9715570a6844d9a2686307db16eff1 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20250109 08:57:32.487802 21627 leader_election.cc:343] T eb4516b9a16041c2adf34be520768f3f P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:32.470170 21758 raft_consensus.cc:513] T eb4516b9a16041c2adf34be520768f3f P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:32.497251 21704 tablet_bootstrap.cc:654] T 7781c00473024307987096b596da12ac P 787ddcc824c2490a808fc5a53452dc5e: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:32.497639 21411 catalog_manager.cc:5526] T c18c1d46a6ca47598d7b161ed70a4e7a P 787ddcc824c2490a808fc5a53452dc5e reported cstate change: term changed from 0 to 1, leader changed from <none> to 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195). New cstate: current_term: 1 leader_uuid: "787ddcc824c2490a808fc5a53452dc5e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: HEALTHY } } }
I20250109 08:57:32.500257 21758 leader_election.cc:290] T eb4516b9a16041c2adf34be520768f3f P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 election: Requested vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
I20250109 08:57:32.503458 21512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb4516b9a16041c2adf34be520768f3f" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47"
I20250109 08:57:32.504032 21512 raft_consensus.cc:3054] T eb4516b9a16041c2adf34be520768f3f P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:32.506878 21731 raft_consensus.cc:491] T f961d3f357944118a69f905621731552 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:32.507375 21731 raft_consensus.cc:513] T f961d3f357944118a69f905621731552 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:32.509449 21703 tablet_bootstrap.cc:654] T d015a10f95eb4f19b6d7c53cb201fa80 P 8bf6e26e98b14566899578a990b27d47: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:32.509714 21731 leader_election.cc:290] T f961d3f357944118a69f905621731552 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:32.510411 21702 tablet_bootstrap.cc:654] T f961d3f357944118a69f905621731552 P a2b1a692435241ee988e0bd1661cd4c3: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:32.537335 21584 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f961d3f357944118a69f905621731552" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3" is_pre_election: true
I20250109 08:57:32.533808 21512 raft_consensus.cc:2463] T eb4516b9a16041c2adf34be520768f3f P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 1.
I20250109 08:57:32.541479 21626 leader_election.cc:304] T eb4516b9a16041c2adf34be520768f3f P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, 8bf6e26e98b14566899578a990b27d47; no voters:
I20250109 08:57:32.541435 21659 raft_consensus.cc:2463] T e03846e461974d0d8944c344e715b86a P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2b1a692435241ee988e0bd1661cd4c3 in term 1.
I20250109 08:57:32.543243 21552 leader_election.cc:304] T e03846e461974d0d8944c344e715b86a P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, a2b1a692435241ee988e0bd1661cd4c3; no voters: 8bf6e26e98b14566899578a990b27d47
I20250109 08:57:32.544785 21907 raft_consensus.cc:2798] T eb4516b9a16041c2adf34be520768f3f P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:57:32.545274 21907 raft_consensus.cc:695] T eb4516b9a16041c2adf34be520768f3f P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Becoming Leader. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Running, Role: LEADER
I20250109 08:57:32.546512 21907 consensus_queue.cc:237] T eb4516b9a16041c2adf34be520768f3f P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:32.544351 21759 raft_consensus.cc:2798] T e03846e461974d0d8944c344e715b86a P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:57:32.549837 21759 raft_consensus.cc:695] T e03846e461974d0d8944c344e715b86a P a2b1a692435241ee988e0bd1661cd4c3 [term 1 LEADER]: Becoming Leader. State: Replica: a2b1a692435241ee988e0bd1661cd4c3, State: Running, Role: LEADER
I20250109 08:57:32.552280 21709 consensus_queue.cc:1035] T 90978f185a374feba3bc19b54f9d97f5 P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:57:32.553064 21759 consensus_queue.cc:237] T e03846e461974d0d8944c344e715b86a P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
W20250109 08:57:32.573860 21964 log.cc:926] Time spent T c18c1d46a6ca47598d7b161ed70a4e7a P 787ddcc824c2490a808fc5a53452dc5e: Append to log took a long time: real 0.052s user 0.001s sys 0.000s
W20250109 08:57:32.574610 21478 leader_election.cc:343] T f961d3f357944118a69f905621731552 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:32.583753 21477 leader_election.cc:304] T e03846e461974d0d8944c344e715b86a P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8bf6e26e98b14566899578a990b27d47; no voters: 787ddcc824c2490a808fc5a53452dc5e, a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:57:32.584826 21704 tablet_bootstrap.cc:492] T 7781c00473024307987096b596da12ac P 787ddcc824c2490a808fc5a53452dc5e: No bootstrap required, opened a new log
I20250109 08:57:32.584110 21410 catalog_manager.cc:5526] T e03846e461974d0d8944c344e715b86a P a2b1a692435241ee988e0bd1661cd4c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194). New cstate: current_term: 1 leader_uuid: "a2b1a692435241ee988e0bd1661cd4c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: UNKNOWN } } }
I20250109 08:57:32.585258 21704 ts_tablet_manager.cc:1397] T 7781c00473024307987096b596da12ac P 787ddcc824c2490a808fc5a53452dc5e: Time spent bootstrapping tablet: real 0.173s user 0.012s sys 0.020s
I20250109 08:57:32.585168 21731 raft_consensus.cc:2743] T e03846e461974d0d8944c344e715b86a P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20250109 08:57:32.588742 21704 raft_consensus.cc:357] T 7781c00473024307987096b596da12ac P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:32.589414 21704 raft_consensus.cc:383] T 7781c00473024307987096b596da12ac P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:32.589689 21704 raft_consensus.cc:738] T 7781c00473024307987096b596da12ac P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Initialized, Role: FOLLOWER
I20250109 08:57:32.590641 21704 consensus_queue.cc:260] T 7781c00473024307987096b596da12ac P 787ddcc824c2490a808fc5a53452dc5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:32.592929 21704 ts_tablet_manager.cc:1428] T 7781c00473024307987096b596da12ac P 787ddcc824c2490a808fc5a53452dc5e: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20250109 08:57:32.593744 21704 tablet_bootstrap.cc:492] T c99aacce09d44f889be5480854a22760 P 787ddcc824c2490a808fc5a53452dc5e: Bootstrap starting.
W20250109 08:57:32.595785 21477 leader_election.cc:343] T 8c0459d2eee64f0a83bd05ad5974f506 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:32.596297 21477 leader_election.cc:304] T 8c0459d2eee64f0a83bd05ad5974f506 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8bf6e26e98b14566899578a990b27d47; no voters: 787ddcc824c2490a808fc5a53452dc5e, a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:57:32.598340 21731 raft_consensus.cc:2743] T 8c0459d2eee64f0a83bd05ad5974f506 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:57:32.599279 21411 catalog_manager.cc:5526] T eb4516b9a16041c2adf34be520768f3f P 787ddcc824c2490a808fc5a53452dc5e reported cstate change: term changed from 0 to 1, leader changed from <none> to 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195). New cstate: current_term: 1 leader_uuid: "787ddcc824c2490a808fc5a53452dc5e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: HEALTHY } } }
I20250109 08:57:32.601840 21704 tablet_bootstrap.cc:654] T c99aacce09d44f889be5480854a22760 P 787ddcc824c2490a808fc5a53452dc5e: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:32.607877 21660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f961d3f357944118a69f905621731552" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e" is_pre_election: true
I20250109 08:57:32.612571 21587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb4516b9a16041c2adf34be520768f3f" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3"
I20250109 08:57:32.612857 21907 consensus_queue.cc:1035] T 239ca74a4c0a455a9db2652d7debb989 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20250109 08:57:32.614140 21477 leader_election.cc:343] T f961d3f357944118a69f905621731552 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:32.616588 21477 leader_election.cc:304] T f961d3f357944118a69f905621731552 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8bf6e26e98b14566899578a990b27d47; no voters: 787ddcc824c2490a808fc5a53452dc5e, a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:57:32.621104 21842 raft_consensus.cc:2743] T f961d3f357944118a69f905621731552 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:57:32.625007 21759 consensus_queue.cc:1035] T c99aacce09d44f889be5480854a22760 P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:57:32.630798 21702 tablet_bootstrap.cc:492] T f961d3f357944118a69f905621731552 P a2b1a692435241ee988e0bd1661cd4c3: No bootstrap required, opened a new log
I20250109 08:57:32.631263 21702 ts_tablet_manager.cc:1397] T f961d3f357944118a69f905621731552 P a2b1a692435241ee988e0bd1661cd4c3: Time spent bootstrapping tablet: real 0.241s user 0.008s sys 0.010s
I20250109 08:57:32.636265 21703 tablet_bootstrap.cc:492] T d015a10f95eb4f19b6d7c53cb201fa80 P 8bf6e26e98b14566899578a990b27d47: No bootstrap required, opened a new log
I20250109 08:57:32.636750 21703 ts_tablet_manager.cc:1397] T d015a10f95eb4f19b6d7c53cb201fa80 P 8bf6e26e98b14566899578a990b27d47: Time spent bootstrapping tablet: real 0.262s user 0.014s sys 0.024s
I20250109 08:57:32.639353 21702 raft_consensus.cc:357] T f961d3f357944118a69f905621731552 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:32.640074 21702 raft_consensus.cc:383] T f961d3f357944118a69f905621731552 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:32.640347 21702 raft_consensus.cc:738] T f961d3f357944118a69f905621731552 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2b1a692435241ee988e0bd1661cd4c3, State: Initialized, Role: FOLLOWER
I20250109 08:57:32.644049 21703 raft_consensus.cc:357] T d015a10f95eb4f19b6d7c53cb201fa80 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:32.646067 21703 raft_consensus.cc:383] T d015a10f95eb4f19b6d7c53cb201fa80 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:32.646343 21703 raft_consensus.cc:738] T d015a10f95eb4f19b6d7c53cb201fa80 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bf6e26e98b14566899578a990b27d47, State: Initialized, Role: FOLLOWER
I20250109 08:57:32.647002 21703 consensus_queue.cc:260] T d015a10f95eb4f19b6d7c53cb201fa80 P 8bf6e26e98b14566899578a990b27d47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:32.649549 21703 ts_tablet_manager.cc:1428] T d015a10f95eb4f19b6d7c53cb201fa80 P 8bf6e26e98b14566899578a990b27d47: Time spent starting tablet: real 0.013s user 0.004s sys 0.001s
I20250109 08:57:32.650498 21703 tablet_bootstrap.cc:492] T a3aa271f2f0144eea9261e0e07ab833b P 8bf6e26e98b14566899578a990b27d47: Bootstrap starting.
W20250109 08:57:32.623363 21627 leader_election.cc:343] T eb4516b9a16041c2adf34be520768f3f P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:32.646267 21702 consensus_queue.cc:260] T f961d3f357944118a69f905621731552 P a2b1a692435241ee988e0bd1661cd4c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:32.657691 21842 consensus_queue.cc:1035] T af330dcd4767423ea4c179be7f88d0c4 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:57:32.660979 21704 tablet_bootstrap.cc:492] T c99aacce09d44f889be5480854a22760 P 787ddcc824c2490a808fc5a53452dc5e: No bootstrap required, opened a new log
I20250109 08:57:32.661167 21703 tablet_bootstrap.cc:654] T a3aa271f2f0144eea9261e0e07ab833b P 8bf6e26e98b14566899578a990b27d47: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:32.661533 21704 ts_tablet_manager.cc:1397] T c99aacce09d44f889be5480854a22760 P 787ddcc824c2490a808fc5a53452dc5e: Time spent bootstrapping tablet: real 0.068s user 0.011s sys 0.014s
W20250109 08:57:32.664839 21552 consensus_peers.cc:487] T c99aacce09d44f889be5480854a22760 P a2b1a692435241ee988e0bd1661cd4c3 -> Peer 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731): Couldn't send request to peer 787ddcc824c2490a808fc5a53452dc5e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:57:32.672487 21702 ts_tablet_manager.cc:1428] T f961d3f357944118a69f905621731552 P a2b1a692435241ee988e0bd1661cd4c3: Time spent starting tablet: real 0.041s user 0.003s sys 0.005s
I20250109 08:57:32.673372 21702 tablet_bootstrap.cc:492] T ab6c3fcdf226433c9e1303004f6c96b4 P a2b1a692435241ee988e0bd1661cd4c3: Bootstrap starting.
I20250109 08:57:32.685914 21702 tablet_bootstrap.cc:654] T ab6c3fcdf226433c9e1303004f6c96b4 P a2b1a692435241ee988e0bd1661cd4c3: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:32.692483 21703 tablet_bootstrap.cc:492] T a3aa271f2f0144eea9261e0e07ab833b P 8bf6e26e98b14566899578a990b27d47: No bootstrap required, opened a new log
I20250109 08:57:32.692993 21703 ts_tablet_manager.cc:1397] T a3aa271f2f0144eea9261e0e07ab833b P 8bf6e26e98b14566899578a990b27d47: Time spent bootstrapping tablet: real 0.043s user 0.013s sys 0.005s
I20250109 08:57:32.693186 21704 raft_consensus.cc:357] T c99aacce09d44f889be5480854a22760 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:32.693882 21704 raft_consensus.cc:383] T c99aacce09d44f889be5480854a22760 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:32.694232 21704 raft_consensus.cc:738] T c99aacce09d44f889be5480854a22760 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Initialized, Role: FOLLOWER
I20250109 08:57:32.694888 21704 consensus_queue.cc:260] T c99aacce09d44f889be5480854a22760 P 787ddcc824c2490a808fc5a53452dc5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:32.696970 21740 consensus_queue.cc:1035] T af330dcd4767423ea4c179be7f88d0c4 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:57:32.701156 21819 raft_consensus.cc:491] T f961d3f357944118a69f905621731552 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:32.701453 21901 raft_consensus.cc:491] T 7781c00473024307987096b596da12ac P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:32.701742 21732 consensus_queue.cc:1035] T 239ca74a4c0a455a9db2652d7debb989 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:57:32.702009 21901 raft_consensus.cc:513] T 7781c00473024307987096b596da12ac P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:32.704180 21901 leader_election.cc:290] T 7781c00473024307987096b596da12ac P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
I20250109 08:57:32.705682 21514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7781c00473024307987096b596da12ac" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47" is_pre_election: true
I20250109 08:57:32.706139 21704 ts_tablet_manager.cc:1428] T c99aacce09d44f889be5480854a22760 P 787ddcc824c2490a808fc5a53452dc5e: Time spent starting tablet: real 0.044s user 0.006s sys 0.000s
I20250109 08:57:32.706483 21589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7781c00473024307987096b596da12ac" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3" is_pre_election: true
I20250109 08:57:32.706948 21704 tablet_bootstrap.cc:492] T a3aa271f2f0144eea9261e0e07ab833b P 787ddcc824c2490a808fc5a53452dc5e: Bootstrap starting.
I20250109 08:57:32.708158 21703 raft_consensus.cc:357] T a3aa271f2f0144eea9261e0e07ab833b P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:32.708830 21703 raft_consensus.cc:383] T a3aa271f2f0144eea9261e0e07ab833b P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:32.709067 21703 raft_consensus.cc:738] T a3aa271f2f0144eea9261e0e07ab833b P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bf6e26e98b14566899578a990b27d47, State: Initialized, Role: FOLLOWER
W20250109 08:57:32.709172 21627 leader_election.cc:343] T 7781c00473024307987096b596da12ac P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:32.709785 21703 consensus_queue.cc:260] T a3aa271f2f0144eea9261e0e07ab833b P 8bf6e26e98b14566899578a990b27d47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:32.713407 21703 ts_tablet_manager.cc:1428] T a3aa271f2f0144eea9261e0e07ab833b P 8bf6e26e98b14566899578a990b27d47: Time spent starting tablet: real 0.020s user 0.005s sys 0.000s
I20250109 08:57:32.714277 21703 tablet_bootstrap.cc:492] T 56bbc306275e4a409e028a7ca1dd1602 P 8bf6e26e98b14566899578a990b27d47: Bootstrap starting.
W20250109 08:57:32.714565 21626 leader_election.cc:343] T 7781c00473024307987096b596da12ac P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:32.715171 21626 leader_election.cc:304] T 7781c00473024307987096b596da12ac P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e; no voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:57:32.715912 21901 raft_consensus.cc:2743] T 7781c00473024307987096b596da12ac P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:57:32.716321 21704 tablet_bootstrap.cc:654] T a3aa271f2f0144eea9261e0e07ab833b P 787ddcc824c2490a808fc5a53452dc5e: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:32.713491 21819 raft_consensus.cc:513] T f961d3f357944118a69f905621731552 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:32.719623 21702 tablet_bootstrap.cc:492] T ab6c3fcdf226433c9e1303004f6c96b4 P a2b1a692435241ee988e0bd1661cd4c3: No bootstrap required, opened a new log
I20250109 08:57:32.720016 21702 ts_tablet_manager.cc:1397] T ab6c3fcdf226433c9e1303004f6c96b4 P a2b1a692435241ee988e0bd1661cd4c3: Time spent bootstrapping tablet: real 0.047s user 0.012s sys 0.002s
I20250109 08:57:32.720207 21819 leader_election.cc:290] T f961d3f357944118a69f905621731552 P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:32.721412 21703 tablet_bootstrap.cc:654] T 56bbc306275e4a409e028a7ca1dd1602 P 8bf6e26e98b14566899578a990b27d47: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:32.722262 21663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f961d3f357944118a69f905621731552" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e" is_pre_election: true
W20250109 08:57:32.723556 21552 leader_election.cc:343] T f961d3f357944118a69f905621731552 P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:32.723935 21702 raft_consensus.cc:357] T ab6c3fcdf226433c9e1303004f6c96b4 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:32.724622 21702 raft_consensus.cc:383] T ab6c3fcdf226433c9e1303004f6c96b4 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:32.724887 21702 raft_consensus.cc:738] T ab6c3fcdf226433c9e1303004f6c96b4 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2b1a692435241ee988e0bd1661cd4c3, State: Initialized, Role: FOLLOWER
I20250109 08:57:32.724797 21514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f961d3f357944118a69f905621731552" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47" is_pre_election: true
I20250109 08:57:32.725605 21514 raft_consensus.cc:2463] T f961d3f357944118a69f905621731552 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2b1a692435241ee988e0bd1661cd4c3 in term 0.
I20250109 08:57:32.726805 21552 leader_election.cc:304] T f961d3f357944118a69f905621731552 P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3; no voters: 787ddcc824c2490a808fc5a53452dc5e
I20250109 08:57:32.727892 21819 raft_consensus.cc:2798] T f961d3f357944118a69f905621731552 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:57:32.728276 21819 raft_consensus.cc:491] T f961d3f357944118a69f905621731552 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:57:32.729211 21819 raft_consensus.cc:3054] T f961d3f357944118a69f905621731552 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:32.731374 21702 consensus_queue.cc:260] T ab6c3fcdf226433c9e1303004f6c96b4 P a2b1a692435241ee988e0bd1661cd4c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:32.733304 21702 ts_tablet_manager.cc:1428] T ab6c3fcdf226433c9e1303004f6c96b4 P a2b1a692435241ee988e0bd1661cd4c3: Time spent starting tablet: real 0.013s user 0.005s sys 0.001s
I20250109 08:57:32.734213 21702 tablet_bootstrap.cc:492] T f5f85ad7c7c94acfaf9d550d55fce48c P a2b1a692435241ee988e0bd1661cd4c3: Bootstrap starting.
I20250109 08:57:32.738636 21819 raft_consensus.cc:513] T f961d3f357944118a69f905621731552 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:32.742095 21901 raft_consensus.cc:491] T b00671c87aac405a81c6543f4defd722 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:32.748553 21901 raft_consensus.cc:513] T b00671c87aac405a81c6543f4defd722 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:32.750286 21901 leader_election.cc:290] T b00671c87aac405a81c6543f4defd722 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
I20250109 08:57:32.741246 21702 tablet_bootstrap.cc:654] T f5f85ad7c7c94acfaf9d550d55fce48c P a2b1a692435241ee988e0bd1661cd4c3: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:32.751621 21514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b00671c87aac405a81c6543f4defd722" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47" is_pre_election: true
W20250109 08:57:32.752897 21626 leader_election.cc:343] T b00671c87aac405a81c6543f4defd722 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:32.753008 21584 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b00671c87aac405a81c6543f4defd722" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3" is_pre_election: true
I20250109 08:57:32.753667 21584 raft_consensus.cc:2463] T b00671c87aac405a81c6543f4defd722 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 0.
I20250109 08:57:32.754297 21704 tablet_bootstrap.cc:492] T a3aa271f2f0144eea9261e0e07ab833b P 787ddcc824c2490a808fc5a53452dc5e: No bootstrap required, opened a new log
I20250109 08:57:32.754789 21704 ts_tablet_manager.cc:1397] T a3aa271f2f0144eea9261e0e07ab833b P 787ddcc824c2490a808fc5a53452dc5e: Time spent bootstrapping tablet: real 0.048s user 0.010s sys 0.006s
I20250109 08:57:32.755434 21627 leader_election.cc:304] T b00671c87aac405a81c6543f4defd722 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, a2b1a692435241ee988e0bd1661cd4c3; no voters: 8bf6e26e98b14566899578a990b27d47
I20250109 08:57:32.756656 21907 raft_consensus.cc:2798] T b00671c87aac405a81c6543f4defd722 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:57:32.756994 21907 raft_consensus.cc:491] T b00671c87aac405a81c6543f4defd722 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:57:32.757297 21907 raft_consensus.cc:3054] T b00671c87aac405a81c6543f4defd722 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:32.757310 21704 raft_consensus.cc:357] T a3aa271f2f0144eea9261e0e07ab833b P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:32.761193 21704 raft_consensus.cc:383] T a3aa271f2f0144eea9261e0e07ab833b P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:32.761430 21704 raft_consensus.cc:738] T a3aa271f2f0144eea9261e0e07ab833b P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Initialized, Role: FOLLOWER
I20250109 08:57:32.762086 21704 consensus_queue.cc:260] T a3aa271f2f0144eea9261e0e07ab833b P 787ddcc824c2490a808fc5a53452dc5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:32.763084 21907 raft_consensus.cc:513] T b00671c87aac405a81c6543f4defd722 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:32.764221 21704 ts_tablet_manager.cc:1428] T a3aa271f2f0144eea9261e0e07ab833b P 787ddcc824c2490a808fc5a53452dc5e: Time spent starting tablet: real 0.009s user 0.005s sys 0.000s
I20250109 08:57:32.765188 21704 tablet_bootstrap.cc:492] T 1b582ab8d8f044d9b9038a28e958feeb P 787ddcc824c2490a808fc5a53452dc5e: Bootstrap starting.
I20250109 08:57:32.769477 21514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b00671c87aac405a81c6543f4defd722" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47"
I20250109 08:57:32.769842 21907 leader_election.cc:290] T b00671c87aac405a81c6543f4defd722 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 election: Requested vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
W20250109 08:57:32.770650 21626 leader_election.cc:343] T b00671c87aac405a81c6543f4defd722 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:32.777472 21584 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b00671c87aac405a81c6543f4defd722" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3"
I20250109 08:57:32.778163 21584 raft_consensus.cc:3054] T b00671c87aac405a81c6543f4defd722 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:32.786476 21704 tablet_bootstrap.cc:654] T 1b582ab8d8f044d9b9038a28e958feeb P 787ddcc824c2490a808fc5a53452dc5e: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:32.789083 21703 tablet_bootstrap.cc:492] T 56bbc306275e4a409e028a7ca1dd1602 P 8bf6e26e98b14566899578a990b27d47: No bootstrap required, opened a new log
I20250109 08:57:32.789553 21703 ts_tablet_manager.cc:1397] T 56bbc306275e4a409e028a7ca1dd1602 P 8bf6e26e98b14566899578a990b27d47: Time spent bootstrapping tablet: real 0.075s user 0.032s sys 0.012s
I20250109 08:57:32.792004 21703 raft_consensus.cc:357] T 56bbc306275e4a409e028a7ca1dd1602 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:32.792781 21703 raft_consensus.cc:383] T 56bbc306275e4a409e028a7ca1dd1602 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:32.793042 21703 raft_consensus.cc:738] T 56bbc306275e4a409e028a7ca1dd1602 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bf6e26e98b14566899578a990b27d47, State: Initialized, Role: FOLLOWER
I20250109 08:57:32.795018 21514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f961d3f357944118a69f905621731552" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47"
I20250109 08:57:32.795590 21514 raft_consensus.cc:3054] T f961d3f357944118a69f905621731552 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:32.798410 21704 tablet_bootstrap.cc:492] T 1b582ab8d8f044d9b9038a28e958feeb P 787ddcc824c2490a808fc5a53452dc5e: No bootstrap required, opened a new log
I20250109 08:57:32.798861 21704 ts_tablet_manager.cc:1397] T 1b582ab8d8f044d9b9038a28e958feeb P 787ddcc824c2490a808fc5a53452dc5e: Time spent bootstrapping tablet: real 0.034s user 0.014s sys 0.001s
I20250109 08:57:32.800323 21662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f961d3f357944118a69f905621731552" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e"
I20250109 08:57:32.801131 21704 raft_consensus.cc:357] T 1b582ab8d8f044d9b9038a28e958feeb P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
W20250109 08:57:32.801578 21552 leader_election.cc:343] T f961d3f357944118a69f905621731552 P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:32.801880 21704 raft_consensus.cc:383] T 1b582ab8d8f044d9b9038a28e958feeb P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:32.793677 21703 consensus_queue.cc:260] T 56bbc306275e4a409e028a7ca1dd1602 P 8bf6e26e98b14566899578a990b27d47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:32.802203 21704 raft_consensus.cc:738] T 1b582ab8d8f044d9b9038a28e958feeb P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Initialized, Role: FOLLOWER
I20250109 08:57:32.801889 21514 raft_consensus.cc:2463] T f961d3f357944118a69f905621731552 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2b1a692435241ee988e0bd1661cd4c3 in term 1.
I20250109 08:57:32.803203 21704 consensus_queue.cc:260] T 1b582ab8d8f044d9b9038a28e958feeb P 787ddcc824c2490a808fc5a53452dc5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:32.803448 21552 leader_election.cc:304] T f961d3f357944118a69f905621731552 P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3; no voters: 787ddcc824c2490a808fc5a53452dc5e
I20250109 08:57:32.804255 21709 raft_consensus.cc:2798] T f961d3f357944118a69f905621731552 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:57:32.804689 21709 raft_consensus.cc:695] T f961d3f357944118a69f905621731552 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 LEADER]: Becoming Leader. State: Replica: a2b1a692435241ee988e0bd1661cd4c3, State: Running, Role: LEADER
I20250109 08:57:32.804800 21584 raft_consensus.cc:2463] T b00671c87aac405a81c6543f4defd722 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 1.
I20250109 08:57:32.805528 21709 consensus_queue.cc:237] T f961d3f357944118a69f905621731552 P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:32.805941 21627 leader_election.cc:304] T b00671c87aac405a81c6543f4defd722 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, a2b1a692435241ee988e0bd1661cd4c3; no voters: 8bf6e26e98b14566899578a990b27d47
I20250109 08:57:32.806985 21907 raft_consensus.cc:2798] T b00671c87aac405a81c6543f4defd722 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:57:32.807423 21907 raft_consensus.cc:695] T b00671c87aac405a81c6543f4defd722 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Becoming Leader. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Running, Role: LEADER
I20250109 08:57:32.808297 21907 consensus_queue.cc:237] T b00671c87aac405a81c6543f4defd722 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:32.813465 21819 leader_election.cc:290] T f961d3f357944118a69f905621731552 P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 election: Requested vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:32.814837 21704 ts_tablet_manager.cc:1428] T 1b582ab8d8f044d9b9038a28e958feeb P 787ddcc824c2490a808fc5a53452dc5e: Time spent starting tablet: real 0.016s user 0.005s sys 0.001s
I20250109 08:57:32.815549 21781 raft_consensus.cc:491] T a3aa271f2f0144eea9261e0e07ab833b P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:32.815752 21704 tablet_bootstrap.cc:492] T ab6c3fcdf226433c9e1303004f6c96b4 P 787ddcc824c2490a808fc5a53452dc5e: Bootstrap starting.
I20250109 08:57:32.816035 21781 raft_consensus.cc:513] T a3aa271f2f0144eea9261e0e07ab833b P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:32.818536 21781 leader_election.cc:290] T a3aa271f2f0144eea9261e0e07ab833b P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:32.821710 21704 tablet_bootstrap.cc:654] T ab6c3fcdf226433c9e1303004f6c96b4 P 787ddcc824c2490a808fc5a53452dc5e: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:32.824800 21662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3aa271f2f0144eea9261e0e07ab833b" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e" is_pre_election: true
I20250109 08:57:32.825446 21662 raft_consensus.cc:2463] T a3aa271f2f0144eea9261e0e07ab833b P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8bf6e26e98b14566899578a990b27d47 in term 0.
I20250109 08:57:32.821161 21589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3aa271f2f0144eea9261e0e07ab833b" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3" is_pre_election: true
W20250109 08:57:32.827068 21478 leader_election.cc:343] T a3aa271f2f0144eea9261e0e07ab833b P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:32.827378 21703 ts_tablet_manager.cc:1428] T 56bbc306275e4a409e028a7ca1dd1602 P 8bf6e26e98b14566899578a990b27d47: Time spent starting tablet: real 0.038s user 0.006s sys 0.000s
I20250109 08:57:32.827214 21410 catalog_manager.cc:5526] T b00671c87aac405a81c6543f4defd722 P 787ddcc824c2490a808fc5a53452dc5e reported cstate change: term changed from 0 to 1, leader changed from <none> to 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195). New cstate: current_term: 1 leader_uuid: "787ddcc824c2490a808fc5a53452dc5e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: HEALTHY } } }
I20250109 08:57:32.828193 21703 tablet_bootstrap.cc:492] T b00671c87aac405a81c6543f4defd722 P 8bf6e26e98b14566899578a990b27d47: Bootstrap starting.
I20250109 08:57:32.829373 21477 leader_election.cc:304] T a3aa271f2f0144eea9261e0e07ab833b P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, 8bf6e26e98b14566899578a990b27d47; no voters: a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:57:32.830092 21742 raft_consensus.cc:2798] T a3aa271f2f0144eea9261e0e07ab833b P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:57:32.830420 21742 raft_consensus.cc:491] T a3aa271f2f0144eea9261e0e07ab833b P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:57:32.830734 21742 raft_consensus.cc:3054] T a3aa271f2f0144eea9261e0e07ab833b P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:32.834900 21702 tablet_bootstrap.cc:492] T f5f85ad7c7c94acfaf9d550d55fce48c P a2b1a692435241ee988e0bd1661cd4c3: No bootstrap required, opened a new log
I20250109 08:57:32.835448 21702 ts_tablet_manager.cc:1397] T f5f85ad7c7c94acfaf9d550d55fce48c P a2b1a692435241ee988e0bd1661cd4c3: Time spent bootstrapping tablet: real 0.101s user 0.026s sys 0.021s
I20250109 08:57:32.840463 21742 raft_consensus.cc:513] T a3aa271f2f0144eea9261e0e07ab833b P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:32.844715 21702 raft_consensus.cc:357] T f5f85ad7c7c94acfaf9d550d55fce48c P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:32.845518 21702 raft_consensus.cc:383] T f5f85ad7c7c94acfaf9d550d55fce48c P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:32.845851 21702 raft_consensus.cc:738] T f5f85ad7c7c94acfaf9d550d55fce48c P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2b1a692435241ee988e0bd1661cd4c3, State: Initialized, Role: FOLLOWER
I20250109 08:57:32.846014 21589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3aa271f2f0144eea9261e0e07ab833b" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3"
I20250109 08:57:32.846580 21702 consensus_queue.cc:260] T f5f85ad7c7c94acfaf9d550d55fce48c P a2b1a692435241ee988e0bd1661cd4c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:32.847076 21759 raft_consensus.cc:491] T ab6c3fcdf226433c9e1303004f6c96b4 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:32.847508 21759 raft_consensus.cc:513] T ab6c3fcdf226433c9e1303004f6c96b4 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:32.849471 21759 leader_election.cc:290] T ab6c3fcdf226433c9e1303004f6c96b4 P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:32.856127 21703 tablet_bootstrap.cc:654] T b00671c87aac405a81c6543f4defd722 P 8bf6e26e98b14566899578a990b27d47: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:32.855420 21412 catalog_manager.cc:5526] T f961d3f357944118a69f905621731552 P a2b1a692435241ee988e0bd1661cd4c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194). New cstate: current_term: 1 leader_uuid: "a2b1a692435241ee988e0bd1661cd4c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: UNKNOWN } } }
W20250109 08:57:32.857161 21478 leader_election.cc:343] T a3aa271f2f0144eea9261e0e07ab833b P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:32.860831 21661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab6c3fcdf226433c9e1303004f6c96b4" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e" is_pre_election: true
I20250109 08:57:32.860860 21512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab6c3fcdf226433c9e1303004f6c96b4" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47" is_pre_election: true
I20250109 08:57:32.864082 21758 consensus_queue.cc:1035] T c18c1d46a6ca47598d7b161ed70a4e7a P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20250109 08:57:32.866236 21552 leader_election.cc:343] T ab6c3fcdf226433c9e1303004f6c96b4 P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Illegal state: must be running to vote when last-logged opid is not known
W20250109 08:57:32.866936 21552 leader_election.cc:343] T ab6c3fcdf226433c9e1303004f6c96b4 P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:32.867336 21552 leader_election.cc:304] T ab6c3fcdf226433c9e1303004f6c96b4 P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2b1a692435241ee988e0bd1661cd4c3; no voters: 787ddcc824c2490a808fc5a53452dc5e, 8bf6e26e98b14566899578a990b27d47
I20250109 08:57:32.869266 21819 raft_consensus.cc:2743] T ab6c3fcdf226433c9e1303004f6c96b4 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:57:32.879164 21710 consensus_queue.cc:1035] T c18c1d46a6ca47598d7b161ed70a4e7a P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250109 08:57:32.885161 21742 leader_election.cc:290] T a3aa271f2f0144eea9261e0e07ab833b P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 election: Requested vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:32.888686 21704 tablet_bootstrap.cc:492] T ab6c3fcdf226433c9e1303004f6c96b4 P 787ddcc824c2490a808fc5a53452dc5e: No bootstrap required, opened a new log
I20250109 08:57:32.889153 21704 ts_tablet_manager.cc:1397] T ab6c3fcdf226433c9e1303004f6c96b4 P 787ddcc824c2490a808fc5a53452dc5e: Time spent bootstrapping tablet: real 0.074s user 0.016s sys 0.000s
I20250109 08:57:32.891326 21702 ts_tablet_manager.cc:1428] T f5f85ad7c7c94acfaf9d550d55fce48c P a2b1a692435241ee988e0bd1661cd4c3: Time spent starting tablet: real 0.056s user 0.006s sys 0.010s
I20250109 08:57:32.891413 21704 raft_consensus.cc:357] T ab6c3fcdf226433c9e1303004f6c96b4 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:32.892158 21704 raft_consensus.cc:383] T ab6c3fcdf226433c9e1303004f6c96b4 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:32.892278 21702 tablet_bootstrap.cc:492] T 64c667f887ea41b4a484ee565b21fa16 P a2b1a692435241ee988e0bd1661cd4c3: Bootstrap starting.
I20250109 08:57:32.893973 21704 raft_consensus.cc:738] T ab6c3fcdf226433c9e1303004f6c96b4 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Initialized, Role: FOLLOWER
I20250109 08:57:32.895249 21704 consensus_queue.cc:260] T ab6c3fcdf226433c9e1303004f6c96b4 P 787ddcc824c2490a808fc5a53452dc5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:32.896960 21661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3aa271f2f0144eea9261e0e07ab833b" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e"
I20250109 08:57:32.897569 21661 raft_consensus.cc:3054] T a3aa271f2f0144eea9261e0e07ab833b P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:32.899629 21704 ts_tablet_manager.cc:1428] T ab6c3fcdf226433c9e1303004f6c96b4 P 787ddcc824c2490a808fc5a53452dc5e: Time spent starting tablet: real 0.010s user 0.006s sys 0.000s
I20250109 08:57:32.899842 21702 tablet_bootstrap.cc:654] T 64c667f887ea41b4a484ee565b21fa16 P a2b1a692435241ee988e0bd1661cd4c3: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:32.906271 21704 tablet_bootstrap.cc:492] T 8c0459d2eee64f0a83bd05ad5974f506 P 787ddcc824c2490a808fc5a53452dc5e: Bootstrap starting.
I20250109 08:57:32.908121 21781 raft_consensus.cc:491] T d015a10f95eb4f19b6d7c53cb201fa80 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:32.908617 21781 raft_consensus.cc:513] T d015a10f95eb4f19b6d7c53cb201fa80 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:32.910451 21781 leader_election.cc:290] T d015a10f95eb4f19b6d7c53cb201fa80 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:32.913225 21584 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d015a10f95eb4f19b6d7c53cb201fa80" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3" is_pre_election: true
W20250109 08:57:32.915318 21478 leader_election.cc:343] T d015a10f95eb4f19b6d7c53cb201fa80 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:32.929283 21662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d015a10f95eb4f19b6d7c53cb201fa80" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e" is_pre_election: true
I20250109 08:57:32.938696 21704 tablet_bootstrap.cc:654] T 8c0459d2eee64f0a83bd05ad5974f506 P 787ddcc824c2490a808fc5a53452dc5e: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:32.938591 21661 raft_consensus.cc:2463] T a3aa271f2f0144eea9261e0e07ab833b P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8bf6e26e98b14566899578a990b27d47 in term 1.
I20250109 08:57:32.945693 21703 tablet_bootstrap.cc:492] T b00671c87aac405a81c6543f4defd722 P 8bf6e26e98b14566899578a990b27d47: No bootstrap required, opened a new log
W20250109 08:57:32.946406 21477 leader_election.cc:343] T d015a10f95eb4f19b6d7c53cb201fa80 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:32.947458 21477 leader_election.cc:304] T d015a10f95eb4f19b6d7c53cb201fa80 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8bf6e26e98b14566899578a990b27d47; no voters: 787ddcc824c2490a808fc5a53452dc5e, a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:57:32.949025 21477 leader_election.cc:304] T a3aa271f2f0144eea9261e0e07ab833b P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, 8bf6e26e98b14566899578a990b27d47; no voters: a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:57:32.949883 21781 raft_consensus.cc:2743] T d015a10f95eb4f19b6d7c53cb201fa80 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:57:32.950505 21781 raft_consensus.cc:2798] T a3aa271f2f0144eea9261e0e07ab833b P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:57:32.951157 21781 raft_consensus.cc:695] T a3aa271f2f0144eea9261e0e07ab833b P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Becoming Leader. State: Replica: 8bf6e26e98b14566899578a990b27d47, State: Running, Role: LEADER
I20250109 08:57:32.950275 21989 consensus_queue.cc:1035] T e03846e461974d0d8944c344e715b86a P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250109 08:57:32.951985 21781 consensus_queue.cc:237] T a3aa271f2f0144eea9261e0e07ab833b P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:32.956231 21703 ts_tablet_manager.cc:1397] T b00671c87aac405a81c6543f4defd722 P 8bf6e26e98b14566899578a990b27d47: Time spent bootstrapping tablet: real 0.128s user 0.015s sys 0.023s
I20250109 08:57:32.985530 21412 catalog_manager.cc:5526] T a3aa271f2f0144eea9261e0e07ab833b P 8bf6e26e98b14566899578a990b27d47 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8bf6e26e98b14566899578a990b27d47 (127.19.226.193). New cstate: current_term: 1 leader_uuid: "8bf6e26e98b14566899578a990b27d47" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: UNKNOWN } } }
I20250109 08:57:32.986301 21819 consensus_queue.cc:1035] T e03846e461974d0d8944c344e715b86a P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:57:32.988143 21732 consensus_queue.cc:1035] T eb4516b9a16041c2adf34be520768f3f P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20250109 08:57:33.001116 21702 tablet_bootstrap.cc:492] T 64c667f887ea41b4a484ee565b21fa16 P a2b1a692435241ee988e0bd1661cd4c3: No bootstrap required, opened a new log
I20250109 08:57:33.001611 21702 ts_tablet_manager.cc:1397] T 64c667f887ea41b4a484ee565b21fa16 P a2b1a692435241ee988e0bd1661cd4c3: Time spent bootstrapping tablet: real 0.109s user 0.035s sys 0.049s
I20250109 08:57:33.002918 21703 raft_consensus.cc:357] T b00671c87aac405a81c6543f4defd722 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:33.004020 21702 raft_consensus.cc:357] T 64c667f887ea41b4a484ee565b21fa16 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:33.004695 21702 raft_consensus.cc:383] T 64c667f887ea41b4a484ee565b21fa16 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:33.004964 21702 raft_consensus.cc:738] T 64c667f887ea41b4a484ee565b21fa16 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2b1a692435241ee988e0bd1661cd4c3, State: Initialized, Role: FOLLOWER
I20250109 08:57:33.005120 21703 raft_consensus.cc:383] T b00671c87aac405a81c6543f4defd722 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:33.005791 21703 raft_consensus.cc:738] T b00671c87aac405a81c6543f4defd722 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bf6e26e98b14566899578a990b27d47, State: Initialized, Role: FOLLOWER
I20250109 08:57:33.005659 21702 consensus_queue.cc:260] T 64c667f887ea41b4a484ee565b21fa16 P a2b1a692435241ee988e0bd1661cd4c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:33.007516 21702 ts_tablet_manager.cc:1428] T 64c667f887ea41b4a484ee565b21fa16 P a2b1a692435241ee988e0bd1661cd4c3: Time spent starting tablet: real 0.006s user 0.004s sys 0.002s
I20250109 08:57:33.007078 21703 consensus_queue.cc:260] T b00671c87aac405a81c6543f4defd722 P 8bf6e26e98b14566899578a990b27d47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:33.008458 21702 tablet_bootstrap.cc:492] T 1d3d5d2a2c88417a97c9d57262a356dd P a2b1a692435241ee988e0bd1661cd4c3: Bootstrap starting.
I20250109 08:57:33.010391 21703 ts_tablet_manager.cc:1428] T b00671c87aac405a81c6543f4defd722 P 8bf6e26e98b14566899578a990b27d47: Time spent starting tablet: real 0.051s user 0.006s sys 0.000s
I20250109 08:57:33.013028 21703 tablet_bootstrap.cc:492] T ab6c3fcdf226433c9e1303004f6c96b4 P 8bf6e26e98b14566899578a990b27d47: Bootstrap starting.
I20250109 08:57:33.014743 21907 raft_consensus.cc:491] T ab6c3fcdf226433c9e1303004f6c96b4 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:33.016767 21907 raft_consensus.cc:513] T ab6c3fcdf226433c9e1303004f6c96b4 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:33.019236 21907 leader_election.cc:290] T ab6c3fcdf226433c9e1303004f6c96b4 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
I20250109 08:57:33.015049 21702 tablet_bootstrap.cc:654] T 1d3d5d2a2c88417a97c9d57262a356dd P a2b1a692435241ee988e0bd1661cd4c3: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:33.015591 21742 raft_consensus.cc:491] T 56bbc306275e4a409e028a7ca1dd1602 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:33.020653 21742 raft_consensus.cc:513] T 56bbc306275e4a409e028a7ca1dd1602 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:33.020952 21589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab6c3fcdf226433c9e1303004f6c96b4" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3" is_pre_election: true
I20250109 08:57:33.021839 21589 raft_consensus.cc:2463] T ab6c3fcdf226433c9e1303004f6c96b4 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 0.
I20250109 08:57:33.029685 21627 leader_election.cc:304] T ab6c3fcdf226433c9e1303004f6c96b4 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, a2b1a692435241ee988e0bd1661cd4c3; no voters:
I20250109 08:57:33.041112 21661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56bbc306275e4a409e028a7ca1dd1602" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e" is_pre_election: true
I20250109 08:57:33.039819 21589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56bbc306275e4a409e028a7ca1dd1602" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3" is_pre_election: true
W20250109 08:57:33.044205 21478 leader_election.cc:343] T 56bbc306275e4a409e028a7ca1dd1602 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:33.028964 21742 leader_election.cc:290] T 56bbc306275e4a409e028a7ca1dd1602 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
W20250109 08:57:33.045905 21477 leader_election.cc:343] T 56bbc306275e4a409e028a7ca1dd1602 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:33.046347 21477 leader_election.cc:304] T 56bbc306275e4a409e028a7ca1dd1602 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8bf6e26e98b14566899578a990b27d47; no voters: 787ddcc824c2490a808fc5a53452dc5e, a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:57:33.047258 21742 raft_consensus.cc:2743] T 56bbc306275e4a409e028a7ca1dd1602 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:57:33.049245 21514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab6c3fcdf226433c9e1303004f6c96b4" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47" is_pre_election: true
W20250109 08:57:33.052047 21626 leader_election.cc:343] T ab6c3fcdf226433c9e1303004f6c96b4 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:33.038827 21907 raft_consensus.cc:2798] T ab6c3fcdf226433c9e1303004f6c96b4 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:57:33.054145 21907 raft_consensus.cc:491] T ab6c3fcdf226433c9e1303004f6c96b4 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:57:33.054512 21907 raft_consensus.cc:3054] T ab6c3fcdf226433c9e1303004f6c96b4 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:33.059495 21732 raft_consensus.cc:491] T c99aacce09d44f889be5480854a22760 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:33.060002 21732 raft_consensus.cc:513] T c99aacce09d44f889be5480854a22760 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:33.063068 21732 leader_election.cc:290] T c99aacce09d44f889be5480854a22760 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
I20250109 08:57:33.064800 21589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c99aacce09d44f889be5480854a22760" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3" is_pre_election: true
I20250109 08:57:33.066102 21514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c99aacce09d44f889be5480854a22760" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47" is_pre_election: true
I20250109 08:57:33.070869 21626 leader_election.cc:304] T c99aacce09d44f889be5480854a22760 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e; no voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:57:33.072990 21901 raft_consensus.cc:2743] T c99aacce09d44f889be5480854a22760 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:57:33.075011 21703 tablet_bootstrap.cc:654] T ab6c3fcdf226433c9e1303004f6c96b4 P 8bf6e26e98b14566899578a990b27d47: Neither blocks nor log segments found. Creating new log.
W20250109 08:57:33.081543 21627 consensus_peers.cc:487] T eb4516b9a16041c2adf34be520768f3f P 787ddcc824c2490a808fc5a53452dc5e -> Peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Couldn't send request to peer a2b1a692435241ee988e0bd1661cd4c3. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:57:33.089105 21704 tablet_bootstrap.cc:492] T 8c0459d2eee64f0a83bd05ad5974f506 P 787ddcc824c2490a808fc5a53452dc5e: No bootstrap required, opened a new log
I20250109 08:57:33.089617 21704 ts_tablet_manager.cc:1397] T 8c0459d2eee64f0a83bd05ad5974f506 P 787ddcc824c2490a808fc5a53452dc5e: Time spent bootstrapping tablet: real 0.189s user 0.040s sys 0.016s
I20250109 08:57:33.096740 21704 raft_consensus.cc:357] T 8c0459d2eee64f0a83bd05ad5974f506 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:33.097519 21704 raft_consensus.cc:383] T 8c0459d2eee64f0a83bd05ad5974f506 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:33.097791 21704 raft_consensus.cc:738] T 8c0459d2eee64f0a83bd05ad5974f506 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Initialized, Role: FOLLOWER
I20250109 08:57:33.098407 21704 consensus_queue.cc:260] T 8c0459d2eee64f0a83bd05ad5974f506 P 787ddcc824c2490a808fc5a53452dc5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:33.102756 21663 raft_consensus.cc:3054] T c99aacce09d44f889be5480854a22760 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:33.105115 21704 ts_tablet_manager.cc:1428] T 8c0459d2eee64f0a83bd05ad5974f506 P 787ddcc824c2490a808fc5a53452dc5e: Time spent starting tablet: real 0.015s user 0.004s sys 0.004s
I20250109 08:57:33.109381 21907 raft_consensus.cc:513] T ab6c3fcdf226433c9e1303004f6c96b4 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:33.110308 21704 tablet_bootstrap.cc:492] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 787ddcc824c2490a808fc5a53452dc5e: Bootstrap starting.
I20250109 08:57:33.116936 21702 tablet_bootstrap.cc:492] T 1d3d5d2a2c88417a97c9d57262a356dd P a2b1a692435241ee988e0bd1661cd4c3: No bootstrap required, opened a new log
I20250109 08:57:33.117462 21702 ts_tablet_manager.cc:1397] T 1d3d5d2a2c88417a97c9d57262a356dd P a2b1a692435241ee988e0bd1661cd4c3: Time spent bootstrapping tablet: real 0.109s user 0.030s sys 0.044s
I20250109 08:57:33.120131 21702 raft_consensus.cc:357] T 1d3d5d2a2c88417a97c9d57262a356dd P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:33.120924 21702 raft_consensus.cc:383] T 1d3d5d2a2c88417a97c9d57262a356dd P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:33.121202 21702 raft_consensus.cc:738] T 1d3d5d2a2c88417a97c9d57262a356dd P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2b1a692435241ee988e0bd1661cd4c3, State: Initialized, Role: FOLLOWER
I20250109 08:57:33.124166 21703 tablet_bootstrap.cc:492] T ab6c3fcdf226433c9e1303004f6c96b4 P 8bf6e26e98b14566899578a990b27d47: No bootstrap required, opened a new log
I20250109 08:57:33.124663 21703 ts_tablet_manager.cc:1397] T ab6c3fcdf226433c9e1303004f6c96b4 P 8bf6e26e98b14566899578a990b27d47: Time spent bootstrapping tablet: real 0.112s user 0.010s sys 0.004s
I20250109 08:57:33.126713 21704 tablet_bootstrap.cc:654] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 787ddcc824c2490a808fc5a53452dc5e: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:33.126994 21703 raft_consensus.cc:357] T ab6c3fcdf226433c9e1303004f6c96b4 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:33.122146 21702 consensus_queue.cc:260] T 1d3d5d2a2c88417a97c9d57262a356dd P a2b1a692435241ee988e0bd1661cd4c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:33.128594 21703 raft_consensus.cc:383] T ab6c3fcdf226433c9e1303004f6c96b4 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:33.128943 21703 raft_consensus.cc:738] T ab6c3fcdf226433c9e1303004f6c96b4 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bf6e26e98b14566899578a990b27d47, State: Initialized, Role: FOLLOWER
I20250109 08:57:33.127991 21907 leader_election.cc:290] T ab6c3fcdf226433c9e1303004f6c96b4 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 election: Requested vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
I20250109 08:57:33.129784 21703 consensus_queue.cc:260] T ab6c3fcdf226433c9e1303004f6c96b4 P 8bf6e26e98b14566899578a990b27d47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:33.130280 21512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab6c3fcdf226433c9e1303004f6c96b4" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47"
I20250109 08:57:33.130630 21702 ts_tablet_manager.cc:1428] T 1d3d5d2a2c88417a97c9d57262a356dd P a2b1a692435241ee988e0bd1661cd4c3: Time spent starting tablet: real 0.013s user 0.007s sys 0.000s
I20250109 08:57:33.128957 21589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab6c3fcdf226433c9e1303004f6c96b4" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3"
I20250109 08:57:33.130842 21512 raft_consensus.cc:3054] T ab6c3fcdf226433c9e1303004f6c96b4 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:33.131793 21702 tablet_bootstrap.cc:492] T 7781c00473024307987096b596da12ac P a2b1a692435241ee988e0bd1661cd4c3: Bootstrap starting.
I20250109 08:57:33.132615 21703 ts_tablet_manager.cc:1428] T ab6c3fcdf226433c9e1303004f6c96b4 P 8bf6e26e98b14566899578a990b27d47: Time spent starting tablet: real 0.008s user 0.003s sys 0.004s
I20250109 08:57:33.136137 21703 tablet_bootstrap.cc:492] T 1b582ab8d8f044d9b9038a28e958feeb P 8bf6e26e98b14566899578a990b27d47: Bootstrap starting.
I20250109 08:57:33.137778 21702 tablet_bootstrap.cc:654] T 7781c00473024307987096b596da12ac P a2b1a692435241ee988e0bd1661cd4c3: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:33.131400 21589 raft_consensus.cc:3054] T ab6c3fcdf226433c9e1303004f6c96b4 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:33.145274 21703 tablet_bootstrap.cc:654] T 1b582ab8d8f044d9b9038a28e958feeb P 8bf6e26e98b14566899578a990b27d47: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:33.147173 21589 raft_consensus.cc:2463] T ab6c3fcdf226433c9e1303004f6c96b4 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 1.
I20250109 08:57:33.149420 21627 leader_election.cc:304] T ab6c3fcdf226433c9e1303004f6c96b4 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, a2b1a692435241ee988e0bd1661cd4c3; no voters:
I20250109 08:57:33.150152 21907 raft_consensus.cc:2798] T ab6c3fcdf226433c9e1303004f6c96b4 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:57:33.150595 21907 raft_consensus.cc:695] T ab6c3fcdf226433c9e1303004f6c96b4 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Becoming Leader. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Running, Role: LEADER
I20250109 08:57:33.151376 21907 consensus_queue.cc:237] T ab6c3fcdf226433c9e1303004f6c96b4 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:33.162357 21512 raft_consensus.cc:2463] T ab6c3fcdf226433c9e1303004f6c96b4 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 1.
I20250109 08:57:33.165225 21704 tablet_bootstrap.cc:492] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 787ddcc824c2490a808fc5a53452dc5e: No bootstrap required, opened a new log
I20250109 08:57:33.165678 21704 ts_tablet_manager.cc:1397] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 787ddcc824c2490a808fc5a53452dc5e: Time spent bootstrapping tablet: real 0.060s user 0.011s sys 0.008s
I20250109 08:57:33.168313 21704 raft_consensus.cc:357] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:33.169035 21704 raft_consensus.cc:383] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:33.169292 21704 raft_consensus.cc:738] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Initialized, Role: FOLLOWER
I20250109 08:57:33.169889 21704 consensus_queue.cc:260] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 787ddcc824c2490a808fc5a53452dc5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:33.172075 21704 ts_tablet_manager.cc:1428] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 787ddcc824c2490a808fc5a53452dc5e: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20250109 08:57:33.177024 21704 tablet_bootstrap.cc:492] T d52f79ecf037457cbcc09d5fa43456cf P 787ddcc824c2490a808fc5a53452dc5e: Bootstrap starting.
I20250109 08:57:33.177919 21410 catalog_manager.cc:5526] T ab6c3fcdf226433c9e1303004f6c96b4 P 787ddcc824c2490a808fc5a53452dc5e reported cstate change: term changed from 0 to 1, leader changed from <none> to 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195). New cstate: current_term: 1 leader_uuid: "787ddcc824c2490a808fc5a53452dc5e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: HEALTHY } } }
I20250109 08:57:33.181236 21989 raft_consensus.cc:491] T f5f85ad7c7c94acfaf9d550d55fce48c P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:33.185207 21989 raft_consensus.cc:513] T f5f85ad7c7c94acfaf9d550d55fce48c P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:33.190984 21704 tablet_bootstrap.cc:654] T d52f79ecf037457cbcc09d5fa43456cf P 787ddcc824c2490a808fc5a53452dc5e: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:33.191670 21989 leader_election.cc:290] T f5f85ad7c7c94acfaf9d550d55fce48c P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:33.195246 21703 tablet_bootstrap.cc:492] T 1b582ab8d8f044d9b9038a28e958feeb P 8bf6e26e98b14566899578a990b27d47: No bootstrap required, opened a new log
I20250109 08:57:33.195880 21702 tablet_bootstrap.cc:492] T 7781c00473024307987096b596da12ac P a2b1a692435241ee988e0bd1661cd4c3: No bootstrap required, opened a new log
I20250109 08:57:33.196664 21702 ts_tablet_manager.cc:1397] T 7781c00473024307987096b596da12ac P a2b1a692435241ee988e0bd1661cd4c3: Time spent bootstrapping tablet: real 0.065s user 0.022s sys 0.010s
I20250109 08:57:33.199167 21702 raft_consensus.cc:357] T 7781c00473024307987096b596da12ac P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:33.196321 21703 ts_tablet_manager.cc:1397] T 1b582ab8d8f044d9b9038a28e958feeb P 8bf6e26e98b14566899578a990b27d47: Time spent bootstrapping tablet: real 0.060s user 0.012s sys 0.019s
I20250109 08:57:33.199440 21512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5f85ad7c7c94acfaf9d550d55fce48c" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47" is_pre_election: true
I20250109 08:57:33.200100 21512 raft_consensus.cc:2463] T f5f85ad7c7c94acfaf9d550d55fce48c P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2b1a692435241ee988e0bd1661cd4c3 in term 0.
I20250109 08:57:33.200472 21702 raft_consensus.cc:383] T 7781c00473024307987096b596da12ac P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:33.200911 21702 raft_consensus.cc:738] T 7781c00473024307987096b596da12ac P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2b1a692435241ee988e0bd1661cd4c3, State: Initialized, Role: FOLLOWER
I20250109 08:57:33.201530 21702 consensus_queue.cc:260] T 7781c00473024307987096b596da12ac P a2b1a692435241ee988e0bd1661cd4c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:33.202116 21663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5f85ad7c7c94acfaf9d550d55fce48c" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e" is_pre_election: true
I20250109 08:57:33.203599 21552 leader_election.cc:304] T f5f85ad7c7c94acfaf9d550d55fce48c P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3; no voters:
I20250109 08:57:33.204388 21702 ts_tablet_manager.cc:1428] T 7781c00473024307987096b596da12ac P a2b1a692435241ee988e0bd1661cd4c3: Time spent starting tablet: real 0.007s user 0.003s sys 0.003s
I20250109 08:57:33.204530 21989 raft_consensus.cc:2798] T f5f85ad7c7c94acfaf9d550d55fce48c P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:57:33.204854 21989 raft_consensus.cc:491] T f5f85ad7c7c94acfaf9d550d55fce48c P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:57:33.205154 21989 raft_consensus.cc:3054] T f5f85ad7c7c94acfaf9d550d55fce48c P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Advancing to term 1
W20250109 08:57:33.208966 21552 leader_election.cc:343] T f5f85ad7c7c94acfaf9d550d55fce48c P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:33.209548 21704 tablet_bootstrap.cc:492] T d52f79ecf037457cbcc09d5fa43456cf P 787ddcc824c2490a808fc5a53452dc5e: No bootstrap required, opened a new log
I20250109 08:57:33.210032 21704 ts_tablet_manager.cc:1397] T d52f79ecf037457cbcc09d5fa43456cf P 787ddcc824c2490a808fc5a53452dc5e: Time spent bootstrapping tablet: real 0.033s user 0.011s sys 0.000s
I20250109 08:57:33.211088 21989 raft_consensus.cc:513] T f5f85ad7c7c94acfaf9d550d55fce48c P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:33.213228 21702 tablet_bootstrap.cc:492] T 126a163a89c448be89ccb9490e9f0168 P a2b1a692435241ee988e0bd1661cd4c3: Bootstrap starting.
I20250109 08:57:33.217972 21781 raft_consensus.cc:491] T b00671c87aac405a81c6543f4defd722 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:33.218698 21989 leader_election.cc:290] T f5f85ad7c7c94acfaf9d550d55fce48c P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 election: Requested vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:33.218452 21781 raft_consensus.cc:513] T b00671c87aac405a81c6543f4defd722 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:33.220469 21514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5f85ad7c7c94acfaf9d550d55fce48c" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47"
I20250109 08:57:33.221006 21514 raft_consensus.cc:3054] T f5f85ad7c7c94acfaf9d550d55fce48c P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:33.222860 21702 tablet_bootstrap.cc:654] T 126a163a89c448be89ccb9490e9f0168 P a2b1a692435241ee988e0bd1661cd4c3: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:33.202262 21703 raft_consensus.cc:357] T 1b582ab8d8f044d9b9038a28e958feeb P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:33.223255 21703 raft_consensus.cc:383] T 1b582ab8d8f044d9b9038a28e958feeb P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:33.223518 21703 raft_consensus.cc:738] T 1b582ab8d8f044d9b9038a28e958feeb P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bf6e26e98b14566899578a990b27d47, State: Initialized, Role: FOLLOWER
I20250109 08:57:33.224084 21703 consensus_queue.cc:260] T 1b582ab8d8f044d9b9038a28e958feeb P 8bf6e26e98b14566899578a990b27d47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:33.225306 21704 raft_consensus.cc:357] T d52f79ecf037457cbcc09d5fa43456cf P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:33.226008 21704 raft_consensus.cc:383] T d52f79ecf037457cbcc09d5fa43456cf P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:33.226246 21704 raft_consensus.cc:738] T d52f79ecf037457cbcc09d5fa43456cf P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Initialized, Role: FOLLOWER
I20250109 08:57:33.226529 21703 ts_tablet_manager.cc:1428] T 1b582ab8d8f044d9b9038a28e958feeb P 8bf6e26e98b14566899578a990b27d47: Time spent starting tablet: real 0.027s user 0.005s sys 0.002s
I20250109 08:57:33.226891 21704 consensus_queue.cc:260] T d52f79ecf037457cbcc09d5fa43456cf P 787ddcc824c2490a808fc5a53452dc5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:33.227378 21703 tablet_bootstrap.cc:492] T 1d3d5d2a2c88417a97c9d57262a356dd P 8bf6e26e98b14566899578a990b27d47: Bootstrap starting.
I20250109 08:57:33.228065 21584 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b00671c87aac405a81c6543f4defd722" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3" is_pre_election: true
I20250109 08:57:33.228804 21584 raft_consensus.cc:2388] T b00671c87aac405a81c6543f4defd722 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8bf6e26e98b14566899578a990b27d47 in current term 1: Already voted for candidate 787ddcc824c2490a808fc5a53452dc5e in this term.
I20250109 08:57:33.228998 21704 ts_tablet_manager.cc:1428] T d52f79ecf037457cbcc09d5fa43456cf P 787ddcc824c2490a808fc5a53452dc5e: Time spent starting tablet: real 0.019s user 0.005s sys 0.000s
I20250109 08:57:33.226182 21781 leader_election.cc:290] T b00671c87aac405a81c6543f4defd722 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:33.228826 21661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5f85ad7c7c94acfaf9d550d55fce48c" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e"
I20250109 08:57:33.229811 21704 tablet_bootstrap.cc:492] T 126a163a89c448be89ccb9490e9f0168 P 787ddcc824c2490a808fc5a53452dc5e: Bootstrap starting.
W20250109 08:57:33.230806 21552 leader_election.cc:343] T f5f85ad7c7c94acfaf9d550d55fce48c P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:33.229154 21514 raft_consensus.cc:2463] T f5f85ad7c7c94acfaf9d550d55fce48c P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2b1a692435241ee988e0bd1661cd4c3 in term 1.
I20250109 08:57:33.233340 21552 leader_election.cc:304] T f5f85ad7c7c94acfaf9d550d55fce48c P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3; no voters: 787ddcc824c2490a808fc5a53452dc5e
I20250109 08:57:33.234333 21989 raft_consensus.cc:2798] T f5f85ad7c7c94acfaf9d550d55fce48c P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:57:33.234788 21989 raft_consensus.cc:695] T f5f85ad7c7c94acfaf9d550d55fce48c P a2b1a692435241ee988e0bd1661cd4c3 [term 1 LEADER]: Becoming Leader. State: Replica: a2b1a692435241ee988e0bd1661cd4c3, State: Running, Role: LEADER
I20250109 08:57:33.229154 21663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b00671c87aac405a81c6543f4defd722" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e" is_pre_election: true
I20250109 08:57:33.235822 21989 consensus_queue.cc:237] T f5f85ad7c7c94acfaf9d550d55fce48c P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:33.236328 21477 leader_election.cc:304] T b00671c87aac405a81c6543f4defd722 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8bf6e26e98b14566899578a990b27d47; no voters: 787ddcc824c2490a808fc5a53452dc5e, a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:57:33.237439 21781 raft_consensus.cc:3054] T b00671c87aac405a81c6543f4defd722 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:33.242517 21702 tablet_bootstrap.cc:492] T 126a163a89c448be89ccb9490e9f0168 P a2b1a692435241ee988e0bd1661cd4c3: No bootstrap required, opened a new log
I20250109 08:57:33.243502 21702 ts_tablet_manager.cc:1397] T 126a163a89c448be89ccb9490e9f0168 P a2b1a692435241ee988e0bd1661cd4c3: Time spent bootstrapping tablet: real 0.030s user 0.007s sys 0.009s
I20250109 08:57:33.244385 21742 raft_consensus.cc:491] T 126a163a89c448be89ccb9490e9f0168 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:33.244879 21742 raft_consensus.cc:513] T 126a163a89c448be89ccb9490e9f0168 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:33.246786 21703 tablet_bootstrap.cc:654] T 1d3d5d2a2c88417a97c9d57262a356dd P 8bf6e26e98b14566899578a990b27d47: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:33.247306 21742 leader_election.cc:290] T 126a163a89c448be89ccb9490e9f0168 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:33.248649 21584 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "126a163a89c448be89ccb9490e9f0168" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3" is_pre_election: true
I20250109 08:57:33.249624 21663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "126a163a89c448be89ccb9490e9f0168" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e" is_pre_election: true
I20250109 08:57:33.242923 21781 raft_consensus.cc:2743] T b00671c87aac405a81c6543f4defd722 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20250109 08:57:33.252550 21478 leader_election.cc:343] T 126a163a89c448be89ccb9490e9f0168 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Illegal state: must be running to vote when last-logged opid is not known
W20250109 08:57:33.252954 21477 leader_election.cc:343] T 126a163a89c448be89ccb9490e9f0168 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:33.253612 21477 leader_election.cc:304] T 126a163a89c448be89ccb9490e9f0168 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8bf6e26e98b14566899578a990b27d47; no voters: 787ddcc824c2490a808fc5a53452dc5e, a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:57:33.243039 21704 tablet_bootstrap.cc:654] T 126a163a89c448be89ccb9490e9f0168 P 787ddcc824c2490a808fc5a53452dc5e: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:33.254462 21781 raft_consensus.cc:2743] T 126a163a89c448be89ccb9490e9f0168 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:57:33.254403 21702 raft_consensus.cc:357] T 126a163a89c448be89ccb9490e9f0168 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:33.257648 21702 raft_consensus.cc:383] T 126a163a89c448be89ccb9490e9f0168 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:33.257911 21702 raft_consensus.cc:738] T 126a163a89c448be89ccb9490e9f0168 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2b1a692435241ee988e0bd1661cd4c3, State: Initialized, Role: FOLLOWER
I20250109 08:57:33.258474 21702 consensus_queue.cc:260] T 126a163a89c448be89ccb9490e9f0168 P a2b1a692435241ee988e0bd1661cd4c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:33.259838 21732 consensus_queue.cc:1035] T b00671c87aac405a81c6543f4defd722 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:57:33.260615 21702 ts_tablet_manager.cc:1428] T 126a163a89c448be89ccb9490e9f0168 P a2b1a692435241ee988e0bd1661cd4c3: Time spent starting tablet: real 0.017s user 0.006s sys 0.000s
I20250109 08:57:33.261699 21702 tablet_bootstrap.cc:492] T 8c0459d2eee64f0a83bd05ad5974f506 P a2b1a692435241ee988e0bd1661cd4c3: Bootstrap starting.
I20250109 08:57:33.269565 21732 raft_consensus.cc:491] T 1b582ab8d8f044d9b9038a28e958feeb P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:33.270044 21732 raft_consensus.cc:513] T 1b582ab8d8f044d9b9038a28e958feeb P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:33.274493 21514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b582ab8d8f044d9b9038a28e958feeb" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47" is_pre_election: true
I20250109 08:57:33.275091 21514 raft_consensus.cc:2463] T 1b582ab8d8f044d9b9038a28e958feeb P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 0.
I20250109 08:57:33.275727 21702 tablet_bootstrap.cc:654] T 8c0459d2eee64f0a83bd05ad5974f506 P a2b1a692435241ee988e0bd1661cd4c3: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:33.276307 21626 leader_election.cc:304] T 1b582ab8d8f044d9b9038a28e958feeb P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, 8bf6e26e98b14566899578a990b27d47; no voters:
I20250109 08:57:33.277110 21907 raft_consensus.cc:2798] T 1b582ab8d8f044d9b9038a28e958feeb P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:57:33.277444 21907 raft_consensus.cc:491] T 1b582ab8d8f044d9b9038a28e958feeb P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:57:33.277760 21907 raft_consensus.cc:3054] T 1b582ab8d8f044d9b9038a28e958feeb P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:33.282120 21584 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b582ab8d8f044d9b9038a28e958feeb" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3" is_pre_election: true
W20250109 08:57:33.283627 21627 leader_election.cc:343] T 1b582ab8d8f044d9b9038a28e958feeb P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:33.281237 21732 leader_election.cc:290] T 1b582ab8d8f044d9b9038a28e958feeb P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
I20250109 08:57:33.275475 21410 catalog_manager.cc:5526] T f5f85ad7c7c94acfaf9d550d55fce48c P a2b1a692435241ee988e0bd1661cd4c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194). New cstate: current_term: 1 leader_uuid: "a2b1a692435241ee988e0bd1661cd4c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: UNKNOWN } } }
I20250109 08:57:33.290540 21989 raft_consensus.cc:491] T 64c667f887ea41b4a484ee565b21fa16 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:33.291025 21989 raft_consensus.cc:513] T 64c667f887ea41b4a484ee565b21fa16 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:33.291759 21703 tablet_bootstrap.cc:492] T 1d3d5d2a2c88417a97c9d57262a356dd P 8bf6e26e98b14566899578a990b27d47: No bootstrap required, opened a new log
I20250109 08:57:33.292237 21703 ts_tablet_manager.cc:1397] T 1d3d5d2a2c88417a97c9d57262a356dd P 8bf6e26e98b14566899578a990b27d47: Time spent bootstrapping tablet: real 0.065s user 0.009s sys 0.002s
I20250109 08:57:33.294067 21989 leader_election.cc:290] T 64c667f887ea41b4a484ee565b21fa16 P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:33.294654 21703 raft_consensus.cc:357] T 1d3d5d2a2c88417a97c9d57262a356dd P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:33.295321 21703 raft_consensus.cc:383] T 1d3d5d2a2c88417a97c9d57262a356dd P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:33.295593 21703 raft_consensus.cc:738] T 1d3d5d2a2c88417a97c9d57262a356dd P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bf6e26e98b14566899578a990b27d47, State: Initialized, Role: FOLLOWER
I20250109 08:57:33.295491 21661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64c667f887ea41b4a484ee565b21fa16" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e" is_pre_election: true
I20250109 08:57:33.296329 21703 consensus_queue.cc:260] T 1d3d5d2a2c88417a97c9d57262a356dd P 8bf6e26e98b14566899578a990b27d47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
W20250109 08:57:33.296855 21552 leader_election.cc:343] T 64c667f887ea41b4a484ee565b21fa16 P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:33.299937 21703 ts_tablet_manager.cc:1428] T 1d3d5d2a2c88417a97c9d57262a356dd P 8bf6e26e98b14566899578a990b27d47: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20250109 08:57:33.300359 21512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64c667f887ea41b4a484ee565b21fa16" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47" is_pre_election: true
W20250109 08:57:33.304279 21552 leader_election.cc:343] T 64c667f887ea41b4a484ee565b21fa16 P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:33.304747 21552 leader_election.cc:304] T 64c667f887ea41b4a484ee565b21fa16 P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2b1a692435241ee988e0bd1661cd4c3; no voters: 787ddcc824c2490a808fc5a53452dc5e, 8bf6e26e98b14566899578a990b27d47
I20250109 08:57:33.305706 21703 tablet_bootstrap.cc:492] T d52f79ecf037457cbcc09d5fa43456cf P 8bf6e26e98b14566899578a990b27d47: Bootstrap starting.
I20250109 08:57:33.307077 21989 raft_consensus.cc:2743] T 64c667f887ea41b4a484ee565b21fa16 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:57:33.307744 21907 raft_consensus.cc:513] T 1b582ab8d8f044d9b9038a28e958feeb P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:33.313932 21710 consensus_queue.cc:1035] T b00671c87aac405a81c6543f4defd722 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:57:33.314229 21907 leader_election.cc:290] T 1b582ab8d8f044d9b9038a28e958feeb P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 election: Requested vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
I20250109 08:57:33.314553 21702 tablet_bootstrap.cc:492] T 8c0459d2eee64f0a83bd05ad5974f506 P a2b1a692435241ee988e0bd1661cd4c3: No bootstrap required, opened a new log
I20250109 08:57:33.315328 21702 ts_tablet_manager.cc:1397] T 8c0459d2eee64f0a83bd05ad5974f506 P a2b1a692435241ee988e0bd1661cd4c3: Time spent bootstrapping tablet: real 0.054s user 0.007s sys 0.005s
I20250109 08:57:33.317692 21702 raft_consensus.cc:357] T 8c0459d2eee64f0a83bd05ad5974f506 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:33.318362 21702 raft_consensus.cc:383] T 8c0459d2eee64f0a83bd05ad5974f506 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:33.318614 21702 raft_consensus.cc:738] T 8c0459d2eee64f0a83bd05ad5974f506 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2b1a692435241ee988e0bd1661cd4c3, State: Initialized, Role: FOLLOWER
I20250109 08:57:33.320070 21584 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b582ab8d8f044d9b9038a28e958feeb" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3"
I20250109 08:57:33.320832 21512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b582ab8d8f044d9b9038a28e958feeb" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47"
I20250109 08:57:33.321465 21512 raft_consensus.cc:3054] T 1b582ab8d8f044d9b9038a28e958feeb P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Advancing to term 1
W20250109 08:57:33.322731 21627 leader_election.cc:343] T 1b582ab8d8f044d9b9038a28e958feeb P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:33.322860 21702 consensus_queue.cc:260] T 8c0459d2eee64f0a83bd05ad5974f506 P a2b1a692435241ee988e0bd1661cd4c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:33.325018 21703 tablet_bootstrap.cc:654] T d52f79ecf037457cbcc09d5fa43456cf P 8bf6e26e98b14566899578a990b27d47: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:33.327056 21702 ts_tablet_manager.cc:1428] T 8c0459d2eee64f0a83bd05ad5974f506 P a2b1a692435241ee988e0bd1661cd4c3: Time spent starting tablet: real 0.011s user 0.001s sys 0.005s
I20250109 08:57:33.327915 21702 tablet_bootstrap.cc:492] T eb4516b9a16041c2adf34be520768f3f P a2b1a692435241ee988e0bd1661cd4c3: Bootstrap starting.
I20250109 08:57:33.331909 21704 tablet_bootstrap.cc:492] T 126a163a89c448be89ccb9490e9f0168 P 787ddcc824c2490a808fc5a53452dc5e: No bootstrap required, opened a new log
I20250109 08:57:33.332442 21704 ts_tablet_manager.cc:1397] T 126a163a89c448be89ccb9490e9f0168 P 787ddcc824c2490a808fc5a53452dc5e: Time spent bootstrapping tablet: real 0.103s user 0.013s sys 0.017s
I20250109 08:57:33.331876 21512 raft_consensus.cc:2463] T 1b582ab8d8f044d9b9038a28e958feeb P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 1.
I20250109 08:57:33.334079 21702 tablet_bootstrap.cc:654] T eb4516b9a16041c2adf34be520768f3f P a2b1a692435241ee988e0bd1661cd4c3: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:33.334225 21626 leader_election.cc:304] T 1b582ab8d8f044d9b9038a28e958feeb P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, 8bf6e26e98b14566899578a990b27d47; no voters: a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:57:33.335230 21704 raft_consensus.cc:357] T 126a163a89c448be89ccb9490e9f0168 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:33.335933 21704 raft_consensus.cc:383] T 126a163a89c448be89ccb9490e9f0168 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:33.336190 21704 raft_consensus.cc:738] T 126a163a89c448be89ccb9490e9f0168 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Initialized, Role: FOLLOWER
I20250109 08:57:33.336798 21704 consensus_queue.cc:260] T 126a163a89c448be89ccb9490e9f0168 P 787ddcc824c2490a808fc5a53452dc5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:33.337466 21907 raft_consensus.cc:2798] T 1b582ab8d8f044d9b9038a28e958feeb P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:57:33.337986 21907 raft_consensus.cc:695] T 1b582ab8d8f044d9b9038a28e958feeb P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Becoming Leader. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Running, Role: LEADER
I20250109 08:57:33.338786 21907 consensus_queue.cc:237] T 1b582ab8d8f044d9b9038a28e958feeb P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:33.360823 21819 consensus_queue.cc:1035] T f961d3f357944118a69f905621731552 P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:57:33.370187 21704 ts_tablet_manager.cc:1428] T 126a163a89c448be89ccb9490e9f0168 P 787ddcc824c2490a808fc5a53452dc5e: Time spent starting tablet: real 0.037s user 0.005s sys 0.000s
I20250109 08:57:33.369732 21411 catalog_manager.cc:5526] T 1b582ab8d8f044d9b9038a28e958feeb P 787ddcc824c2490a808fc5a53452dc5e reported cstate change: term changed from 0 to 1, leader changed from <none> to 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195). New cstate: current_term: 1 leader_uuid: "787ddcc824c2490a808fc5a53452dc5e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: HEALTHY } } }
I20250109 08:57:33.373257 21704 tablet_bootstrap.cc:492] T f961d3f357944118a69f905621731552 P 787ddcc824c2490a808fc5a53452dc5e: Bootstrap starting.
I20250109 08:57:33.379575 21703 tablet_bootstrap.cc:492] T d52f79ecf037457cbcc09d5fa43456cf P 8bf6e26e98b14566899578a990b27d47: No bootstrap required, opened a new log
I20250109 08:57:33.380136 21703 ts_tablet_manager.cc:1397] T d52f79ecf037457cbcc09d5fa43456cf P 8bf6e26e98b14566899578a990b27d47: Time spent bootstrapping tablet: real 0.075s user 0.017s sys 0.004s
W20250109 08:57:33.382151 21552 consensus_peers.cc:487] T f961d3f357944118a69f905621731552 P a2b1a692435241ee988e0bd1661cd4c3 -> Peer 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731): Couldn't send request to peer 787ddcc824c2490a808fc5a53452dc5e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:57:33.382874 21703 raft_consensus.cc:357] T d52f79ecf037457cbcc09d5fa43456cf P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:33.383702 21703 raft_consensus.cc:383] T d52f79ecf037457cbcc09d5fa43456cf P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:33.386519 21703 raft_consensus.cc:738] T d52f79ecf037457cbcc09d5fa43456cf P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bf6e26e98b14566899578a990b27d47, State: Initialized, Role: FOLLOWER
I20250109 08:57:33.387257 21703 consensus_queue.cc:260] T d52f79ecf037457cbcc09d5fa43456cf P 8bf6e26e98b14566899578a990b27d47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:33.388288 21704 tablet_bootstrap.cc:654] T f961d3f357944118a69f905621731552 P 787ddcc824c2490a808fc5a53452dc5e: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:33.389879 21703 ts_tablet_manager.cc:1428] T d52f79ecf037457cbcc09d5fa43456cf P 8bf6e26e98b14566899578a990b27d47: Time spent starting tablet: real 0.009s user 0.004s sys 0.001s
I20250109 08:57:33.390754 21703 tablet_bootstrap.cc:492] T 7781c00473024307987096b596da12ac P 8bf6e26e98b14566899578a990b27d47: Bootstrap starting.
I20250109 08:57:33.392346 21702 tablet_bootstrap.cc:492] T eb4516b9a16041c2adf34be520768f3f P a2b1a692435241ee988e0bd1661cd4c3: No bootstrap required, opened a new log
I20250109 08:57:33.392866 21702 ts_tablet_manager.cc:1397] T eb4516b9a16041c2adf34be520768f3f P a2b1a692435241ee988e0bd1661cd4c3: Time spent bootstrapping tablet: real 0.065s user 0.033s sys 0.031s
I20250109 08:57:33.395906 21702 raft_consensus.cc:357] T eb4516b9a16041c2adf34be520768f3f P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:33.399504 21702 raft_consensus.cc:383] T eb4516b9a16041c2adf34be520768f3f P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:33.399852 21702 raft_consensus.cc:738] T eb4516b9a16041c2adf34be520768f3f P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2b1a692435241ee988e0bd1661cd4c3, State: Initialized, Role: FOLLOWER
I20250109 08:57:33.400578 21702 consensus_queue.cc:260] T eb4516b9a16041c2adf34be520768f3f P a2b1a692435241ee988e0bd1661cd4c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:33.404711 21702 ts_tablet_manager.cc:1428] T eb4516b9a16041c2adf34be520768f3f P a2b1a692435241ee988e0bd1661cd4c3: Time spent starting tablet: real 0.012s user 0.006s sys 0.000s
I20250109 08:57:33.405730 21702 tablet_bootstrap.cc:492] T a3aa271f2f0144eea9261e0e07ab833b P a2b1a692435241ee988e0bd1661cd4c3: Bootstrap starting.
I20250109 08:57:33.399235 21703 tablet_bootstrap.cc:654] T 7781c00473024307987096b596da12ac P 8bf6e26e98b14566899578a990b27d47: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:33.407053 21704 tablet_bootstrap.cc:492] T f961d3f357944118a69f905621731552 P 787ddcc824c2490a808fc5a53452dc5e: No bootstrap required, opened a new log
I20250109 08:57:33.408571 21704 ts_tablet_manager.cc:1397] T f961d3f357944118a69f905621731552 P 787ddcc824c2490a808fc5a53452dc5e: Time spent bootstrapping tablet: real 0.035s user 0.008s sys 0.003s
I20250109 08:57:33.417940 21704 raft_consensus.cc:357] T f961d3f357944118a69f905621731552 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:33.418639 21704 raft_consensus.cc:383] T f961d3f357944118a69f905621731552 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:33.418912 21704 raft_consensus.cc:738] T f961d3f357944118a69f905621731552 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Initialized, Role: FOLLOWER
I20250109 08:57:33.420629 21704 consensus_queue.cc:260] T f961d3f357944118a69f905621731552 P 787ddcc824c2490a808fc5a53452dc5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:33.422777 21704 ts_tablet_manager.cc:1428] T f961d3f357944118a69f905621731552 P 787ddcc824c2490a808fc5a53452dc5e: Time spent starting tablet: real 0.014s user 0.005s sys 0.001s
I20250109 08:57:33.423596 21704 tablet_bootstrap.cc:492] T d015a10f95eb4f19b6d7c53cb201fa80 P 787ddcc824c2490a808fc5a53452dc5e: Bootstrap starting.
I20250109 08:57:33.425345 21702 tablet_bootstrap.cc:654] T a3aa271f2f0144eea9261e0e07ab833b P a2b1a692435241ee988e0bd1661cd4c3: Neither blocks nor log segments found. Creating new log.
W20250109 08:57:33.427834 21460 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20250109 08:57:33.434474 21703 tablet_bootstrap.cc:492] T 7781c00473024307987096b596da12ac P 8bf6e26e98b14566899578a990b27d47: No bootstrap required, opened a new log
I20250109 08:57:33.434942 21703 ts_tablet_manager.cc:1397] T 7781c00473024307987096b596da12ac P 8bf6e26e98b14566899578a990b27d47: Time spent bootstrapping tablet: real 0.044s user 0.011s sys 0.002s
I20250109 08:57:33.438297 21704 tablet_bootstrap.cc:654] T d015a10f95eb4f19b6d7c53cb201fa80 P 787ddcc824c2490a808fc5a53452dc5e: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:33.442469 21703 raft_consensus.cc:357] T 7781c00473024307987096b596da12ac P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:33.443187 21703 raft_consensus.cc:383] T 7781c00473024307987096b596da12ac P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:33.443434 21703 raft_consensus.cc:738] T 7781c00473024307987096b596da12ac P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bf6e26e98b14566899578a990b27d47, State: Initialized, Role: FOLLOWER
I20250109 08:57:33.443985 21703 consensus_queue.cc:260] T 7781c00473024307987096b596da12ac P 8bf6e26e98b14566899578a990b27d47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:33.446115 21703 ts_tablet_manager.cc:1428] T 7781c00473024307987096b596da12ac P 8bf6e26e98b14566899578a990b27d47: Time spent starting tablet: real 0.011s user 0.005s sys 0.001s
I20250109 08:57:33.446913 21703 tablet_bootstrap.cc:492] T 5717a02ef53a4b22a96f431ca6c20d93 P 8bf6e26e98b14566899578a990b27d47: Bootstrap starting.
I20250109 08:57:33.451988 21710 raft_consensus.cc:491] T d52f79ecf037457cbcc09d5fa43456cf P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:33.452510 21710 raft_consensus.cc:513] T d52f79ecf037457cbcc09d5fa43456cf P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:33.454497 21710 leader_election.cc:290] T d52f79ecf037457cbcc09d5fa43456cf P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
I20250109 08:57:33.457446 21514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d52f79ecf037457cbcc09d5fa43456cf" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47" is_pre_election: true
I20250109 08:57:33.458065 21514 raft_consensus.cc:2463] T d52f79ecf037457cbcc09d5fa43456cf P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 0.
I20250109 08:57:33.458138 21589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d52f79ecf037457cbcc09d5fa43456cf" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3" is_pre_election: true
I20250109 08:57:33.459101 21626 leader_election.cc:304] T d52f79ecf037457cbcc09d5fa43456cf P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, 8bf6e26e98b14566899578a990b27d47; no voters:
I20250109 08:57:33.460311 21710 raft_consensus.cc:2798] T d52f79ecf037457cbcc09d5fa43456cf P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:57:33.460688 21710 raft_consensus.cc:491] T d52f79ecf037457cbcc09d5fa43456cf P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:57:33.461037 21710 raft_consensus.cc:3054] T d52f79ecf037457cbcc09d5fa43456cf P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Advancing to term 1
W20250109 08:57:33.461635 21627 leader_election.cc:343] T d52f79ecf037457cbcc09d5fa43456cf P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:33.469403 21710 raft_consensus.cc:513] T d52f79ecf037457cbcc09d5fa43456cf P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:33.471374 21710 leader_election.cc:290] T d52f79ecf037457cbcc09d5fa43456cf P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 election: Requested vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
I20250109 08:57:33.472043 21512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d52f79ecf037457cbcc09d5fa43456cf" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47"
I20250109 08:57:33.472676 21512 raft_consensus.cc:3054] T d52f79ecf037457cbcc09d5fa43456cf P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:33.473104 21589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d52f79ecf037457cbcc09d5fa43456cf" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3"
W20250109 08:57:33.474120 21627 leader_election.cc:343] T d52f79ecf037457cbcc09d5fa43456cf P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:33.478493 21704 tablet_bootstrap.cc:492] T d015a10f95eb4f19b6d7c53cb201fa80 P 787ddcc824c2490a808fc5a53452dc5e: No bootstrap required, opened a new log
I20250109 08:57:33.478734 21702 tablet_bootstrap.cc:492] T a3aa271f2f0144eea9261e0e07ab833b P a2b1a692435241ee988e0bd1661cd4c3: No bootstrap required, opened a new log
I20250109 08:57:33.479068 21704 ts_tablet_manager.cc:1397] T d015a10f95eb4f19b6d7c53cb201fa80 P 787ddcc824c2490a808fc5a53452dc5e: Time spent bootstrapping tablet: real 0.056s user 0.016s sys 0.000s
I20250109 08:57:33.479176 21702 ts_tablet_manager.cc:1397] T a3aa271f2f0144eea9261e0e07ab833b P a2b1a692435241ee988e0bd1661cd4c3: Time spent bootstrapping tablet: real 0.074s user 0.023s sys 0.013s
I20250109 08:57:33.480547 21759 raft_consensus.cc:491] T 126a163a89c448be89ccb9490e9f0168 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:33.480998 21759 raft_consensus.cc:513] T 126a163a89c448be89ccb9490e9f0168 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:33.482910 21759 leader_election.cc:290] T 126a163a89c448be89ccb9490e9f0168 P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:33.482264 21702 raft_consensus.cc:357] T a3aa271f2f0144eea9261e0e07ab833b P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:33.483377 21702 raft_consensus.cc:383] T a3aa271f2f0144eea9261e0e07ab833b P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:33.483631 21702 raft_consensus.cc:738] T a3aa271f2f0144eea9261e0e07ab833b P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2b1a692435241ee988e0bd1661cd4c3, State: Initialized, Role: FOLLOWER
I20250109 08:57:33.484201 21702 consensus_queue.cc:260] T a3aa271f2f0144eea9261e0e07ab833b P a2b1a692435241ee988e0bd1661cd4c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:33.485230 21514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "126a163a89c448be89ccb9490e9f0168" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47" is_pre_election: true
I20250109 08:57:33.485800 21514 raft_consensus.cc:2463] T 126a163a89c448be89ccb9490e9f0168 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2b1a692435241ee988e0bd1661cd4c3 in term 0.
I20250109 08:57:33.486313 21702 ts_tablet_manager.cc:1428] T a3aa271f2f0144eea9261e0e07ab833b P a2b1a692435241ee988e0bd1661cd4c3: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20250109 08:57:33.487156 21702 tablet_bootstrap.cc:492] T 2935d69446cd458e92046d2b2f1b74ab P a2b1a692435241ee988e0bd1661cd4c3: Bootstrap starting.
I20250109 08:57:33.488865 21552 leader_election.cc:304] T 126a163a89c448be89ccb9490e9f0168 P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3; no voters:
I20250109 08:57:33.489850 21819 raft_consensus.cc:491] T 7781c00473024307987096b596da12ac P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:33.490281 21819 raft_consensus.cc:513] T 7781c00473024307987096b596da12ac P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:33.491945 21759 raft_consensus.cc:2798] T 126a163a89c448be89ccb9490e9f0168 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:57:33.492062 21819 leader_election.cc:290] T 7781c00473024307987096b596da12ac P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:33.492496 21702 tablet_bootstrap.cc:654] T 2935d69446cd458e92046d2b2f1b74ab P a2b1a692435241ee988e0bd1661cd4c3: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:33.486425 21661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "126a163a89c448be89ccb9490e9f0168" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e" is_pre_election: true
I20250109 08:57:33.492847 21759 raft_consensus.cc:491] T 126a163a89c448be89ccb9490e9f0168 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:57:33.502825 21759 raft_consensus.cc:3054] T 126a163a89c448be89ccb9490e9f0168 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:33.480126 21710 raft_consensus.cc:491] T 8c0459d2eee64f0a83bd05ad5974f506 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:33.494407 21663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7781c00473024307987096b596da12ac" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e" is_pre_election: true
I20250109 08:57:33.495538 21704 raft_consensus.cc:357] T d015a10f95eb4f19b6d7c53cb201fa80 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:33.505016 21704 raft_consensus.cc:383] T d015a10f95eb4f19b6d7c53cb201fa80 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:33.496919 21703 tablet_bootstrap.cc:654] T 5717a02ef53a4b22a96f431ca6c20d93 P 8bf6e26e98b14566899578a990b27d47: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:33.505062 21663 raft_consensus.cc:2463] T 7781c00473024307987096b596da12ac P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2b1a692435241ee988e0bd1661cd4c3 in term 0.
I20250109 08:57:33.499660 21511 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7781c00473024307987096b596da12ac" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47" is_pre_election: true
I20250109 08:57:33.492929 21907 raft_consensus.cc:491] T 126a163a89c448be89ccb9490e9f0168 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:33.506265 21511 raft_consensus.cc:2463] T 7781c00473024307987096b596da12ac P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2b1a692435241ee988e0bd1661cd4c3 in term 0.
I20250109 08:57:33.506678 21552 leader_election.cc:304] T 7781c00473024307987096b596da12ac P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, a2b1a692435241ee988e0bd1661cd4c3; no voters:
I20250109 08:57:33.507421 21989 raft_consensus.cc:2798] T 7781c00473024307987096b596da12ac P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:57:33.507112 21907 raft_consensus.cc:513] T 126a163a89c448be89ccb9490e9f0168 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:33.507771 21989 raft_consensus.cc:491] T 7781c00473024307987096b596da12ac P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:57:33.508116 21989 raft_consensus.cc:3054] T 7781c00473024307987096b596da12ac P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:33.508608 21661 raft_consensus.cc:2463] T 126a163a89c448be89ccb9490e9f0168 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2b1a692435241ee988e0bd1661cd4c3 in term 0.
I20250109 08:57:33.508922 21709 raft_consensus.cc:491] T a3aa271f2f0144eea9261e0e07ab833b P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:33.509359 21709 raft_consensus.cc:513] T a3aa271f2f0144eea9261e0e07ab833b P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:33.501554 21512 raft_consensus.cc:2463] T d52f79ecf037457cbcc09d5fa43456cf P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 1.
I20250109 08:57:33.519404 21584 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "126a163a89c448be89ccb9490e9f0168" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3" is_pre_election: true
I20250109 08:57:33.505733 21704 raft_consensus.cc:738] T d015a10f95eb4f19b6d7c53cb201fa80 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Initialized, Role: FOLLOWER
I20250109 08:57:33.520078 21589 raft_consensus.cc:3054] T a3aa271f2f0144eea9261e0e07ab833b P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:33.511157 21709 leader_election.cc:290] T a3aa271f2f0144eea9261e0e07ab833b P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:33.517570 21661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3aa271f2f0144eea9261e0e07ab833b" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e" is_pre_election: true
I20250109 08:57:33.517918 21907 leader_election.cc:290] T 126a163a89c448be89ccb9490e9f0168 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
I20250109 08:57:33.533247 21661 raft_consensus.cc:2388] T a3aa271f2f0144eea9261e0e07ab833b P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2b1a692435241ee988e0bd1661cd4c3 in current term 1: Already voted for candidate 8bf6e26e98b14566899578a990b27d47 in this term.
I20250109 08:57:33.534482 21989 raft_consensus.cc:513] T 7781c00473024307987096b596da12ac P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:33.515612 21511 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3aa271f2f0144eea9261e0e07ab833b" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47" is_pre_election: true
I20250109 08:57:33.515566 21759 raft_consensus.cc:513] T 126a163a89c448be89ccb9490e9f0168 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:33.527371 21514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "126a163a89c448be89ccb9490e9f0168" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47" is_pre_election: true
I20250109 08:57:33.537676 21514 raft_consensus.cc:2463] T 126a163a89c448be89ccb9490e9f0168 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 0.
I20250109 08:57:33.537595 21781 consensus_queue.cc:1035] T a3aa271f2f0144eea9261e0e07ab833b P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:57:33.536087 21626 leader_election.cc:304] T d52f79ecf037457cbcc09d5fa43456cf P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, 8bf6e26e98b14566899578a990b27d47; no voters: a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:57:33.539317 21626 leader_election.cc:304] T 126a163a89c448be89ccb9490e9f0168 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, 8bf6e26e98b14566899578a990b27d47; no voters:
I20250109 08:57:33.540244 21758 raft_consensus.cc:2798] T d52f79ecf037457cbcc09d5fa43456cf P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:57:33.510118 21710 raft_consensus.cc:513] T 8c0459d2eee64f0a83bd05ad5974f506 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:33.540702 21758 raft_consensus.cc:695] T d52f79ecf037457cbcc09d5fa43456cf P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Becoming Leader. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Running, Role: LEADER
I20250109 08:57:33.541105 21907 raft_consensus.cc:2798] T 126a163a89c448be89ccb9490e9f0168 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:57:33.541589 21907 raft_consensus.cc:491] T 126a163a89c448be89ccb9490e9f0168 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:57:33.542054 21907 raft_consensus.cc:3054] T 126a163a89c448be89ccb9490e9f0168 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:33.543936 21586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c0459d2eee64f0a83bd05ad5974f506" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3" is_pre_election: true
I20250109 08:57:33.544466 21586 raft_consensus.cc:2463] T 8c0459d2eee64f0a83bd05ad5974f506 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 0.
I20250109 08:57:33.544766 21514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c0459d2eee64f0a83bd05ad5974f506" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47" is_pre_election: true
I20250109 08:57:33.545343 21514 raft_consensus.cc:2463] T 8c0459d2eee64f0a83bd05ad5974f506 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 0.
I20250109 08:57:33.541615 21758 consensus_queue.cc:237] T d52f79ecf037457cbcc09d5fa43456cf P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:33.546079 21627 leader_election.cc:304] T 8c0459d2eee64f0a83bd05ad5974f506 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, a2b1a692435241ee988e0bd1661cd4c3; no voters:
I20250109 08:57:33.546641 21901 raft_consensus.cc:2798] T 8c0459d2eee64f0a83bd05ad5974f506 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:57:33.546974 21901 raft_consensus.cc:491] T 8c0459d2eee64f0a83bd05ad5974f506 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:57:33.547322 21901 raft_consensus.cc:3054] T 8c0459d2eee64f0a83bd05ad5974f506 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:33.547300 21704 consensus_queue.cc:260] T d015a10f95eb4f19b6d7c53cb201fa80 P 787ddcc824c2490a808fc5a53452dc5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:33.550464 21710 leader_election.cc:290] T 8c0459d2eee64f0a83bd05ad5974f506 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
I20250109 08:57:33.552527 21989 leader_election.cc:290] T 7781c00473024307987096b596da12ac P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 election: Requested vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:33.554330 21704 ts_tablet_manager.cc:1428] T d015a10f95eb4f19b6d7c53cb201fa80 P 787ddcc824c2490a808fc5a53452dc5e: Time spent starting tablet: real 0.061s user 0.005s sys 0.001s
I20250109 08:57:33.555222 21704 tablet_bootstrap.cc:492] T 64c667f887ea41b4a484ee565b21fa16 P 787ddcc824c2490a808fc5a53452dc5e: Bootstrap starting.
I20250109 08:57:33.552405 21781 consensus_queue.cc:1035] T a3aa271f2f0144eea9261e0e07ab833b P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20250109 08:57:33.557219 21901 raft_consensus.cc:513] T 8c0459d2eee64f0a83bd05ad5974f506 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:33.587271 21732 raft_consensus.cc:491] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:33.587863 21732 raft_consensus.cc:513] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:33.589727 21732 leader_election.cc:290] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
I20250109 08:57:33.578883 21584 raft_consensus.cc:2388] T 126a163a89c448be89ccb9490e9f0168 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 787ddcc824c2490a808fc5a53452dc5e in current term 1: Already voted for candidate a2b1a692435241ee988e0bd1661cd4c3 in this term.
I20250109 08:57:33.607249 21819 raft_consensus.cc:491] T eb4516b9a16041c2adf34be520768f3f P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:33.607720 21514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d31dbfb2b4a4a6ca4500f92df9dbb98" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47" is_pre_election: true
I20250109 08:57:33.610209 21759 leader_election.cc:290] T 126a163a89c448be89ccb9490e9f0168 P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 election: Requested vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:33.611749 21901 leader_election.cc:290] T 8c0459d2eee64f0a83bd05ad5974f506 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 election: Requested vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
W20250109 08:57:33.614799 21626 leader_election.cc:343] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:33.607805 21819 raft_consensus.cc:513] T eb4516b9a16041c2adf34be520768f3f P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:33.617125 21702 tablet_bootstrap.cc:492] T 2935d69446cd458e92046d2b2f1b74ab P a2b1a692435241ee988e0bd1661cd4c3: No bootstrap required, opened a new log
I20250109 08:57:33.615888 21511 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c0459d2eee64f0a83bd05ad5974f506" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47"
I20250109 08:57:33.617288 21589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d31dbfb2b4a4a6ca4500f92df9dbb98" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3" is_pre_election: true
I20250109 08:57:33.618476 21702 ts_tablet_manager.cc:1397] T 2935d69446cd458e92046d2b2f1b74ab P a2b1a692435241ee988e0bd1661cd4c3: Time spent bootstrapping tablet: real 0.131s user 0.016s sys 0.023s
I20250109 08:57:33.611848 21410 catalog_manager.cc:5526] T d52f79ecf037457cbcc09d5fa43456cf P 787ddcc824c2490a808fc5a53452dc5e reported cstate change: term changed from 0 to 1, leader changed from <none> to 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195). New cstate: current_term: 1 leader_uuid: "787ddcc824c2490a808fc5a53452dc5e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: HEALTHY } } }
W20250109 08:57:33.619058 21552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.226.193:42725, user_credentials={real_user=slave}} blocked reactor thread for 107249us
I20250109 08:57:33.621227 21514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7781c00473024307987096b596da12ac" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47"
I20250109 08:57:33.621790 21514 raft_consensus.cc:3054] T 7781c00473024307987096b596da12ac P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:33.623687 21709 raft_consensus.cc:491] T 1d3d5d2a2c88417a97c9d57262a356dd P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:33.624089 21709 raft_consensus.cc:513] T 1d3d5d2a2c88417a97c9d57262a356dd P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:33.626013 21552 leader_election.cc:304] T a3aa271f2f0144eea9261e0e07ab833b P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2b1a692435241ee988e0bd1661cd4c3; no voters: 787ddcc824c2490a808fc5a53452dc5e, 8bf6e26e98b14566899578a990b27d47
I20250109 08:57:33.628419 21514 raft_consensus.cc:2463] T 7781c00473024307987096b596da12ac P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2b1a692435241ee988e0bd1661cd4c3 in term 1.
I20250109 08:57:33.630656 21709 leader_election.cc:290] T 1d3d5d2a2c88417a97c9d57262a356dd P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:33.631309 21552 leader_election.cc:304] T 7781c00473024307987096b596da12ac P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3; no voters:
I20250109 08:57:33.631819 21709 raft_consensus.cc:2743] T a3aa271f2f0144eea9261e0e07ab833b P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:57:33.632228 21709 raft_consensus.cc:2798] T 7781c00473024307987096b596da12ac P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:57:33.632557 21709 raft_consensus.cc:695] T 7781c00473024307987096b596da12ac P a2b1a692435241ee988e0bd1661cd4c3 [term 1 LEADER]: Becoming Leader. State: Replica: a2b1a692435241ee988e0bd1661cd4c3, State: Running, Role: LEADER
I20250109 08:57:33.635995 21512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "126a163a89c448be89ccb9490e9f0168" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47"
I20250109 08:57:33.636598 21512 raft_consensus.cc:3054] T 126a163a89c448be89ccb9490e9f0168 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:33.641592 21586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c0459d2eee64f0a83bd05ad5974f506" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3"
W20250109 08:57:33.642021 21627 leader_election.cc:343] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:33.642189 21586 raft_consensus.cc:3054] T 8c0459d2eee64f0a83bd05ad5974f506 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:33.642421 21627 leader_election.cc:304] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e; no voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:57:33.627380 21511 raft_consensus.cc:3054] T 8c0459d2eee64f0a83bd05ad5974f506 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:33.643417 21710 raft_consensus.cc:2743] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:57:33.622130 21842 raft_consensus.cc:491] T 1d3d5d2a2c88417a97c9d57262a356dd P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:33.633209 21709 consensus_queue.cc:237] T 7781c00473024307987096b596da12ac P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:33.580749 21907 raft_consensus.cc:513] T 126a163a89c448be89ccb9490e9f0168 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:33.646042 21819 leader_election.cc:290] T eb4516b9a16041c2adf34be520768f3f P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:33.650696 21842 raft_consensus.cc:513] T 1d3d5d2a2c88417a97c9d57262a356dd P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:33.678683 21509 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d3d5d2a2c88417a97c9d57262a356dd" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47" is_pre_election: true
I20250109 08:57:33.684001 21842 leader_election.cc:290] T 1d3d5d2a2c88417a97c9d57262a356dd P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:33.675390 21661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7781c00473024307987096b596da12ac" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e"
I20250109 08:57:33.694792 21703 tablet_bootstrap.cc:492] T 5717a02ef53a4b22a96f431ca6c20d93 P 8bf6e26e98b14566899578a990b27d47: No bootstrap required, opened a new log
I20250109 08:57:33.695257 21703 ts_tablet_manager.cc:1397] T 5717a02ef53a4b22a96f431ca6c20d93 P 8bf6e26e98b14566899578a990b27d47: Time spent bootstrapping tablet: real 0.249s user 0.022s sys 0.014s
I20250109 08:57:33.705329 21586 raft_consensus.cc:2463] T 8c0459d2eee64f0a83bd05ad5974f506 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 1.
I20250109 08:57:33.707234 21511 raft_consensus.cc:2463] T 8c0459d2eee64f0a83bd05ad5974f506 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 1.
I20250109 08:57:33.707304 21510 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb4516b9a16041c2adf34be520768f3f" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47" is_pre_election: true
I20250109 08:57:33.709179 21660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d3d5d2a2c88417a97c9d57262a356dd" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e" is_pre_election: true
I20250109 08:57:33.697611 21661 raft_consensus.cc:3054] T 7781c00473024307987096b596da12ac P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:33.715498 21512 raft_consensus.cc:2463] T 126a163a89c448be89ccb9490e9f0168 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2b1a692435241ee988e0bd1661cd4c3 in term 1.
I20250109 08:57:33.681715 21509 raft_consensus.cc:2463] T 1d3d5d2a2c88417a97c9d57262a356dd P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2b1a692435241ee988e0bd1661cd4c3 in term 0.
I20250109 08:57:33.730118 21627 leader_election.cc:304] T 8c0459d2eee64f0a83bd05ad5974f506 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, a2b1a692435241ee988e0bd1661cd4c3; no voters:
I20250109 08:57:33.731827 21901 raft_consensus.cc:2798] T 8c0459d2eee64f0a83bd05ad5974f506 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:57:33.732107 21585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d3d5d2a2c88417a97c9d57262a356dd" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3" is_pre_election: true
I20250109 08:57:33.732352 21901 raft_consensus.cc:695] T 8c0459d2eee64f0a83bd05ad5974f506 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Becoming Leader. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Running, Role: LEADER
I20250109 08:57:33.732802 21585 raft_consensus.cc:2463] T 1d3d5d2a2c88417a97c9d57262a356dd P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8bf6e26e98b14566899578a990b27d47 in term 0.
I20250109 08:57:33.739591 21663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d3d5d2a2c88417a97c9d57262a356dd" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e" is_pre_election: true
I20250109 08:57:33.733186 21901 consensus_queue.cc:237] T 8c0459d2eee64f0a83bd05ad5974f506 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:33.754398 21704 tablet_bootstrap.cc:654] T 64c667f887ea41b4a484ee565b21fa16 P 787ddcc824c2490a808fc5a53452dc5e: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:33.759665 21663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb4516b9a16041c2adf34be520768f3f" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e" is_pre_election: true
I20250109 08:57:33.761444 21552 leader_election.cc:304] T 126a163a89c448be89ccb9490e9f0168 P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3; no voters:
I20250109 08:57:33.762284 21709 raft_consensus.cc:2798] T 126a163a89c448be89ccb9490e9f0168 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:57:33.762709 21709 raft_consensus.cc:695] T 126a163a89c448be89ccb9490e9f0168 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 LEADER]: Becoming Leader. State: Replica: a2b1a692435241ee988e0bd1661cd4c3, State: Running, Role: LEADER
I20250109 08:57:33.763948 21552 leader_election.cc:304] T 1d3d5d2a2c88417a97c9d57262a356dd P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3; no voters:
I20250109 08:57:33.763264 21709 consensus_queue.cc:237] T 126a163a89c448be89ccb9490e9f0168 P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:33.757867 21907 leader_election.cc:290] T 126a163a89c448be89ccb9490e9f0168 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 election: Requested vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
I20250109 08:57:33.758221 21659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "126a163a89c448be89ccb9490e9f0168" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e"
I20250109 08:57:33.772132 21703 raft_consensus.cc:357] T 5717a02ef53a4b22a96f431ca6c20d93 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:33.774045 21703 raft_consensus.cc:383] T 5717a02ef53a4b22a96f431ca6c20d93 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:33.774274 21703 raft_consensus.cc:738] T 5717a02ef53a4b22a96f431ca6c20d93 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bf6e26e98b14566899578a990b27d47, State: Initialized, Role: FOLLOWER
I20250109 08:57:33.774956 21703 consensus_queue.cc:260] T 5717a02ef53a4b22a96f431ca6c20d93 P 8bf6e26e98b14566899578a990b27d47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
W20250109 08:57:33.774287 21477 leader_election.cc:343] T 1d3d5d2a2c88417a97c9d57262a356dd P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:33.766604 22033 raft_consensus.cc:2798] T 1d3d5d2a2c88417a97c9d57262a356dd P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:57:33.762482 21661 raft_consensus.cc:2463] T 7781c00473024307987096b596da12ac P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2b1a692435241ee988e0bd1661cd4c3 in term 1.
I20250109 08:57:33.781162 21702 raft_consensus.cc:357] T 2935d69446cd458e92046d2b2f1b74ab P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:33.781931 21702 raft_consensus.cc:383] T 2935d69446cd458e92046d2b2f1b74ab P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:33.782214 21702 raft_consensus.cc:738] T 2935d69446cd458e92046d2b2f1b74ab P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2b1a692435241ee988e0bd1661cd4c3, State: Initialized, Role: FOLLOWER
I20250109 08:57:33.782879 21702 consensus_queue.cc:260] T 2935d69446cd458e92046d2b2f1b74ab P a2b1a692435241ee988e0bd1661cd4c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:33.771250 21511 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "126a163a89c448be89ccb9490e9f0168" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47"
I20250109 08:57:33.778561 22033 raft_consensus.cc:491] T 1d3d5d2a2c88417a97c9d57262a356dd P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:57:33.787071 21582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "126a163a89c448be89ccb9490e9f0168" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3"
I20250109 08:57:33.796502 22033 raft_consensus.cc:3054] T 1d3d5d2a2c88417a97c9d57262a356dd P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:33.798699 21703 ts_tablet_manager.cc:1428] T 5717a02ef53a4b22a96f431ca6c20d93 P 8bf6e26e98b14566899578a990b27d47: Time spent starting tablet: real 0.103s user 0.007s sys 0.000s
I20250109 08:57:33.801101 21703 tablet_bootstrap.cc:492] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 8bf6e26e98b14566899578a990b27d47: Bootstrap starting.
I20250109 08:57:33.788048 21659 raft_consensus.cc:2388] T 126a163a89c448be89ccb9490e9f0168 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate a2b1a692435241ee988e0bd1661cd4c3 in current term 1: Already voted for candidate 787ddcc824c2490a808fc5a53452dc5e in this term.
I20250109 08:57:33.784901 21584 raft_consensus.cc:3054] T eb4516b9a16041c2adf34be520768f3f P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:33.804265 21732 raft_consensus.cc:491] T d015a10f95eb4f19b6d7c53cb201fa80 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:33.863085 21732 raft_consensus.cc:513] T d015a10f95eb4f19b6d7c53cb201fa80 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:33.867002 21758 consensus_queue.cc:1035] T ab6c3fcdf226433c9e1303004f6c96b4 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20250109 08:57:33.872506 21702 ts_tablet_manager.cc:1428] T 2935d69446cd458e92046d2b2f1b74ab P a2b1a692435241ee988e0bd1661cd4c3: Time spent starting tablet: real 0.254s user 0.009s sys 0.013s
I20250109 08:57:33.867249 21511 raft_consensus.cc:2388] T 126a163a89c448be89ccb9490e9f0168 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 787ddcc824c2490a808fc5a53452dc5e in current term 1: Already voted for candidate a2b1a692435241ee988e0bd1661cd4c3 in this term.
I20250109 08:57:33.874138 21702 tablet_bootstrap.cc:492] T d52f79ecf037457cbcc09d5fa43456cf P a2b1a692435241ee988e0bd1661cd4c3: Bootstrap starting.
I20250109 08:57:33.880069 21732 leader_election.cc:290] T d015a10f95eb4f19b6d7c53cb201fa80 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
I20250109 08:57:33.905139 21758 raft_consensus.cc:491] T f961d3f357944118a69f905621731552 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:33.919749 21552 leader_election.cc:304] T eb4516b9a16041c2adf34be520768f3f P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2b1a692435241ee988e0bd1661cd4c3; no voters: 787ddcc824c2490a808fc5a53452dc5e, 8bf6e26e98b14566899578a990b27d47
W20250109 08:57:33.912902 21626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.226.193:42725, user_credentials={real_user=slave}} blocked reactor thread for 43650.4us
I20250109 08:57:33.919162 21758 raft_consensus.cc:513] T f961d3f357944118a69f905621731552 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:33.933866 21758 leader_election.cc:290] T f961d3f357944118a69f905621731552 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
I20250109 08:57:33.925484 21583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d015a10f95eb4f19b6d7c53cb201fa80" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3" is_pre_election: true
I20250109 08:57:33.949023 21702 tablet_bootstrap.cc:654] T d52f79ecf037457cbcc09d5fa43456cf P a2b1a692435241ee988e0bd1661cd4c3: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:33.955371 21478 leader_election.cc:304] T 1d3d5d2a2c88417a97c9d57262a356dd P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3; no voters: 787ddcc824c2490a808fc5a53452dc5e
I20250109 08:57:33.954537 22033 raft_consensus.cc:513] T 1d3d5d2a2c88417a97c9d57262a356dd P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:33.973769 22029 consensus_queue.cc:1035] T f5f85ad7c7c94acfaf9d550d55fce48c P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20250109 08:57:33.973747 21702 tablet_bootstrap.cc:492] T d52f79ecf037457cbcc09d5fa43456cf P a2b1a692435241ee988e0bd1661cd4c3: No bootstrap required, opened a new log
W20250109 08:57:33.981570 21627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.226.194:45527, user_credentials={real_user=slave}} blocked reactor thread for 63871.9us
I20250109 08:57:33.981886 21702 ts_tablet_manager.cc:1397] T d52f79ecf037457cbcc09d5fa43456cf P a2b1a692435241ee988e0bd1661cd4c3: Time spent bootstrapping tablet: real 0.108s user 0.017s sys 0.004s
I20250109 08:57:33.980293 22038 raft_consensus.cc:2798] T 1d3d5d2a2c88417a97c9d57262a356dd P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:57:33.983824 21704 tablet_bootstrap.cc:492] T 64c667f887ea41b4a484ee565b21fa16 P 787ddcc824c2490a808fc5a53452dc5e: No bootstrap required, opened a new log
I20250109 08:57:33.984524 21704 ts_tablet_manager.cc:1397] T 64c667f887ea41b4a484ee565b21fa16 P 787ddcc824c2490a808fc5a53452dc5e: Time spent bootstrapping tablet: real 0.429s user 0.015s sys 0.016s
I20250109 08:57:33.993706 21703 tablet_bootstrap.cc:654] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 8bf6e26e98b14566899578a990b27d47: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:34.001413 22039 raft_consensus.cc:491] T 2935d69446cd458e92046d2b2f1b74ab P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:34.008716 21411 catalog_manager.cc:5526] T 7781c00473024307987096b596da12ac P a2b1a692435241ee988e0bd1661cd4c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194). New cstate: current_term: 1 leader_uuid: "a2b1a692435241ee988e0bd1661cd4c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: UNKNOWN } } }
I20250109 08:57:34.011029 21704 raft_consensus.cc:357] T 64c667f887ea41b4a484ee565b21fa16 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:34.010857 21411 catalog_manager.cc:5526] T 126a163a89c448be89ccb9490e9f0168 P a2b1a692435241ee988e0bd1661cd4c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194). New cstate: current_term: 1 leader_uuid: "a2b1a692435241ee988e0bd1661cd4c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: UNKNOWN } } }
I20250109 08:57:33.995373 22038 raft_consensus.cc:491] T 1d3d5d2a2c88417a97c9d57262a356dd P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:57:34.011722 21704 raft_consensus.cc:383] T 64c667f887ea41b4a484ee565b21fa16 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:34.013513 21704 raft_consensus.cc:738] T 64c667f887ea41b4a484ee565b21fa16 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Initialized, Role: FOLLOWER
I20250109 08:57:34.014240 21704 consensus_queue.cc:260] T 64c667f887ea41b4a484ee565b21fa16 P 787ddcc824c2490a808fc5a53452dc5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:34.016548 21704 ts_tablet_manager.cc:1428] T 64c667f887ea41b4a484ee565b21fa16 P 787ddcc824c2490a808fc5a53452dc5e: Time spent starting tablet: real 0.032s user 0.007s sys 0.000s
I20250109 08:57:34.017484 21704 tablet_bootstrap.cc:492] T 5717a02ef53a4b22a96f431ca6c20d93 P 787ddcc824c2490a808fc5a53452dc5e: Bootstrap starting.
I20250109 08:57:34.018214 22033 leader_election.cc:290] T 1d3d5d2a2c88417a97c9d57262a356dd P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 election: Requested vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:34.019570 21702 raft_consensus.cc:357] T d52f79ecf037457cbcc09d5fa43456cf P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:34.020036 21702 raft_consensus.cc:383] T d52f79ecf037457cbcc09d5fa43456cf P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:34.020191 21702 raft_consensus.cc:738] T d52f79ecf037457cbcc09d5fa43456cf P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2b1a692435241ee988e0bd1661cd4c3, State: Initialized, Role: FOLLOWER
I20250109 08:57:34.020674 21702 consensus_queue.cc:260] T d52f79ecf037457cbcc09d5fa43456cf P a2b1a692435241ee988e0bd1661cd4c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:34.022616 21702 ts_tablet_manager.cc:1428] T d52f79ecf037457cbcc09d5fa43456cf P a2b1a692435241ee988e0bd1661cd4c3: Time spent starting tablet: real 0.040s user 0.003s sys 0.000s
I20250109 08:57:34.023075 22039 raft_consensus.cc:513] T 2935d69446cd458e92046d2b2f1b74ab P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:34.024753 22039 leader_election.cc:290] T 2935d69446cd458e92046d2b2f1b74ab P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:34.014168 21587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f961d3f357944118a69f905621731552" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3" is_pre_election: true
I20250109 08:57:34.031390 21512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d3d5d2a2c88417a97c9d57262a356dd" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47"
I20250109 08:57:34.029158 22038 raft_consensus.cc:3054] T 1d3d5d2a2c88417a97c9d57262a356dd P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:34.037572 21702 tablet_bootstrap.cc:492] T d015a10f95eb4f19b6d7c53cb201fa80 P a2b1a692435241ee988e0bd1661cd4c3: Bootstrap starting.
I20250109 08:57:34.030318 21626 leader_election.cc:304] T 126a163a89c448be89ccb9490e9f0168 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e; no voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:57:34.039054 21508 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2935d69446cd458e92046d2b2f1b74ab" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47" is_pre_election: true
W20250109 08:57:34.044997 21478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.226.194:45527, user_credentials={real_user=slave}} blocked reactor thread for 62158.2us
I20250109 08:57:34.045146 22042 raft_consensus.cc:2743] T 126a163a89c448be89ccb9490e9f0168 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20250109 08:57:34.049857 21552 leader_election.cc:343] T 2935d69446cd458e92046d2b2f1b74ab P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:34.052639 22041 raft_consensus.cc:491] T 64c667f887ea41b4a484ee565b21fa16 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:34.057509 22041 raft_consensus.cc:513] T 64c667f887ea41b4a484ee565b21fa16 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
W20250109 08:57:34.076318 21627 leader_election.cc:343] T d015a10f95eb4f19b6d7c53cb201fa80 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Illegal state: must be running to vote when last-logged opid is not known
W20250109 08:57:34.082617 21627 consensus_peers.cc:487] T 1b582ab8d8f044d9b9038a28e958feeb P 787ddcc824c2490a808fc5a53452dc5e -> Peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Couldn't send request to peer a2b1a692435241ee988e0bd1661cd4c3. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:57:34.080679 21657 raft_consensus.cc:3054] T f961d3f357944118a69f905621731552 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:34.091742 22044 consensus_queue.cc:1035] T ab6c3fcdf226433c9e1303004f6c96b4 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20250109 08:57:34.096488 21508 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d015a10f95eb4f19b6d7c53cb201fa80" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47" is_pre_election: true
I20250109 08:57:34.097167 21508 raft_consensus.cc:2463] T d015a10f95eb4f19b6d7c53cb201fa80 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 0.
I20250109 08:57:34.081075 21660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d3d5d2a2c88417a97c9d57262a356dd" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e"
I20250109 08:57:34.080675 21656 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2935d69446cd458e92046d2b2f1b74ab" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e" is_pre_election: true
I20250109 08:57:34.101707 22041 leader_election.cc:290] T 64c667f887ea41b4a484ee565b21fa16 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
W20250109 08:57:34.105279 21552 consensus_peers.cc:487] T f5f85ad7c7c94acfaf9d550d55fce48c P a2b1a692435241ee988e0bd1661cd4c3 -> Peer 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731): Couldn't send request to peer 787ddcc824c2490a808fc5a53452dc5e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:57:34.106660 21411 catalog_manager.cc:5526] T 8c0459d2eee64f0a83bd05ad5974f506 P 787ddcc824c2490a808fc5a53452dc5e reported cstate change: term changed from 0 to 1, leader changed from <none> to 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195). New cstate: current_term: 1 leader_uuid: "787ddcc824c2490a808fc5a53452dc5e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: HEALTHY } } }
I20250109 08:57:34.108557 21704 tablet_bootstrap.cc:654] T 5717a02ef53a4b22a96f431ca6c20d93 P 787ddcc824c2490a808fc5a53452dc5e: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:34.106592 22038 raft_consensus.cc:513] T 1d3d5d2a2c88417a97c9d57262a356dd P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:34.116343 21512 raft_consensus.cc:2388] T 1d3d5d2a2c88417a97c9d57262a356dd P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate a2b1a692435241ee988e0bd1661cd4c3 in current term 1: Already voted for candidate 8bf6e26e98b14566899578a990b27d47 in this term.
I20250109 08:57:34.116894 21510 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f961d3f357944118a69f905621731552" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47" is_pre_election: true
I20250109 08:57:34.120430 21656 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d3d5d2a2c88417a97c9d57262a356dd" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e"
W20250109 08:57:34.122047 21477 leader_election.cc:343] T 1d3d5d2a2c88417a97c9d57262a356dd P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731): Illegal state: must be running to vote when last-logged opid is not known
W20250109 08:57:34.124029 21552 leader_election.cc:343] T 1d3d5d2a2c88417a97c9d57262a356dd P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:34.124473 21552 leader_election.cc:304] T 1d3d5d2a2c88417a97c9d57262a356dd P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2b1a692435241ee988e0bd1661cd4c3; no voters: 787ddcc824c2490a808fc5a53452dc5e, 8bf6e26e98b14566899578a990b27d47
W20250109 08:57:34.125509 21552 leader_election.cc:343] T 2935d69446cd458e92046d2b2f1b74ab P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:34.125913 21552 leader_election.cc:304] T 2935d69446cd458e92046d2b2f1b74ab P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2b1a692435241ee988e0bd1661cd4c3; no voters: 787ddcc824c2490a808fc5a53452dc5e, 8bf6e26e98b14566899578a990b27d47
I20250109 08:57:34.127187 22039 raft_consensus.cc:2743] T 2935d69446cd458e92046d2b2f1b74ab P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:57:34.125978 22029 raft_consensus.cc:2743] T 1d3d5d2a2c88417a97c9d57262a356dd P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20250109 08:57:34.135010 21581 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d3d5d2a2c88417a97c9d57262a356dd" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3"
I20250109 08:57:34.135834 21581 raft_consensus.cc:2388] T 1d3d5d2a2c88417a97c9d57262a356dd P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 8bf6e26e98b14566899578a990b27d47 in current term 1: Already voted for candidate a2b1a692435241ee988e0bd1661cd4c3 in this term.
I20250109 08:57:34.137953 21478 leader_election.cc:304] T 1d3d5d2a2c88417a97c9d57262a356dd P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8bf6e26e98b14566899578a990b27d47; no voters: 787ddcc824c2490a808fc5a53452dc5e, a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:57:34.139520 22032 raft_consensus.cc:2743] T 1d3d5d2a2c88417a97c9d57262a356dd P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20250109 08:57:34.138707 21509 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64c667f887ea41b4a484ee565b21fa16" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47" is_pre_election: true
I20250109 08:57:34.144016 22032 raft_consensus.cc:491] T 5717a02ef53a4b22a96f431ca6c20d93 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:34.144227 21626 leader_election.cc:304] T d015a10f95eb4f19b6d7c53cb201fa80 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, 8bf6e26e98b14566899578a990b27d47; no voters: a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:57:34.125945 22038 leader_election.cc:290] T 1d3d5d2a2c88417a97c9d57262a356dd P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 election: Requested vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:34.148928 21710 raft_consensus.cc:2798] T d015a10f95eb4f19b6d7c53cb201fa80 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:57:34.149303 21710 raft_consensus.cc:491] T d015a10f95eb4f19b6d7c53cb201fa80 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:57:34.149610 21710 raft_consensus.cc:3054] T d015a10f95eb4f19b6d7c53cb201fa80 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:34.155678 21702 tablet_bootstrap.cc:654] T d015a10f95eb4f19b6d7c53cb201fa80 P a2b1a692435241ee988e0bd1661cd4c3: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:34.144483 22032 raft_consensus.cc:513] T 5717a02ef53a4b22a96f431ca6c20d93 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:34.157639 22032 leader_election.cc:290] T 5717a02ef53a4b22a96f431ca6c20d93 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:34.158766 21656 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5717a02ef53a4b22a96f431ca6c20d93" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e" is_pre_election: true
W20250109 08:57:34.160255 21477 leader_election.cc:343] T 5717a02ef53a4b22a96f431ca6c20d93 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:34.162812 21585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64c667f887ea41b4a484ee565b21fa16" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3" is_pre_election: true
I20250109 08:57:34.163463 21585 raft_consensus.cc:2463] T 64c667f887ea41b4a484ee565b21fa16 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 0.
I20250109 08:57:34.165257 21580 raft_consensus.cc:3054] T d52f79ecf037457cbcc09d5fa43456cf P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:34.171006 21627 leader_election.cc:304] T 64c667f887ea41b4a484ee565b21fa16 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, a2b1a692435241ee988e0bd1661cd4c3; no voters:
I20250109 08:57:34.174110 21585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5717a02ef53a4b22a96f431ca6c20d93" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3" is_pre_election: true
I20250109 08:57:34.174669 21907 raft_consensus.cc:2798] T 64c667f887ea41b4a484ee565b21fa16 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:57:34.175051 21907 raft_consensus.cc:491] T 64c667f887ea41b4a484ee565b21fa16 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:57:34.175379 21907 raft_consensus.cc:3054] T 64c667f887ea41b4a484ee565b21fa16 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Advancing to term 1
W20250109 08:57:34.176677 21478 leader_election.cc:343] T 5717a02ef53a4b22a96f431ca6c20d93 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:34.177104 21478 leader_election.cc:304] T 5717a02ef53a4b22a96f431ca6c20d93 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8bf6e26e98b14566899578a990b27d47; no voters: 787ddcc824c2490a808fc5a53452dc5e, a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:57:34.189157 22038 raft_consensus.cc:2743] T 5717a02ef53a4b22a96f431ca6c20d93 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:57:34.190086 21907 raft_consensus.cc:513] T 64c667f887ea41b4a484ee565b21fa16 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:34.192094 21907 leader_election.cc:290] T 64c667f887ea41b4a484ee565b21fa16 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 election: Requested vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
I20250109 08:57:34.192906 21626 leader_election.cc:304] T f961d3f357944118a69f905621731552 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e; no voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3
W20250109 08:57:34.202688 22054 log.cc:926] Time spent T f5f85ad7c7c94acfaf9d550d55fce48c P 8bf6e26e98b14566899578a990b27d47: Append to log took a long time: real 0.074s user 0.000s sys 0.001s
I20250109 08:57:34.211277 21703 tablet_bootstrap.cc:492] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 8bf6e26e98b14566899578a990b27d47: No bootstrap required, opened a new log
I20250109 08:57:34.211767 21703 ts_tablet_manager.cc:1397] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 8bf6e26e98b14566899578a990b27d47: Time spent bootstrapping tablet: real 0.411s user 0.016s sys 0.009s
I20250109 08:57:34.218259 21588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64c667f887ea41b4a484ee565b21fa16" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3"
I20250109 08:57:34.218874 21588 raft_consensus.cc:3054] T 64c667f887ea41b4a484ee565b21fa16 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:34.220439 21703 raft_consensus.cc:357] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:34.221128 21703 raft_consensus.cc:383] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:34.221374 21703 raft_consensus.cc:738] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bf6e26e98b14566899578a990b27d47, State: Initialized, Role: FOLLOWER
I20250109 08:57:34.221956 21703 consensus_queue.cc:260] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 8bf6e26e98b14566899578a990b27d47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:34.227141 21710 raft_consensus.cc:513] T d015a10f95eb4f19b6d7c53cb201fa80 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:34.231194 21901 consensus_queue.cc:1035] T d52f79ecf037457cbcc09d5fa43456cf P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:57:34.235304 21710 leader_election.cc:290] T d015a10f95eb4f19b6d7c53cb201fa80 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 election: Requested vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
I20250109 08:57:34.236552 21703 ts_tablet_manager.cc:1428] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 8bf6e26e98b14566899578a990b27d47: Time spent starting tablet: real 0.025s user 0.007s sys 0.000s
I20250109 08:57:34.237583 21703 tablet_bootstrap.cc:492] T 7c19ed9fdde5463f84e43bea7b860d03 P 8bf6e26e98b14566899578a990b27d47: Bootstrap starting.
I20250109 08:57:34.238765 21585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d015a10f95eb4f19b6d7c53cb201fa80" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3"
I20250109 08:57:34.249189 22036 consensus_queue.cc:1035] T 7781c00473024307987096b596da12ac P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20250109 08:57:34.249480 21901 consensus_queue.cc:1035] T 1b582ab8d8f044d9b9038a28e958feeb P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20250109 08:57:34.251859 21627 leader_election.cc:343] T d015a10f95eb4f19b6d7c53cb201fa80 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Illegal state: must be running to vote when last-logged opid is not known
W20250109 08:57:34.253180 21626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.226.193:42725, user_credentials={real_user=slave}} blocked reactor thread for 46440.4us
I20250109 08:57:34.277675 22047 raft_consensus.cc:2743] T f961d3f357944118a69f905621731552 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:57:34.274552 21989 raft_consensus.cc:2743] T eb4516b9a16041c2adf34be520768f3f P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:57:34.287511 22036 consensus_queue.cc:1035] T 7781c00473024307987096b596da12ac P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:57:34.299535 21588 raft_consensus.cc:2463] T 64c667f887ea41b4a484ee565b21fa16 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 1.
I20250109 08:57:34.299588 21702 tablet_bootstrap.cc:492] T d015a10f95eb4f19b6d7c53cb201fa80 P a2b1a692435241ee988e0bd1661cd4c3: No bootstrap required, opened a new log
I20250109 08:57:34.300734 21627 leader_election.cc:304] T 64c667f887ea41b4a484ee565b21fa16 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, a2b1a692435241ee988e0bd1661cd4c3; no voters:
I20250109 08:57:34.300976 21702 ts_tablet_manager.cc:1397] T d015a10f95eb4f19b6d7c53cb201fa80 P a2b1a692435241ee988e0bd1661cd4c3: Time spent bootstrapping tablet: real 0.266s user 0.009s sys 0.008s
I20250109 08:57:34.301118 21704 tablet_bootstrap.cc:492] T 5717a02ef53a4b22a96f431ca6c20d93 P 787ddcc824c2490a808fc5a53452dc5e: No bootstrap required, opened a new log
I20250109 08:57:34.301549 21704 ts_tablet_manager.cc:1397] T 5717a02ef53a4b22a96f431ca6c20d93 P 787ddcc824c2490a808fc5a53452dc5e: Time spent bootstrapping tablet: real 0.284s user 0.015s sys 0.005s
I20250109 08:57:34.304529 22047 raft_consensus.cc:2798] T 64c667f887ea41b4a484ee565b21fa16 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:57:34.305058 22047 raft_consensus.cc:695] T 64c667f887ea41b4a484ee565b21fa16 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Becoming Leader. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Running, Role: LEADER
W20250109 08:57:34.293738 21626 leader_election.cc:343] T 64c667f887ea41b4a484ee565b21fa16 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:34.305811 22047 consensus_queue.cc:237] T 64c667f887ea41b4a484ee565b21fa16 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:34.320133 21702 raft_consensus.cc:357] T d015a10f95eb4f19b6d7c53cb201fa80 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:34.320917 21702 raft_consensus.cc:383] T d015a10f95eb4f19b6d7c53cb201fa80 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:34.321202 21702 raft_consensus.cc:738] T d015a10f95eb4f19b6d7c53cb201fa80 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2b1a692435241ee988e0bd1661cd4c3, State: Initialized, Role: FOLLOWER
I20250109 08:57:34.321789 21702 consensus_queue.cc:260] T d015a10f95eb4f19b6d7c53cb201fa80 P a2b1a692435241ee988e0bd1661cd4c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:34.346460 21704 raft_consensus.cc:357] T 5717a02ef53a4b22a96f431ca6c20d93 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:34.347239 21704 raft_consensus.cc:383] T 5717a02ef53a4b22a96f431ca6c20d93 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:34.347489 21704 raft_consensus.cc:738] T 5717a02ef53a4b22a96f431ca6c20d93 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Initialized, Role: FOLLOWER
I20250109 08:57:34.347635 22036 consensus_queue.cc:1035] T 126a163a89c448be89ccb9490e9f0168 P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:57:34.348063 21704 consensus_queue.cc:260] T 5717a02ef53a4b22a96f431ca6c20d93 P 787ddcc824c2490a808fc5a53452dc5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:34.350222 21704 ts_tablet_manager.cc:1428] T 5717a02ef53a4b22a96f431ca6c20d93 P 787ddcc824c2490a808fc5a53452dc5e: Time spent starting tablet: real 0.048s user 0.005s sys 0.000s
I20250109 08:57:34.351223 21704 tablet_bootstrap.cc:492] T edc9db96e0ff4aa5839a6922f3eb5f24 P 787ddcc824c2490a808fc5a53452dc5e: Bootstrap starting.
I20250109 08:57:34.351219 21702 ts_tablet_manager.cc:1428] T d015a10f95eb4f19b6d7c53cb201fa80 P a2b1a692435241ee988e0bd1661cd4c3: Time spent starting tablet: real 0.050s user 0.003s sys 0.003s
I20250109 08:57:34.355641 21703 tablet_bootstrap.cc:654] T 7c19ed9fdde5463f84e43bea7b860d03 P 8bf6e26e98b14566899578a990b27d47: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:34.364351 21702 tablet_bootstrap.cc:492] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P a2b1a692435241ee988e0bd1661cd4c3: Bootstrap starting.
I20250109 08:57:34.358793 22036 consensus_queue.cc:1035] T 126a163a89c448be89ccb9490e9f0168 P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20250109 08:57:34.367954 21511 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64c667f887ea41b4a484ee565b21fa16" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47"
I20250109 08:57:34.372294 21989 raft_consensus.cc:491] T d015a10f95eb4f19b6d7c53cb201fa80 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:34.373193 21989 raft_consensus.cc:513] T d015a10f95eb4f19b6d7c53cb201fa80 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:34.377398 21989 leader_election.cc:290] T d015a10f95eb4f19b6d7c53cb201fa80 P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:34.379655 21512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d015a10f95eb4f19b6d7c53cb201fa80" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47" is_pre_election: true
I20250109 08:57:34.380398 21512 raft_consensus.cc:2463] T d015a10f95eb4f19b6d7c53cb201fa80 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2b1a692435241ee988e0bd1661cd4c3 in term 0.
I20250109 08:57:34.381326 21552 leader_election.cc:304] T d015a10f95eb4f19b6d7c53cb201fa80 P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3; no voters:
I20250109 08:57:34.382015 21989 raft_consensus.cc:2798] T d015a10f95eb4f19b6d7c53cb201fa80 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:57:34.382344 21989 raft_consensus.cc:491] T d015a10f95eb4f19b6d7c53cb201fa80 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:57:34.382643 21989 raft_consensus.cc:3054] T d015a10f95eb4f19b6d7c53cb201fa80 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:34.382941 21511 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d015a10f95eb4f19b6d7c53cb201fa80" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47"
I20250109 08:57:34.383493 21511 raft_consensus.cc:3054] T d015a10f95eb4f19b6d7c53cb201fa80 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Advancing to term 1
W20250109 08:57:34.447124 22064 log.cc:926] Time spent T d52f79ecf037457cbcc09d5fa43456cf P a2b1a692435241ee988e0bd1661cd4c3: Append to log took a long time: real 0.134s user 0.000s sys 0.000s
I20250109 08:57:34.403074 21658 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d015a10f95eb4f19b6d7c53cb201fa80" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e" is_pre_election: true
I20250109 08:57:34.406280 21989 raft_consensus.cc:513] T d015a10f95eb4f19b6d7c53cb201fa80 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:34.469022 21658 raft_consensus.cc:2388] T d015a10f95eb4f19b6d7c53cb201fa80 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2b1a692435241ee988e0bd1661cd4c3 in current term 1: Already voted for candidate 787ddcc824c2490a808fc5a53452dc5e in this term.
W20250109 08:57:34.426203 22063 log.cc:926] Time spent T ab6c3fcdf226433c9e1303004f6c96b4 P 787ddcc824c2490a808fc5a53452dc5e: Append to log took a long time: real 0.100s user 0.000s sys 0.001s
I20250109 08:57:34.385730 21410 catalog_manager.cc:5526] T 64c667f887ea41b4a484ee565b21fa16 P 787ddcc824c2490a808fc5a53452dc5e reported cstate change: term changed from 0 to 1, leader changed from <none> to 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195). New cstate: current_term: 1 leader_uuid: "787ddcc824c2490a808fc5a53452dc5e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: HEALTHY } } }
I20250109 08:57:34.467377 21703 tablet_bootstrap.cc:492] T 7c19ed9fdde5463f84e43bea7b860d03 P 8bf6e26e98b14566899578a990b27d47: No bootstrap required, opened a new log
I20250109 08:57:34.475704 21703 ts_tablet_manager.cc:1397] T 7c19ed9fdde5463f84e43bea7b860d03 P 8bf6e26e98b14566899578a990b27d47: Time spent bootstrapping tablet: real 0.238s user 0.018s sys 0.018s
I20250109 08:57:34.479081 21511 raft_consensus.cc:2463] T d015a10f95eb4f19b6d7c53cb201fa80 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 1.
I20250109 08:57:34.478933 21703 raft_consensus.cc:357] T 7c19ed9fdde5463f84e43bea7b860d03 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:34.479629 21703 raft_consensus.cc:383] T 7c19ed9fdde5463f84e43bea7b860d03 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:34.479938 21703 raft_consensus.cc:738] T 7c19ed9fdde5463f84e43bea7b860d03 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bf6e26e98b14566899578a990b27d47, State: Initialized, Role: FOLLOWER
I20250109 08:57:34.479806 21989 leader_election.cc:290] T d015a10f95eb4f19b6d7c53cb201fa80 P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 election: Requested vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:34.480872 21703 consensus_queue.cc:260] T 7c19ed9fdde5463f84e43bea7b860d03 P 8bf6e26e98b14566899578a990b27d47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:34.482024 22047 consensus_queue.cc:1035] T 8c0459d2eee64f0a83bd05ad5974f506 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:57:34.463511 21704 tablet_bootstrap.cc:654] T edc9db96e0ff4aa5839a6922f3eb5f24 P 787ddcc824c2490a808fc5a53452dc5e: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:34.485363 21703 ts_tablet_manager.cc:1428] T 7c19ed9fdde5463f84e43bea7b860d03 P 8bf6e26e98b14566899578a990b27d47: Time spent starting tablet: real 0.009s user 0.006s sys 0.000s
I20250109 08:57:34.486073 21658 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d015a10f95eb4f19b6d7c53cb201fa80" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e"
I20250109 08:57:34.487061 21658 raft_consensus.cc:2388] T d015a10f95eb4f19b6d7c53cb201fa80 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate a2b1a692435241ee988e0bd1661cd4c3 in current term 1: Already voted for candidate 787ddcc824c2490a808fc5a53452dc5e in this term.
I20250109 08:57:34.493281 22044 consensus_queue.cc:1035] T d52f79ecf037457cbcc09d5fa43456cf P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:57:34.464574 21702 tablet_bootstrap.cc:654] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P a2b1a692435241ee988e0bd1661cd4c3: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:34.524884 21511 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d015a10f95eb4f19b6d7c53cb201fa80" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47"
I20250109 08:57:34.530702 21511 raft_consensus.cc:2388] T d015a10f95eb4f19b6d7c53cb201fa80 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate a2b1a692435241ee988e0bd1661cd4c3 in current term 1: Already voted for candidate 787ddcc824c2490a808fc5a53452dc5e in this term.
I20250109 08:57:34.527082 21907 consensus_queue.cc:1035] T 8c0459d2eee64f0a83bd05ad5974f506 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20250109 08:57:34.528060 21626 leader_election.cc:343] T 64c667f887ea41b4a484ee565b21fa16 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:34.549403 21552 leader_election.cc:304] T d015a10f95eb4f19b6d7c53cb201fa80 P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2b1a692435241ee988e0bd1661cd4c3; no voters: 787ddcc824c2490a808fc5a53452dc5e, 8bf6e26e98b14566899578a990b27d47
I20250109 08:57:34.550346 21989 raft_consensus.cc:2743] T d015a10f95eb4f19b6d7c53cb201fa80 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20250109 08:57:34.556133 21703 tablet_bootstrap.cc:492] T 64c667f887ea41b4a484ee565b21fa16 P 8bf6e26e98b14566899578a990b27d47: Bootstrap starting.
I20250109 08:57:34.560292 21626 leader_election.cc:304] T d015a10f95eb4f19b6d7c53cb201fa80 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, 8bf6e26e98b14566899578a990b27d47; no voters: a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:57:34.561028 21907 raft_consensus.cc:2798] T d015a10f95eb4f19b6d7c53cb201fa80 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:57:34.561441 21907 raft_consensus.cc:695] T d015a10f95eb4f19b6d7c53cb201fa80 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Becoming Leader. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Running, Role: LEADER
I20250109 08:57:34.562119 21907 consensus_queue.cc:237] T d015a10f95eb4f19b6d7c53cb201fa80 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:34.576061 21702 tablet_bootstrap.cc:492] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P a2b1a692435241ee988e0bd1661cd4c3: No bootstrap required, opened a new log
I20250109 08:57:34.576563 21702 ts_tablet_manager.cc:1397] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P a2b1a692435241ee988e0bd1661cd4c3: Time spent bootstrapping tablet: real 0.212s user 0.006s sys 0.013s
I20250109 08:57:34.579046 21702 raft_consensus.cc:357] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:34.579723 21702 raft_consensus.cc:383] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:34.579973 21702 raft_consensus.cc:738] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2b1a692435241ee988e0bd1661cd4c3, State: Initialized, Role: FOLLOWER
I20250109 08:57:34.580669 21702 consensus_queue.cc:260] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P a2b1a692435241ee988e0bd1661cd4c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:34.582945 21702 ts_tablet_manager.cc:1428] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P a2b1a692435241ee988e0bd1661cd4c3: Time spent starting tablet: real 0.006s user 0.004s sys 0.001s
I20250109 08:57:34.583793 21702 tablet_bootstrap.cc:492] T 1b582ab8d8f044d9b9038a28e958feeb P a2b1a692435241ee988e0bd1661cd4c3: Bootstrap starting.
I20250109 08:57:34.592700 21704 tablet_bootstrap.cc:492] T edc9db96e0ff4aa5839a6922f3eb5f24 P 787ddcc824c2490a808fc5a53452dc5e: No bootstrap required, opened a new log
I20250109 08:57:34.593154 21704 ts_tablet_manager.cc:1397] T edc9db96e0ff4aa5839a6922f3eb5f24 P 787ddcc824c2490a808fc5a53452dc5e: Time spent bootstrapping tablet: real 0.242s user 0.034s sys 0.021s
I20250109 08:57:34.605633 21703 tablet_bootstrap.cc:654] T 64c667f887ea41b4a484ee565b21fa16 P 8bf6e26e98b14566899578a990b27d47: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:34.609154 21702 tablet_bootstrap.cc:654] T 1b582ab8d8f044d9b9038a28e958feeb P a2b1a692435241ee988e0bd1661cd4c3: Neither blocks nor log segments found. Creating new log.
W20250109 08:57:34.617671 21460 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20250109 08:57:34.622910 21703 tablet_bootstrap.cc:492] T 64c667f887ea41b4a484ee565b21fa16 P 8bf6e26e98b14566899578a990b27d47: No bootstrap required, opened a new log
I20250109 08:57:34.624209 21703 ts_tablet_manager.cc:1397] T 64c667f887ea41b4a484ee565b21fa16 P 8bf6e26e98b14566899578a990b27d47: Time spent bootstrapping tablet: real 0.138s user 0.009s sys 0.002s
I20250109 08:57:34.629812 21704 raft_consensus.cc:357] T edc9db96e0ff4aa5839a6922f3eb5f24 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:34.630481 21704 raft_consensus.cc:383] T edc9db96e0ff4aa5839a6922f3eb5f24 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:34.630703 21704 raft_consensus.cc:738] T edc9db96e0ff4aa5839a6922f3eb5f24 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Initialized, Role: FOLLOWER
I20250109 08:57:34.631199 21704 consensus_queue.cc:260] T edc9db96e0ff4aa5839a6922f3eb5f24 P 787ddcc824c2490a808fc5a53452dc5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:34.632989 21704 ts_tablet_manager.cc:1428] T edc9db96e0ff4aa5839a6922f3eb5f24 P 787ddcc824c2490a808fc5a53452dc5e: Time spent starting tablet: real 0.040s user 0.004s sys 0.002s
I20250109 08:57:34.627254 21703 raft_consensus.cc:357] T 64c667f887ea41b4a484ee565b21fa16 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:34.633322 21703 raft_consensus.cc:383] T 64c667f887ea41b4a484ee565b21fa16 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:34.633587 21703 raft_consensus.cc:738] T 64c667f887ea41b4a484ee565b21fa16 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bf6e26e98b14566899578a990b27d47, State: Initialized, Role: FOLLOWER
I20250109 08:57:34.633812 21704 tablet_bootstrap.cc:492] T f5f85ad7c7c94acfaf9d550d55fce48c P 787ddcc824c2490a808fc5a53452dc5e: Bootstrap starting.
I20250109 08:57:34.635751 21702 tablet_bootstrap.cc:492] T 1b582ab8d8f044d9b9038a28e958feeb P a2b1a692435241ee988e0bd1661cd4c3: No bootstrap required, opened a new log
I20250109 08:57:34.636170 21702 ts_tablet_manager.cc:1397] T 1b582ab8d8f044d9b9038a28e958feeb P a2b1a692435241ee988e0bd1661cd4c3: Time spent bootstrapping tablet: real 0.053s user 0.010s sys 0.000s
I20250109 08:57:34.639076 21704 tablet_bootstrap.cc:654] T f5f85ad7c7c94acfaf9d550d55fce48c P 787ddcc824c2490a808fc5a53452dc5e: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:34.638736 21702 raft_consensus.cc:357] T 1b582ab8d8f044d9b9038a28e958feeb P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:34.639415 21702 raft_consensus.cc:383] T 1b582ab8d8f044d9b9038a28e958feeb P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:34.639662 21702 raft_consensus.cc:738] T 1b582ab8d8f044d9b9038a28e958feeb P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2b1a692435241ee988e0bd1661cd4c3, State: Initialized, Role: FOLLOWER
I20250109 08:57:34.634322 21703 consensus_queue.cc:260] T 64c667f887ea41b4a484ee565b21fa16 P 8bf6e26e98b14566899578a990b27d47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:34.642889 21703 ts_tablet_manager.cc:1428] T 64c667f887ea41b4a484ee565b21fa16 P 8bf6e26e98b14566899578a990b27d47: Time spent starting tablet: real 0.018s user 0.005s sys 0.001s
I20250109 08:57:34.640328 21702 consensus_queue.cc:260] T 1b582ab8d8f044d9b9038a28e958feeb P a2b1a692435241ee988e0bd1661cd4c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:34.643750 21703 tablet_bootstrap.cc:492] T 90a4a2381cbe40f0b204be325ee9fea1 P 8bf6e26e98b14566899578a990b27d47: Bootstrap starting.
I20250109 08:57:34.650401 21702 ts_tablet_manager.cc:1428] T 1b582ab8d8f044d9b9038a28e958feeb P a2b1a692435241ee988e0bd1661cd4c3: Time spent starting tablet: real 0.014s user 0.006s sys 0.000s
I20250109 08:57:34.651388 21702 tablet_bootstrap.cc:492] T 56bbc306275e4a409e028a7ca1dd1602 P a2b1a692435241ee988e0bd1661cd4c3: Bootstrap starting.
I20250109 08:57:34.661377 21703 tablet_bootstrap.cc:654] T 90a4a2381cbe40f0b204be325ee9fea1 P 8bf6e26e98b14566899578a990b27d47: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:34.662715 21781 raft_consensus.cc:491] T 7c19ed9fdde5463f84e43bea7b860d03 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:34.663199 21781 raft_consensus.cc:513] T 7c19ed9fdde5463f84e43bea7b860d03 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:34.665704 21702 tablet_bootstrap.cc:654] T 56bbc306275e4a409e028a7ca1dd1602 P a2b1a692435241ee988e0bd1661cd4c3: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:34.665930 22050 raft_consensus.cc:491] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:34.666379 22050 raft_consensus.cc:513] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:34.668756 22050 leader_election.cc:290] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:34.670507 21412 catalog_manager.cc:5526] T d015a10f95eb4f19b6d7c53cb201fa80 P 787ddcc824c2490a808fc5a53452dc5e reported cstate change: term changed from 0 to 1, leader changed from <none> to 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195). New cstate: current_term: 1 leader_uuid: "787ddcc824c2490a808fc5a53452dc5e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: HEALTHY } } }
I20250109 08:57:34.671777 21658 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d31dbfb2b4a4a6ca4500f92df9dbb98" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e" is_pre_election: true
I20250109 08:57:34.672389 21658 raft_consensus.cc:2463] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8bf6e26e98b14566899578a990b27d47 in term 0.
I20250109 08:57:34.673719 21657 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c19ed9fdde5463f84e43bea7b860d03" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e" is_pre_election: true
I20250109 08:57:34.675827 21477 leader_election.cc:304] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, 8bf6e26e98b14566899578a990b27d47; no voters:
W20250109 08:57:34.676661 21477 leader_election.cc:343] T 7c19ed9fdde5463f84e43bea7b860d03 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:34.680296 21586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c19ed9fdde5463f84e43bea7b860d03" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3" is_pre_election: true
I20250109 08:57:34.682384 21586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d31dbfb2b4a4a6ca4500f92df9dbb98" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3" is_pre_election: true
I20250109 08:57:34.682937 21586 raft_consensus.cc:2463] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8bf6e26e98b14566899578a990b27d47 in term 0.
I20250109 08:57:34.688299 21907 raft_consensus.cc:491] T edc9db96e0ff4aa5839a6922f3eb5f24 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:34.688732 21907 raft_consensus.cc:513] T edc9db96e0ff4aa5839a6922f3eb5f24 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:34.689249 22052 raft_consensus.cc:2798] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:57:34.689594 22052 raft_consensus.cc:491] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:57:34.689899 22052 raft_consensus.cc:3054] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:34.690761 21907 leader_election.cc:290] T edc9db96e0ff4aa5839a6922f3eb5f24 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
I20250109 08:57:34.692734 21511 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "edc9db96e0ff4aa5839a6922f3eb5f24" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47" is_pre_election: true
I20250109 08:57:34.692903 21585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "edc9db96e0ff4aa5839a6922f3eb5f24" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3" is_pre_election: true
I20250109 08:57:34.665638 21781 leader_election.cc:290] T 7c19ed9fdde5463f84e43bea7b860d03 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
W20250109 08:57:34.694169 21626 leader_election.cc:343] T edc9db96e0ff4aa5839a6922f3eb5f24 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Illegal state: must be running to vote when last-logged opid is not known
W20250109 08:57:34.693434 21478 leader_election.cc:343] T 7c19ed9fdde5463f84e43bea7b860d03 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:34.701925 21478 leader_election.cc:304] T 7c19ed9fdde5463f84e43bea7b860d03 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8bf6e26e98b14566899578a990b27d47; no voters: 787ddcc824c2490a808fc5a53452dc5e, a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:57:34.705535 21781 raft_consensus.cc:2743] T 7c19ed9fdde5463f84e43bea7b860d03 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:57:34.705215 22052 raft_consensus.cc:513] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
W20250109 08:57:34.697572 21627 leader_election.cc:343] T edc9db96e0ff4aa5839a6922f3eb5f24 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:34.706880 21627 leader_election.cc:304] T edc9db96e0ff4aa5839a6922f3eb5f24 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e; no voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:57:34.708434 21907 raft_consensus.cc:2743] T edc9db96e0ff4aa5839a6922f3eb5f24 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:57:34.708755 22052 leader_election.cc:290] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 election: Requested vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:34.714784 21704 tablet_bootstrap.cc:492] T f5f85ad7c7c94acfaf9d550d55fce48c P 787ddcc824c2490a808fc5a53452dc5e: No bootstrap required, opened a new log
I20250109 08:57:34.715286 21704 ts_tablet_manager.cc:1397] T f5f85ad7c7c94acfaf9d550d55fce48c P 787ddcc824c2490a808fc5a53452dc5e: Time spent bootstrapping tablet: real 0.082s user 0.030s sys 0.047s
I20250109 08:57:34.717684 21704 raft_consensus.cc:357] T f5f85ad7c7c94acfaf9d550d55fce48c P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:34.718376 21704 raft_consensus.cc:383] T f5f85ad7c7c94acfaf9d550d55fce48c P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:34.718614 21704 raft_consensus.cc:738] T f5f85ad7c7c94acfaf9d550d55fce48c P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Initialized, Role: FOLLOWER
I20250109 08:57:34.719180 21586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d31dbfb2b4a4a6ca4500f92df9dbb98" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3"
I20250109 08:57:34.719784 21586 raft_consensus.cc:3054] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:34.719184 21704 consensus_queue.cc:260] T f5f85ad7c7c94acfaf9d550d55fce48c P 787ddcc824c2490a808fc5a53452dc5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:34.721957 21703 tablet_bootstrap.cc:492] T 90a4a2381cbe40f0b204be325ee9fea1 P 8bf6e26e98b14566899578a990b27d47: No bootstrap required, opened a new log
I20250109 08:57:34.722373 21703 ts_tablet_manager.cc:1397] T 90a4a2381cbe40f0b204be325ee9fea1 P 8bf6e26e98b14566899578a990b27d47: Time spent bootstrapping tablet: real 0.079s user 0.010s sys 0.024s
I20250109 08:57:34.722860 21658 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d31dbfb2b4a4a6ca4500f92df9dbb98" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e"
I20250109 08:57:34.723455 21658 raft_consensus.cc:3054] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:34.724658 21703 raft_consensus.cc:357] T 90a4a2381cbe40f0b204be325ee9fea1 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:34.725281 21702 tablet_bootstrap.cc:492] T 56bbc306275e4a409e028a7ca1dd1602 P a2b1a692435241ee988e0bd1661cd4c3: No bootstrap required, opened a new log
I20250109 08:57:34.725731 21702 ts_tablet_manager.cc:1397] T 56bbc306275e4a409e028a7ca1dd1602 P a2b1a692435241ee988e0bd1661cd4c3: Time spent bootstrapping tablet: real 0.075s user 0.025s sys 0.010s
I20250109 08:57:34.725302 21703 raft_consensus.cc:383] T 90a4a2381cbe40f0b204be325ee9fea1 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:34.728706 21703 raft_consensus.cc:738] T 90a4a2381cbe40f0b204be325ee9fea1 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bf6e26e98b14566899578a990b27d47, State: Initialized, Role: FOLLOWER
I20250109 08:57:34.729362 21703 consensus_queue.cc:260] T 90a4a2381cbe40f0b204be325ee9fea1 P 8bf6e26e98b14566899578a990b27d47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:34.731619 21703 ts_tablet_manager.cc:1428] T 90a4a2381cbe40f0b204be325ee9fea1 P 8bf6e26e98b14566899578a990b27d47: Time spent starting tablet: real 0.009s user 0.003s sys 0.003s
I20250109 08:57:34.728001 21702 raft_consensus.cc:357] T 56bbc306275e4a409e028a7ca1dd1602 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:34.732712 21702 raft_consensus.cc:383] T 56bbc306275e4a409e028a7ca1dd1602 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:34.732991 21702 raft_consensus.cc:738] T 56bbc306275e4a409e028a7ca1dd1602 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2b1a692435241ee988e0bd1661cd4c3, State: Initialized, Role: FOLLOWER
I20250109 08:57:34.733513 21703 tablet_bootstrap.cc:492] T edc9db96e0ff4aa5839a6922f3eb5f24 P 8bf6e26e98b14566899578a990b27d47: Bootstrap starting.
I20250109 08:57:34.735116 21586 raft_consensus.cc:2463] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8bf6e26e98b14566899578a990b27d47 in term 1.
I20250109 08:57:34.723096 21704 ts_tablet_manager.cc:1428] T f5f85ad7c7c94acfaf9d550d55fce48c P 787ddcc824c2490a808fc5a53452dc5e: Time spent starting tablet: real 0.008s user 0.005s sys 0.001s
I20250109 08:57:34.736796 21478 leader_election.cc:304] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3; no voters:
I20250109 08:57:34.737103 21704 tablet_bootstrap.cc:492] T 7c19ed9fdde5463f84e43bea7b860d03 P 787ddcc824c2490a808fc5a53452dc5e: Bootstrap starting.
I20250109 08:57:34.737529 21781 raft_consensus.cc:2798] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:57:34.737962 21781 raft_consensus.cc:695] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Becoming Leader. State: Replica: 8bf6e26e98b14566899578a990b27d47, State: Running, Role: LEADER
I20250109 08:57:34.738883 21512 raft_consensus.cc:3054] T 90a4a2381cbe40f0b204be325ee9fea1 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:34.739581 21702 consensus_queue.cc:260] T 56bbc306275e4a409e028a7ca1dd1602 P a2b1a692435241ee988e0bd1661cd4c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:34.738687 21781 consensus_queue.cc:237] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:34.742574 21704 tablet_bootstrap.cc:654] T 7c19ed9fdde5463f84e43bea7b860d03 P 787ddcc824c2490a808fc5a53452dc5e: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:34.750233 21702 ts_tablet_manager.cc:1428] T 56bbc306275e4a409e028a7ca1dd1602 P a2b1a692435241ee988e0bd1661cd4c3: Time spent starting tablet: real 0.024s user 0.005s sys 0.000s
I20250109 08:57:34.751204 21702 tablet_bootstrap.cc:492] T 5717a02ef53a4b22a96f431ca6c20d93 P a2b1a692435241ee988e0bd1661cd4c3: Bootstrap starting.
I20250109 08:57:34.760164 22052 raft_consensus.cc:491] T 56bbc306275e4a409e028a7ca1dd1602 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:34.760807 22052 raft_consensus.cc:513] T 56bbc306275e4a409e028a7ca1dd1602 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:34.763962 21586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56bbc306275e4a409e028a7ca1dd1602" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3" is_pre_election: true
I20250109 08:57:34.764608 21586 raft_consensus.cc:2463] T 56bbc306275e4a409e028a7ca1dd1602 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8bf6e26e98b14566899578a990b27d47 in term 0.
I20250109 08:57:34.764989 21907 raft_consensus.cc:491] T 5717a02ef53a4b22a96f431ca6c20d93 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:34.765393 21907 raft_consensus.cc:513] T 5717a02ef53a4b22a96f431ca6c20d93 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:34.767294 21702 tablet_bootstrap.cc:654] T 5717a02ef53a4b22a96f431ca6c20d93 P a2b1a692435241ee988e0bd1661cd4c3: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:34.767634 21511 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5717a02ef53a4b22a96f431ca6c20d93" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47" is_pre_election: true
I20250109 08:57:34.767791 21478 leader_election.cc:304] T 56bbc306275e4a409e028a7ca1dd1602 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3; no voters:
I20250109 08:57:34.768323 21511 raft_consensus.cc:2463] T 5717a02ef53a4b22a96f431ca6c20d93 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 0.
I20250109 08:57:34.768759 22050 raft_consensus.cc:2798] T 56bbc306275e4a409e028a7ca1dd1602 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:57:34.769132 22050 raft_consensus.cc:491] T 56bbc306275e4a409e028a7ca1dd1602 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:57:34.757790 21703 tablet_bootstrap.cc:654] T edc9db96e0ff4aa5839a6922f3eb5f24 P 8bf6e26e98b14566899578a990b27d47: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:34.769385 21626 leader_election.cc:304] T 5717a02ef53a4b22a96f431ca6c20d93 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, 8bf6e26e98b14566899578a990b27d47; no voters:
I20250109 08:57:34.769443 22050 raft_consensus.cc:3054] T 56bbc306275e4a409e028a7ca1dd1602 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:34.770900 22047 raft_consensus.cc:2798] T 5717a02ef53a4b22a96f431ca6c20d93 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:57:34.771255 22047 raft_consensus.cc:491] T 5717a02ef53a4b22a96f431ca6c20d93 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:57:34.771560 22047 raft_consensus.cc:3054] T 5717a02ef53a4b22a96f431ca6c20d93 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:34.775408 21658 raft_consensus.cc:2463] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8bf6e26e98b14566899578a990b27d47 in term 1.
I20250109 08:57:34.775614 21657 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56bbc306275e4a409e028a7ca1dd1602" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e" is_pre_election: true
W20250109 08:57:34.780256 21477 leader_election.cc:343] T 56bbc306275e4a409e028a7ca1dd1602 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:34.775930 22052 leader_election.cc:290] T 56bbc306275e4a409e028a7ca1dd1602 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:34.781337 21703 tablet_bootstrap.cc:492] T edc9db96e0ff4aa5839a6922f3eb5f24 P 8bf6e26e98b14566899578a990b27d47: No bootstrap required, opened a new log
I20250109 08:57:34.781785 21703 ts_tablet_manager.cc:1397] T edc9db96e0ff4aa5839a6922f3eb5f24 P 8bf6e26e98b14566899578a990b27d47: Time spent bootstrapping tablet: real 0.049s user 0.017s sys 0.000s
I20250109 08:57:34.777828 22050 raft_consensus.cc:513] T 56bbc306275e4a409e028a7ca1dd1602 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:34.783756 21710 consensus_queue.cc:1035] T 64c667f887ea41b4a484ee565b21fa16 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:57:34.787106 21585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56bbc306275e4a409e028a7ca1dd1602" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3"
I20250109 08:57:34.788079 21585 raft_consensus.cc:3054] T 56bbc306275e4a409e028a7ca1dd1602 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:34.789952 21511 raft_consensus.cc:3054] T 64c667f887ea41b4a484ee565b21fa16 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:34.797345 21710 consensus_queue.cc:1035] T 64c667f887ea41b4a484ee565b21fa16 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:57:34.798030 21703 raft_consensus.cc:357] T edc9db96e0ff4aa5839a6922f3eb5f24 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:34.798785 21703 raft_consensus.cc:383] T edc9db96e0ff4aa5839a6922f3eb5f24 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:34.799057 21703 raft_consensus.cc:738] T edc9db96e0ff4aa5839a6922f3eb5f24 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bf6e26e98b14566899578a990b27d47, State: Initialized, Role: FOLLOWER
I20250109 08:57:34.802268 21703 consensus_queue.cc:260] T edc9db96e0ff4aa5839a6922f3eb5f24 P 8bf6e26e98b14566899578a990b27d47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:34.803148 21907 leader_election.cc:290] T 5717a02ef53a4b22a96f431ca6c20d93 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
I20250109 08:57:34.803773 21588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5717a02ef53a4b22a96f431ca6c20d93" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3" is_pre_election: true
W20250109 08:57:34.805095 21627 leader_election.cc:343] T 5717a02ef53a4b22a96f431ca6c20d93 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:34.807572 21585 raft_consensus.cc:2463] T 56bbc306275e4a409e028a7ca1dd1602 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8bf6e26e98b14566899578a990b27d47 in term 1.
I20250109 08:57:34.810742 22047 raft_consensus.cc:513] T 5717a02ef53a4b22a96f431ca6c20d93 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:34.813288 21510 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5717a02ef53a4b22a96f431ca6c20d93" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47"
I20250109 08:57:34.813834 21510 raft_consensus.cc:3054] T 5717a02ef53a4b22a96f431ca6c20d93 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:34.819895 21703 ts_tablet_manager.cc:1428] T edc9db96e0ff4aa5839a6922f3eb5f24 P 8bf6e26e98b14566899578a990b27d47: Time spent starting tablet: real 0.038s user 0.008s sys 0.000s
I20250109 08:57:34.820890 21703 tablet_bootstrap.cc:492] T 2935d69446cd458e92046d2b2f1b74ab P 8bf6e26e98b14566899578a990b27d47: Bootstrap starting.
I20250109 08:57:34.824141 21702 tablet_bootstrap.cc:492] T 5717a02ef53a4b22a96f431ca6c20d93 P a2b1a692435241ee988e0bd1661cd4c3: No bootstrap required, opened a new log
I20250109 08:57:34.824448 22029 raft_consensus.cc:491] T 1b582ab8d8f044d9b9038a28e958feeb P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:34.824926 22029 raft_consensus.cc:513] T 1b582ab8d8f044d9b9038a28e958feeb P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:34.824577 21702 ts_tablet_manager.cc:1397] T 5717a02ef53a4b22a96f431ca6c20d93 P a2b1a692435241ee988e0bd1661cd4c3: Time spent bootstrapping tablet: real 0.074s user 0.017s sys 0.014s
I20250109 08:57:34.827934 21657 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b582ab8d8f044d9b9038a28e958feeb" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e" is_pre_election: true
I20250109 08:57:34.829103 21703 tablet_bootstrap.cc:654] T 2935d69446cd458e92046d2b2f1b74ab P 8bf6e26e98b14566899578a990b27d47: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:34.829418 22029 leader_election.cc:290] T 1b582ab8d8f044d9b9038a28e958feeb P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:34.830196 21702 raft_consensus.cc:357] T 5717a02ef53a4b22a96f431ca6c20d93 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:34.830439 21513 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b582ab8d8f044d9b9038a28e958feeb" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47" is_pre_election: true
I20250109 08:57:34.830916 21702 raft_consensus.cc:383] T 5717a02ef53a4b22a96f431ca6c20d93 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:34.831204 21702 raft_consensus.cc:738] T 5717a02ef53a4b22a96f431ca6c20d93 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2b1a692435241ee988e0bd1661cd4c3, State: Initialized, Role: FOLLOWER
I20250109 08:57:34.831673 21552 leader_election.cc:304] T 1b582ab8d8f044d9b9038a28e958feeb P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2b1a692435241ee988e0bd1661cd4c3; no voters: 787ddcc824c2490a808fc5a53452dc5e, 8bf6e26e98b14566899578a990b27d47
I20250109 08:57:34.831804 21702 consensus_queue.cc:260] T 5717a02ef53a4b22a96f431ca6c20d93 P a2b1a692435241ee988e0bd1661cd4c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:34.832274 21510 raft_consensus.cc:2463] T 5717a02ef53a4b22a96f431ca6c20d93 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 1.
I20250109 08:57:34.834642 21626 leader_election.cc:304] T 5717a02ef53a4b22a96f431ca6c20d93 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, 8bf6e26e98b14566899578a990b27d47; no voters:
I20250109 08:57:34.835973 21710 raft_consensus.cc:2798] T 5717a02ef53a4b22a96f431ca6c20d93 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:57:34.836568 22036 raft_consensus.cc:2743] T 1b582ab8d8f044d9b9038a28e958feeb P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:57:34.837941 21702 ts_tablet_manager.cc:1428] T 5717a02ef53a4b22a96f431ca6c20d93 P a2b1a692435241ee988e0bd1661cd4c3: Time spent starting tablet: real 0.012s user 0.005s sys 0.000s
I20250109 08:57:34.838074 21710 raft_consensus.cc:695] T 5717a02ef53a4b22a96f431ca6c20d93 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Becoming Leader. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Running, Role: LEADER
I20250109 08:57:34.839043 21710 consensus_queue.cc:237] T 5717a02ef53a4b22a96f431ca6c20d93 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:34.840116 21703 tablet_bootstrap.cc:492] T 2935d69446cd458e92046d2b2f1b74ab P 8bf6e26e98b14566899578a990b27d47: No bootstrap required, opened a new log
I20250109 08:57:34.840778 21703 ts_tablet_manager.cc:1397] T 2935d69446cd458e92046d2b2f1b74ab P 8bf6e26e98b14566899578a990b27d47: Time spent bootstrapping tablet: real 0.020s user 0.007s sys 0.009s
I20250109 08:57:34.843211 21703 raft_consensus.cc:357] T 2935d69446cd458e92046d2b2f1b74ab P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:34.843904 21703 raft_consensus.cc:383] T 2935d69446cd458e92046d2b2f1b74ab P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:34.844156 21703 raft_consensus.cc:738] T 2935d69446cd458e92046d2b2f1b74ab P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bf6e26e98b14566899578a990b27d47, State: Initialized, Role: FOLLOWER
I20250109 08:57:34.846011 21703 consensus_queue.cc:260] T 2935d69446cd458e92046d2b2f1b74ab P 8bf6e26e98b14566899578a990b27d47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:34.787194 21658 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56bbc306275e4a409e028a7ca1dd1602" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e"
W20250109 08:57:34.851044 21477 leader_election.cc:343] T 56bbc306275e4a409e028a7ca1dd1602 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:57:34.787484 22050 leader_election.cc:290] T 56bbc306275e4a409e028a7ca1dd1602 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 election: Requested vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:34.848234 21703 ts_tablet_manager.cc:1428] T 2935d69446cd458e92046d2b2f1b74ab P 8bf6e26e98b14566899578a990b27d47: Time spent starting tablet: real 0.007s user 0.004s sys 0.000s
I20250109 08:57:34.855751 21702 tablet_bootstrap.cc:492] T 7c19ed9fdde5463f84e43bea7b860d03 P a2b1a692435241ee988e0bd1661cd4c3: Bootstrap starting.
W20250109 08:57:34.862665 21478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.226.194:45527, user_credentials={real_user=slave}} blocked reactor thread for 57448.1us
I20250109 08:57:34.864058 21478 leader_election.cc:304] T 56bbc306275e4a409e028a7ca1dd1602 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3; no voters: 787ddcc824c2490a808fc5a53452dc5e
I20250109 08:57:34.864727 22032 raft_consensus.cc:2798] T 56bbc306275e4a409e028a7ca1dd1602 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:57:34.865170 22032 raft_consensus.cc:695] T 56bbc306275e4a409e028a7ca1dd1602 P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Becoming Leader. State: Replica: 8bf6e26e98b14566899578a990b27d47, State: Running, Role: LEADER
I20250109 08:57:34.866080 21702 tablet_bootstrap.cc:654] T 7c19ed9fdde5463f84e43bea7b860d03 P a2b1a692435241ee988e0bd1661cd4c3: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:34.865916 22032 consensus_queue.cc:237] T 56bbc306275e4a409e028a7ca1dd1602 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:34.871647 21585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5717a02ef53a4b22a96f431ca6c20d93" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3"
I20250109 08:57:34.872380 21585 raft_consensus.cc:3054] T 5717a02ef53a4b22a96f431ca6c20d93 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:34.885352 21702 tablet_bootstrap.cc:492] T 7c19ed9fdde5463f84e43bea7b860d03 P a2b1a692435241ee988e0bd1661cd4c3: No bootstrap required, opened a new log
I20250109 08:57:34.885823 21702 ts_tablet_manager.cc:1397] T 7c19ed9fdde5463f84e43bea7b860d03 P a2b1a692435241ee988e0bd1661cd4c3: Time spent bootstrapping tablet: real 0.030s user 0.013s sys 0.000s
I20250109 08:57:34.886371 22047 leader_election.cc:290] T 5717a02ef53a4b22a96f431ca6c20d93 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 election: Requested vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
I20250109 08:57:34.897275 21702 raft_consensus.cc:357] T 7c19ed9fdde5463f84e43bea7b860d03 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:34.897991 21702 raft_consensus.cc:383] T 7c19ed9fdde5463f84e43bea7b860d03 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:34.898248 21702 raft_consensus.cc:738] T 7c19ed9fdde5463f84e43bea7b860d03 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2b1a692435241ee988e0bd1661cd4c3, State: Initialized, Role: FOLLOWER
I20250109 08:57:34.898806 21704 tablet_bootstrap.cc:492] T 7c19ed9fdde5463f84e43bea7b860d03 P 787ddcc824c2490a808fc5a53452dc5e: No bootstrap required, opened a new log
I20250109 08:57:34.899271 21704 ts_tablet_manager.cc:1397] T 7c19ed9fdde5463f84e43bea7b860d03 P 787ddcc824c2490a808fc5a53452dc5e: Time spent bootstrapping tablet: real 0.162s user 0.046s sys 0.028s
I20250109 08:57:34.899658 21585 raft_consensus.cc:2463] T 5717a02ef53a4b22a96f431ca6c20d93 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 1.
I20250109 08:57:34.898849 21702 consensus_queue.cc:260] T 7c19ed9fdde5463f84e43bea7b860d03 P a2b1a692435241ee988e0bd1661cd4c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:34.901820 21704 raft_consensus.cc:357] T 7c19ed9fdde5463f84e43bea7b860d03 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:34.902421 21702 ts_tablet_manager.cc:1428] T 7c19ed9fdde5463f84e43bea7b860d03 P a2b1a692435241ee988e0bd1661cd4c3: Time spent starting tablet: real 0.016s user 0.006s sys 0.000s
I20250109 08:57:34.902545 21704 raft_consensus.cc:383] T 7c19ed9fdde5463f84e43bea7b860d03 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:34.902964 21704 raft_consensus.cc:738] T 7c19ed9fdde5463f84e43bea7b860d03 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Initialized, Role: FOLLOWER
I20250109 08:57:34.903594 21704 consensus_queue.cc:260] T 7c19ed9fdde5463f84e43bea7b860d03 P 787ddcc824c2490a808fc5a53452dc5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:34.905726 21704 ts_tablet_manager.cc:1428] T 7c19ed9fdde5463f84e43bea7b860d03 P 787ddcc824c2490a808fc5a53452dc5e: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20250109 08:57:34.906544 21704 tablet_bootstrap.cc:492] T 1d3d5d2a2c88417a97c9d57262a356dd P 787ddcc824c2490a808fc5a53452dc5e: Bootstrap starting.
I20250109 08:57:34.910884 21704 tablet_bootstrap.cc:654] T 1d3d5d2a2c88417a97c9d57262a356dd P 787ddcc824c2490a808fc5a53452dc5e: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:34.912454 21412 catalog_manager.cc:5526] T 5717a02ef53a4b22a96f431ca6c20d93 P 787ddcc824c2490a808fc5a53452dc5e reported cstate change: term changed from 0 to 1, leader changed from <none> to 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195). New cstate: current_term: 1 leader_uuid: "787ddcc824c2490a808fc5a53452dc5e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: HEALTHY } } }
I20250109 08:57:34.913589 21702 tablet_bootstrap.cc:492] T edc9db96e0ff4aa5839a6922f3eb5f24 P a2b1a692435241ee988e0bd1661cd4c3: Bootstrap starting.
I20250109 08:57:34.919862 21702 tablet_bootstrap.cc:654] T edc9db96e0ff4aa5839a6922f3eb5f24 P a2b1a692435241ee988e0bd1661cd4c3: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:34.930899 21410 catalog_manager.cc:5526] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 8bf6e26e98b14566899578a990b27d47 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8bf6e26e98b14566899578a990b27d47 (127.19.226.193). New cstate: current_term: 1 leader_uuid: "8bf6e26e98b14566899578a990b27d47" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: UNKNOWN } } }
I20250109 08:57:34.932658 21410 catalog_manager.cc:5526] T 56bbc306275e4a409e028a7ca1dd1602 P 8bf6e26e98b14566899578a990b27d47 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8bf6e26e98b14566899578a990b27d47 (127.19.226.193). New cstate: current_term: 1 leader_uuid: "8bf6e26e98b14566899578a990b27d47" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: UNKNOWN } } }
I20250109 08:57:34.956584 21704 tablet_bootstrap.cc:492] T 1d3d5d2a2c88417a97c9d57262a356dd P 787ddcc824c2490a808fc5a53452dc5e: No bootstrap required, opened a new log
I20250109 08:57:34.957027 21704 ts_tablet_manager.cc:1397] T 1d3d5d2a2c88417a97c9d57262a356dd P 787ddcc824c2490a808fc5a53452dc5e: Time spent bootstrapping tablet: real 0.051s user 0.018s sys 0.028s
I20250109 08:57:34.959251 21704 raft_consensus.cc:357] T 1d3d5d2a2c88417a97c9d57262a356dd P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:34.959918 21704 raft_consensus.cc:383] T 1d3d5d2a2c88417a97c9d57262a356dd P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:34.960150 21704 raft_consensus.cc:738] T 1d3d5d2a2c88417a97c9d57262a356dd P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Initialized, Role: FOLLOWER
I20250109 08:57:34.960700 21704 consensus_queue.cc:260] T 1d3d5d2a2c88417a97c9d57262a356dd P 787ddcc824c2490a808fc5a53452dc5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:34.962548 21704 ts_tablet_manager.cc:1428] T 1d3d5d2a2c88417a97c9d57262a356dd P 787ddcc824c2490a808fc5a53452dc5e: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20250109 08:57:34.963729 21704 tablet_bootstrap.cc:492] T 56bbc306275e4a409e028a7ca1dd1602 P 787ddcc824c2490a808fc5a53452dc5e: Bootstrap starting.
I20250109 08:57:34.965119 21702 tablet_bootstrap.cc:492] T edc9db96e0ff4aa5839a6922f3eb5f24 P a2b1a692435241ee988e0bd1661cd4c3: No bootstrap required, opened a new log
I20250109 08:57:34.965560 21702 ts_tablet_manager.cc:1397] T edc9db96e0ff4aa5839a6922f3eb5f24 P a2b1a692435241ee988e0bd1661cd4c3: Time spent bootstrapping tablet: real 0.052s user 0.025s sys 0.009s
I20250109 08:57:34.966692 22047 raft_consensus.cc:491] T 1d3d5d2a2c88417a97c9d57262a356dd P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:34.967224 22047 raft_consensus.cc:513] T 1d3d5d2a2c88417a97c9d57262a356dd P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:34.969070 22047 leader_election.cc:290] T 1d3d5d2a2c88417a97c9d57262a356dd P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
I20250109 08:57:34.968863 21702 raft_consensus.cc:357] T edc9db96e0ff4aa5839a6922f3eb5f24 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:34.969518 21702 raft_consensus.cc:383] T edc9db96e0ff4aa5839a6922f3eb5f24 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:34.969769 21702 raft_consensus.cc:738] T edc9db96e0ff4aa5839a6922f3eb5f24 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2b1a692435241ee988e0bd1661cd4c3, State: Initialized, Role: FOLLOWER
I20250109 08:57:34.970392 21702 consensus_queue.cc:260] T edc9db96e0ff4aa5839a6922f3eb5f24 P a2b1a692435241ee988e0bd1661cd4c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:34.970901 21585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d3d5d2a2c88417a97c9d57262a356dd" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3" is_pre_election: true
I20250109 08:57:34.971669 21585 raft_consensus.cc:2388] T 1d3d5d2a2c88417a97c9d57262a356dd P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 787ddcc824c2490a808fc5a53452dc5e in current term 1: Already voted for candidate a2b1a692435241ee988e0bd1661cd4c3 in this term.
I20250109 08:57:34.971733 21704 tablet_bootstrap.cc:654] T 56bbc306275e4a409e028a7ca1dd1602 P 787ddcc824c2490a808fc5a53452dc5e: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:34.974400 21702 ts_tablet_manager.cc:1428] T edc9db96e0ff4aa5839a6922f3eb5f24 P a2b1a692435241ee988e0bd1661cd4c3: Time spent starting tablet: real 0.009s user 0.006s sys 0.000s
I20250109 08:57:34.975893 21512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d3d5d2a2c88417a97c9d57262a356dd" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47" is_pre_election: true
I20250109 08:57:34.976621 21512 raft_consensus.cc:2388] T 1d3d5d2a2c88417a97c9d57262a356dd P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 787ddcc824c2490a808fc5a53452dc5e in current term 1: Already voted for candidate 8bf6e26e98b14566899578a990b27d47 in this term.
I20250109 08:57:34.980194 21626 leader_election.cc:304] T 1d3d5d2a2c88417a97c9d57262a356dd P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e; no voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:57:34.981603 21710 raft_consensus.cc:3054] T 1d3d5d2a2c88417a97c9d57262a356dd P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:34.988992 21710 raft_consensus.cc:2743] T 1d3d5d2a2c88417a97c9d57262a356dd P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:57:34.991487 21704 tablet_bootstrap.cc:492] T 56bbc306275e4a409e028a7ca1dd1602 P 787ddcc824c2490a808fc5a53452dc5e: No bootstrap required, opened a new log
I20250109 08:57:34.991815 21704 ts_tablet_manager.cc:1397] T 56bbc306275e4a409e028a7ca1dd1602 P 787ddcc824c2490a808fc5a53452dc5e: Time spent bootstrapping tablet: real 0.028s user 0.010s sys 0.010s
I20250109 08:57:34.993484 21704 raft_consensus.cc:357] T 56bbc306275e4a409e028a7ca1dd1602 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:34.993953 21704 raft_consensus.cc:383] T 56bbc306275e4a409e028a7ca1dd1602 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:34.994134 21704 raft_consensus.cc:738] T 56bbc306275e4a409e028a7ca1dd1602 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Initialized, Role: FOLLOWER
I20250109 08:57:34.994534 21704 consensus_queue.cc:260] T 56bbc306275e4a409e028a7ca1dd1602 P 787ddcc824c2490a808fc5a53452dc5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:34.995931 21704 ts_tablet_manager.cc:1428] T 56bbc306275e4a409e028a7ca1dd1602 P 787ddcc824c2490a808fc5a53452dc5e: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20250109 08:57:34.996891 21704 tablet_bootstrap.cc:492] T 2935d69446cd458e92046d2b2f1b74ab P 787ddcc824c2490a808fc5a53452dc5e: Bootstrap starting.
I20250109 08:57:35.001358 21704 tablet_bootstrap.cc:654] T 2935d69446cd458e92046d2b2f1b74ab P 787ddcc824c2490a808fc5a53452dc5e: Neither blocks nor log segments found. Creating new log.
I20250109 08:57:35.005760 21704 tablet_bootstrap.cc:492] T 2935d69446cd458e92046d2b2f1b74ab P 787ddcc824c2490a808fc5a53452dc5e: No bootstrap required, opened a new log
I20250109 08:57:35.006219 21704 ts_tablet_manager.cc:1397] T 2935d69446cd458e92046d2b2f1b74ab P 787ddcc824c2490a808fc5a53452dc5e: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.002s
I20250109 08:57:35.008625 21704 raft_consensus.cc:357] T 2935d69446cd458e92046d2b2f1b74ab P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:35.009097 21704 raft_consensus.cc:383] T 2935d69446cd458e92046d2b2f1b74ab P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:57:35.009282 21704 raft_consensus.cc:738] T 2935d69446cd458e92046d2b2f1b74ab P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Initialized, Role: FOLLOWER
I20250109 08:57:35.009789 21704 consensus_queue.cc:260] T 2935d69446cd458e92046d2b2f1b74ab P 787ddcc824c2490a808fc5a53452dc5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:35.011544 21704 ts_tablet_manager.cc:1428] T 2935d69446cd458e92046d2b2f1b74ab P 787ddcc824c2490a808fc5a53452dc5e: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20250109 08:57:35.042842 22047 raft_consensus.cc:491] T f5f85ad7c7c94acfaf9d550d55fce48c P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:35.043243 22047 raft_consensus.cc:513] T f5f85ad7c7c94acfaf9d550d55fce48c P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:35.044875 22047 leader_election.cc:290] T f5f85ad7c7c94acfaf9d550d55fce48c P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
I20250109 08:57:35.045511 21512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5f85ad7c7c94acfaf9d550d55fce48c" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47" is_pre_election: true
I20250109 08:57:35.045758 21585 raft_consensus.cc:3054] T 1b582ab8d8f044d9b9038a28e958feeb P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:35.045727 21588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5f85ad7c7c94acfaf9d550d55fce48c" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3" is_pre_election: true
I20250109 08:57:35.046924 21627 leader_election.cc:304] T f5f85ad7c7c94acfaf9d550d55fce48c P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e; no voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:57:35.047447 22047 raft_consensus.cc:2743] T f5f85ad7c7c94acfaf9d550d55fce48c P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:57:35.058609 22047 consensus_queue.cc:1035] T d015a10f95eb4f19b6d7c53cb201fa80 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:57:35.068955 21710 consensus_queue.cc:1035] T d015a10f95eb4f19b6d7c53cb201fa80 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:57:35.083806 22092 raft_consensus.cc:491] T edc9db96e0ff4aa5839a6922f3eb5f24 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:35.083922 22032 raft_consensus.cc:491] T 2935d69446cd458e92046d2b2f1b74ab P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:35.084385 22032 raft_consensus.cc:513] T 2935d69446cd458e92046d2b2f1b74ab P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:35.084277 22092 raft_consensus.cc:513] T edc9db96e0ff4aa5839a6922f3eb5f24 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:35.085969 22032 leader_election.cc:290] T 2935d69446cd458e92046d2b2f1b74ab P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:35.086199 22092 leader_election.cc:290] T edc9db96e0ff4aa5839a6922f3eb5f24 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:35.086840 21588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2935d69446cd458e92046d2b2f1b74ab" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3" is_pre_election: true
I20250109 08:57:35.087215 21585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "edc9db96e0ff4aa5839a6922f3eb5f24" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3" is_pre_election: true
I20250109 08:57:35.087448 21588 raft_consensus.cc:2463] T 2935d69446cd458e92046d2b2f1b74ab P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8bf6e26e98b14566899578a990b27d47 in term 0.
I20250109 08:57:35.087540 21657 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "edc9db96e0ff4aa5839a6922f3eb5f24" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e" is_pre_election: true
I20250109 08:57:35.087853 21585 raft_consensus.cc:2463] T edc9db96e0ff4aa5839a6922f3eb5f24 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8bf6e26e98b14566899578a990b27d47 in term 0.
I20250109 08:57:35.088151 21657 raft_consensus.cc:2463] T edc9db96e0ff4aa5839a6922f3eb5f24 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8bf6e26e98b14566899578a990b27d47 in term 0.
I20250109 08:57:35.087620 21658 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2935d69446cd458e92046d2b2f1b74ab" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e" is_pre_election: true
I20250109 08:57:35.088728 21478 leader_election.cc:304] T 2935d69446cd458e92046d2b2f1b74ab P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3; no voters:
I20250109 08:57:35.088881 21658 raft_consensus.cc:2463] T 2935d69446cd458e92046d2b2f1b74ab P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8bf6e26e98b14566899578a990b27d47 in term 0.
I20250109 08:57:35.089164 21477 leader_election.cc:304] T edc9db96e0ff4aa5839a6922f3eb5f24 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, 8bf6e26e98b14566899578a990b27d47; no voters:
I20250109 08:57:35.089452 22032 raft_consensus.cc:2798] T 2935d69446cd458e92046d2b2f1b74ab P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:57:35.089931 22092 raft_consensus.cc:2798] T edc9db96e0ff4aa5839a6922f3eb5f24 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:57:35.090317 22092 raft_consensus.cc:491] T edc9db96e0ff4aa5839a6922f3eb5f24 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:57:35.090592 22032 raft_consensus.cc:491] T 2935d69446cd458e92046d2b2f1b74ab P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:57:35.090641 22092 raft_consensus.cc:3054] T edc9db96e0ff4aa5839a6922f3eb5f24 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:35.090981 22032 raft_consensus.cc:3054] T 2935d69446cd458e92046d2b2f1b74ab P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:35.098783 22032 raft_consensus.cc:513] T 2935d69446cd458e92046d2b2f1b74ab P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:35.099853 22092 raft_consensus.cc:513] T edc9db96e0ff4aa5839a6922f3eb5f24 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:35.101114 21585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2935d69446cd458e92046d2b2f1b74ab" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3"
I20250109 08:57:35.101254 21658 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2935d69446cd458e92046d2b2f1b74ab" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e"
I20250109 08:57:35.101830 21658 raft_consensus.cc:3054] T 2935d69446cd458e92046d2b2f1b74ab P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:35.101835 21585 raft_consensus.cc:3054] T 2935d69446cd458e92046d2b2f1b74ab P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:35.102200 21588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "edc9db96e0ff4aa5839a6922f3eb5f24" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3"
I20250109 08:57:35.102800 21588 raft_consensus.cc:3054] T edc9db96e0ff4aa5839a6922f3eb5f24 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:35.103178 21657 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "edc9db96e0ff4aa5839a6922f3eb5f24" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e"
I20250109 08:57:35.103719 21657 raft_consensus.cc:3054] T edc9db96e0ff4aa5839a6922f3eb5f24 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:35.107939 21585 raft_consensus.cc:2463] T 2935d69446cd458e92046d2b2f1b74ab P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8bf6e26e98b14566899578a990b27d47 in term 1.
I20250109 08:57:35.109163 21478 leader_election.cc:304] T 2935d69446cd458e92046d2b2f1b74ab P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3; no voters:
I20250109 08:57:35.109009 21588 raft_consensus.cc:2463] T edc9db96e0ff4aa5839a6922f3eb5f24 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8bf6e26e98b14566899578a990b27d47 in term 1.
I20250109 08:57:35.109992 21781 raft_consensus.cc:2798] T 2935d69446cd458e92046d2b2f1b74ab P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:57:35.110430 21781 raft_consensus.cc:695] T 2935d69446cd458e92046d2b2f1b74ab P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Becoming Leader. State: Replica: 8bf6e26e98b14566899578a990b27d47, State: Running, Role: LEADER
I20250109 08:57:35.107947 21658 raft_consensus.cc:2463] T 2935d69446cd458e92046d2b2f1b74ab P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8bf6e26e98b14566899578a990b27d47 in term 1.
I20250109 08:57:35.111168 21781 consensus_queue.cc:237] T 2935d69446cd458e92046d2b2f1b74ab P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:35.112114 22032 leader_election.cc:290] T 2935d69446cd458e92046d2b2f1b74ab P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 election: Requested vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:35.115063 21478 leader_election.cc:304] T edc9db96e0ff4aa5839a6922f3eb5f24 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3; no voters:
I20250109 08:57:35.115805 21781 raft_consensus.cc:2798] T edc9db96e0ff4aa5839a6922f3eb5f24 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:57:35.116432 21781 raft_consensus.cc:695] T edc9db96e0ff4aa5839a6922f3eb5f24 P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Becoming Leader. State: Replica: 8bf6e26e98b14566899578a990b27d47, State: Running, Role: LEADER
I20250109 08:57:35.117174 21781 consensus_queue.cc:237] T edc9db96e0ff4aa5839a6922f3eb5f24 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:35.118623 21657 raft_consensus.cc:2463] T edc9db96e0ff4aa5839a6922f3eb5f24 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8bf6e26e98b14566899578a990b27d47 in term 1.
I20250109 08:57:35.119926 22092 leader_election.cc:290] T edc9db96e0ff4aa5839a6922f3eb5f24 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 1 election: Requested vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:35.122309 21710 raft_consensus.cc:491] T 7c19ed9fdde5463f84e43bea7b860d03 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:35.122732 21710 raft_consensus.cc:513] T 7c19ed9fdde5463f84e43bea7b860d03 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:35.124397 21710 leader_election.cc:290] T 7c19ed9fdde5463f84e43bea7b860d03 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
I20250109 08:57:35.126030 21585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c19ed9fdde5463f84e43bea7b860d03" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3" is_pre_election: true
I20250109 08:57:35.126754 21585 raft_consensus.cc:2463] T 7c19ed9fdde5463f84e43bea7b860d03 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 0.
I20250109 08:57:35.127895 21512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c19ed9fdde5463f84e43bea7b860d03" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47" is_pre_election: true
I20250109 08:57:35.128674 21512 raft_consensus.cc:2463] T 7c19ed9fdde5463f84e43bea7b860d03 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 0.
I20250109 08:57:35.130616 21627 leader_election.cc:304] T 7c19ed9fdde5463f84e43bea7b860d03 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, a2b1a692435241ee988e0bd1661cd4c3; no voters:
I20250109 08:57:35.131314 21710 raft_consensus.cc:2798] T 7c19ed9fdde5463f84e43bea7b860d03 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:57:35.131639 21710 raft_consensus.cc:491] T 7c19ed9fdde5463f84e43bea7b860d03 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:57:35.131922 21710 raft_consensus.cc:3054] T 7c19ed9fdde5463f84e43bea7b860d03 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:35.132184 22092 consensus_queue.cc:1035] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:57:35.137499 21412 catalog_manager.cc:5526] T 2935d69446cd458e92046d2b2f1b74ab P 8bf6e26e98b14566899578a990b27d47 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8bf6e26e98b14566899578a990b27d47 (127.19.226.193). New cstate: current_term: 1 leader_uuid: "8bf6e26e98b14566899578a990b27d47" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: UNKNOWN } } }
I20250109 08:57:35.150741 21710 raft_consensus.cc:513] T 7c19ed9fdde5463f84e43bea7b860d03 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:35.152400 21710 leader_election.cc:290] T 7c19ed9fdde5463f84e43bea7b860d03 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 election: Requested vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
I20250109 08:57:35.153041 21512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c19ed9fdde5463f84e43bea7b860d03" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47"
I20250109 08:57:35.154574 21512 raft_consensus.cc:3054] T 7c19ed9fdde5463f84e43bea7b860d03 P 8bf6e26e98b14566899578a990b27d47 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:35.157004 21588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c19ed9fdde5463f84e43bea7b860d03" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3"
I20250109 08:57:35.157577 21588 raft_consensus.cc:3054] T 7c19ed9fdde5463f84e43bea7b860d03 P a2b1a692435241ee988e0bd1661cd4c3 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:35.158361 22092 consensus_queue.cc:1035] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:57:35.139091 21412 catalog_manager.cc:5526] T edc9db96e0ff4aa5839a6922f3eb5f24 P 8bf6e26e98b14566899578a990b27d47 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8bf6e26e98b14566899578a990b27d47 (127.19.226.193). New cstate: current_term: 1 leader_uuid: "8bf6e26e98b14566899578a990b27d47" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: UNKNOWN } } }
I20250109 08:57:35.163480 21588 raft_consensus.cc:2463] T 7c19ed9fdde5463f84e43bea7b860d03 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 1.
I20250109 08:57:35.167766 21627 leader_election.cc:304] T 7c19ed9fdde5463f84e43bea7b860d03 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, a2b1a692435241ee988e0bd1661cd4c3; no voters:
I20250109 08:57:35.168969 21907 raft_consensus.cc:2798] T 7c19ed9fdde5463f84e43bea7b860d03 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:57:35.169413 21907 raft_consensus.cc:695] T 7c19ed9fdde5463f84e43bea7b860d03 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Becoming Leader. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Running, Role: LEADER
I20250109 08:57:35.170122 21907 consensus_queue.cc:237] T 7c19ed9fdde5463f84e43bea7b860d03 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:35.170894 21512 raft_consensus.cc:2463] T 7c19ed9fdde5463f84e43bea7b860d03 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 1.
I20250109 08:57:35.204236 21412 catalog_manager.cc:5526] T 7c19ed9fdde5463f84e43bea7b860d03 P 787ddcc824c2490a808fc5a53452dc5e reported cstate change: term changed from 0 to 1, leader changed from <none> to 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195). New cstate: current_term: 1 leader_uuid: "787ddcc824c2490a808fc5a53452dc5e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: HEALTHY } } }
I20250109 08:57:35.244333 21907 consensus_queue.cc:1035] T 5717a02ef53a4b22a96f431ca6c20d93 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:57:35.246997 21657 raft_consensus.cc:3054] T f5f85ad7c7c94acfaf9d550d55fce48c P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:35.260354 22044 consensus_queue.cc:1035] T 5717a02ef53a4b22a96f431ca6c20d93 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:57:35.290289 21657 raft_consensus.cc:3054] T 56bbc306275e4a409e028a7ca1dd1602 P 787ddcc824c2490a808fc5a53452dc5e [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:57:35.296101 22052 consensus_queue.cc:1035] T 56bbc306275e4a409e028a7ca1dd1602 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:57:35.304550 21781 consensus_queue.cc:1035] T 56bbc306275e4a409e028a7ca1dd1602 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:57:35.541808 22092 consensus_queue.cc:1035] T 2935d69446cd458e92046d2b2f1b74ab P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:57:35.554913 21781 consensus_queue.cc:1035] T 2935d69446cd458e92046d2b2f1b74ab P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:57:35.568439 21781 consensus_queue.cc:1035] T edc9db96e0ff4aa5839a6922f3eb5f24 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:57:35.587841 22092 consensus_queue.cc:1035] T edc9db96e0ff4aa5839a6922f3eb5f24 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:57:35.635723 21907 consensus_queue.cc:1035] T 7c19ed9fdde5463f84e43bea7b860d03 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:57:35.648697 21907 consensus_queue.cc:1035] T 7c19ed9fdde5463f84e43bea7b860d03 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20250109 08:57:35.682235 21460 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20250109 08:57:36.029911 22118 raft_consensus.cc:491] T 1d3d5d2a2c88417a97c9d57262a356dd P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:57:36.030377 22118 raft_consensus.cc:513] T 1d3d5d2a2c88417a97c9d57262a356dd P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:36.032231 22118 leader_election.cc:290] T 1d3d5d2a2c88417a97c9d57262a356dd P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:36.033339 21658 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d3d5d2a2c88417a97c9d57262a356dd" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e" is_pre_election: true
I20250109 08:57:36.034085 21658 raft_consensus.cc:2463] T 1d3d5d2a2c88417a97c9d57262a356dd P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2b1a692435241ee988e0bd1661cd4c3 in term 1.
I20250109 08:57:36.034905 21509 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d3d5d2a2c88417a97c9d57262a356dd" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47" is_pre_election: true
I20250109 08:57:36.035485 21509 raft_consensus.cc:2463] T 1d3d5d2a2c88417a97c9d57262a356dd P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2b1a692435241ee988e0bd1661cd4c3 in term 1.
I20250109 08:57:36.035427 21552 leader_election.cc:304] T 1d3d5d2a2c88417a97c9d57262a356dd P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, a2b1a692435241ee988e0bd1661cd4c3; no voters:
I20250109 08:57:36.036230 22118 raft_consensus.cc:2798] T 1d3d5d2a2c88417a97c9d57262a356dd P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20250109 08:57:36.036562 22118 raft_consensus.cc:491] T 1d3d5d2a2c88417a97c9d57262a356dd P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:57:36.036844 22118 raft_consensus.cc:3054] T 1d3d5d2a2c88417a97c9d57262a356dd P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:57:36.042275 22118 raft_consensus.cc:513] T 1d3d5d2a2c88417a97c9d57262a356dd P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:36.044085 22118 leader_election.cc:290] T 1d3d5d2a2c88417a97c9d57262a356dd P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 2 election: Requested vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:36.044848 21658 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d3d5d2a2c88417a97c9d57262a356dd" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e"
I20250109 08:57:36.045194 21509 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d3d5d2a2c88417a97c9d57262a356dd" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf6e26e98b14566899578a990b27d47"
I20250109 08:57:36.045382 21658 raft_consensus.cc:3054] T 1d3d5d2a2c88417a97c9d57262a356dd P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:57:36.045703 21509 raft_consensus.cc:3054] T 1d3d5d2a2c88417a97c9d57262a356dd P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:57:36.050690 21658 raft_consensus.cc:2463] T 1d3d5d2a2c88417a97c9d57262a356dd P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2b1a692435241ee988e0bd1661cd4c3 in term 2.
I20250109 08:57:36.051503 21552 leader_election.cc:304] T 1d3d5d2a2c88417a97c9d57262a356dd P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, a2b1a692435241ee988e0bd1661cd4c3; no voters:
I20250109 08:57:36.051906 21509 raft_consensus.cc:2463] T 1d3d5d2a2c88417a97c9d57262a356dd P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2b1a692435241ee988e0bd1661cd4c3 in term 2.
I20250109 08:57:36.052066 22118 raft_consensus.cc:2798] T 1d3d5d2a2c88417a97c9d57262a356dd P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:57:36.052565 22118 raft_consensus.cc:695] T 1d3d5d2a2c88417a97c9d57262a356dd P a2b1a692435241ee988e0bd1661cd4c3 [term 2 LEADER]: Becoming Leader. State: Replica: a2b1a692435241ee988e0bd1661cd4c3, State: Running, Role: LEADER
I20250109 08:57:36.053308 22118 consensus_queue.cc:237] T 1d3d5d2a2c88417a97c9d57262a356dd P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:36.065765 21410 catalog_manager.cc:5526] T 1d3d5d2a2c88417a97c9d57262a356dd P a2b1a692435241ee988e0bd1661cd4c3 reported cstate change: term changed from 0 to 2, leader changed from <none> to a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194). New cstate: current_term: 2 leader_uuid: "a2b1a692435241ee988e0bd1661cd4c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: UNKNOWN } } }
I20250109 08:57:36.484953 21658 raft_consensus.cc:1270] T 1d3d5d2a2c88417a97c9d57262a356dd P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Refusing update from remote peer a2b1a692435241ee988e0bd1661cd4c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20250109 08:57:36.486277 22036 consensus_queue.cc:1035] T 1d3d5d2a2c88417a97c9d57262a356dd P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:57:36.503136 21509 raft_consensus.cc:1270] T 1d3d5d2a2c88417a97c9d57262a356dd P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Refusing update from remote peer a2b1a692435241ee988e0bd1661cd4c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20250109 08:57:36.504792 22036 consensus_queue.cc:1035] T 1d3d5d2a2c88417a97c9d57262a356dd P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:57:37.746960 20363 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250109 08:57:37.774660 22132 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250109 08:57:37.776053 22133 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250109 08:57:37.781167 22135 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250109 08:57:37.787423 20363 server_base.cc:1034] running on GCE node
I20250109 08:57:37.788594 20363 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250109 08:57:37.788856 20363 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250109 08:57:37.789077 20363 hybrid_clock.cc:648] HybridClock initialized: now 1736413057789056 us; error 0 us; skew 500 ppm
I20250109 08:57:37.789785 20363 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250109 08:57:37.794485 20363 webserver.cc:458] Webserver started at http://127.19.226.196:33041/ using document root <none> and password file <none>
I20250109 08:57:37.795010 20363 fs_manager.cc:362] Metadata directory not provided
I20250109 08:57:37.795229 20363 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250109 08:57:37.795522 20363 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250109 08:57:37.796651 20363 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736412987538469-20363-0/minicluster-data/ts-3-root/instance:
uuid: "88935ab835544e9889d05a6587d8cf21"
format_stamp: "Formatted at 2025-01-09 08:57:37 on dist-test-slave-g8ds"
I20250109 08:57:37.801774 20363 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20250109 08:57:37.804857 22140 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250109 08:57:37.805562 20363 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20250109 08:57:37.805843 20363 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736412987538469-20363-0/minicluster-data/ts-3-root
uuid: "88935ab835544e9889d05a6587d8cf21"
format_stamp: "Formatted at 2025-01-09 08:57:37 on dist-test-slave-g8ds"
I20250109 08:57:37.806097 20363 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736412987538469-20363-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736412987538469-20363-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736412987538469-20363-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250109 08:57:37.824537 20363 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250109 08:57:37.825816 20363 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250109 08:57:37.827281 20363 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250109 08:57:37.829595 20363 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250109 08:57:37.829793 20363 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250109 08:57:37.830022 20363 ts_tablet_manager.cc:610] Registered 0 tablets
I20250109 08:57:37.830178 20363 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250109 08:57:37.877311 20363 rpc_server.cc:307] RPC server started. Bound to: 127.19.226.196:44363
I20250109 08:57:37.877436 22202 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.226.196:44363 every 8 connection(s)
I20250109 08:57:37.881278 20363 auto_leader_rebalancer-test.cc:413] add a tserver: 88935ab835544e9889d05a6587d8cf21
I20250109 08:57:37.895395 22203 heartbeater.cc:346] Connected to a master server at 127.19.226.254:38499
I20250109 08:57:37.895790 22203 heartbeater.cc:463] Registering TS with master...
I20250109 08:57:37.896472 22203 heartbeater.cc:510] Master 127.19.226.254:38499 requested a full tablet report, sending...
I20250109 08:57:37.898183 21411 ts_manager.cc:194] Registered new tserver with Master: 88935ab835544e9889d05a6587d8cf21 (127.19.226.196:44363)
I20250109 08:57:37.899540 21411 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:42722
I20250109 08:57:38.875149 21510 consensus_queue.cc:237] T edc9db96e0ff4aa5839a6922f3eb5f24 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:57:38.893505 21584 raft_consensus.cc:1270] T edc9db96e0ff4aa5839a6922f3eb5f24 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Refusing update from remote peer 8bf6e26e98b14566899578a990b27d47: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250109 08:57:38.893981 21658 raft_consensus.cc:1270] T edc9db96e0ff4aa5839a6922f3eb5f24 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Refusing update from remote peer 8bf6e26e98b14566899578a990b27d47: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250109 08:57:38.897274 22050 consensus_queue.cc:1035] T edc9db96e0ff4aa5839a6922f3eb5f24 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20250109 08:57:38.898543 22207 consensus_queue.cc:1035] T edc9db96e0ff4aa5839a6922f3eb5f24 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:57:38.902557 22203 heartbeater.cc:502] Master 127.19.226.254:38499 was elected leader, sending a full tablet report...
I20250109 08:57:38.911780 21781 raft_consensus.cc:2949] T edc9db96e0ff4aa5839a6922f3eb5f24 P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
W20250109 08:57:38.914286 21478 consensus_peers.cc:487] T edc9db96e0ff4aa5839a6922f3eb5f24 P 8bf6e26e98b14566899578a990b27d47 -> Peer 88935ab835544e9889d05a6587d8cf21 (127.19.226.196:44363): Couldn't send request to peer 88935ab835544e9889d05a6587d8cf21. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: edc9db96e0ff4aa5839a6922f3eb5f24. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:57:38.918220 21584 raft_consensus.cc:2949] T edc9db96e0ff4aa5839a6922f3eb5f24 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:57:38.921242 21660 raft_consensus.cc:2949] T edc9db96e0ff4aa5839a6922f3eb5f24 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:57:38.943169 21411 catalog_manager.cc:5526] T edc9db96e0ff4aa5839a6922f3eb5f24 P 8bf6e26e98b14566899578a990b27d47 reported cstate change: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New cstate: current_term: 1 leader_uuid: "8bf6e26e98b14566899578a990b27d47" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20250109 08:57:38.973138 21660 consensus_queue.cc:237] T 90a4a2381cbe40f0b204be325ee9fea1 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:57:38.981689 21585 raft_consensus.cc:1270] T 90a4a2381cbe40f0b204be325ee9fea1 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250109 08:57:38.982867 21510 raft_consensus.cc:1270] T 90a4a2381cbe40f0b204be325ee9fea1 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250109 08:57:38.984951 22208 consensus_queue.cc:1035] T 90a4a2381cbe40f0b204be325ee9fea1 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20250109 08:57:38.990168 22208 consensus_queue.cc:1035] T 90a4a2381cbe40f0b204be325ee9fea1 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20250109 08:57:39.001475 21627 consensus_peers.cc:487] T 90a4a2381cbe40f0b204be325ee9fea1 P 787ddcc824c2490a808fc5a53452dc5e -> Peer 88935ab835544e9889d05a6587d8cf21 (127.19.226.196:44363): Couldn't send request to peer 88935ab835544e9889d05a6587d8cf21. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 90a4a2381cbe40f0b204be325ee9fea1. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:57:39.002599 21907 raft_consensus.cc:2949] T 90a4a2381cbe40f0b204be325ee9fea1 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:57:39.004271 21585 raft_consensus.cc:2949] T 90a4a2381cbe40f0b204be325ee9fea1 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:57:39.004033 21510 raft_consensus.cc:2949] T 90a4a2381cbe40f0b204be325ee9fea1 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:57:39.022272 21510 consensus_queue.cc:237] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:57:39.023126 21412 catalog_manager.cc:5526] T 90a4a2381cbe40f0b204be325ee9fea1 P 8bf6e26e98b14566899578a990b27d47 reported cstate change: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New cstate: current_term: 1 leader_uuid: "787ddcc824c2490a808fc5a53452dc5e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
W20250109 08:57:39.032352 21478 consensus_peers.cc:487] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 8bf6e26e98b14566899578a990b27d47 -> Peer 88935ab835544e9889d05a6587d8cf21 (127.19.226.196:44363): Couldn't send request to peer 88935ab835544e9889d05a6587d8cf21. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5d31dbfb2b4a4a6ca4500f92df9dbb98. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:57:39.033823 21660 raft_consensus.cc:1270] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Refusing update from remote peer 8bf6e26e98b14566899578a990b27d47: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250109 08:57:39.036824 21585 raft_consensus.cc:1270] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Refusing update from remote peer 8bf6e26e98b14566899578a990b27d47: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250109 08:57:39.036605 22221 consensus_queue.cc:1035] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20250109 08:57:39.041033 22221 consensus_queue.cc:1035] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:57:39.051466 21781 raft_consensus.cc:2949] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:57:39.053439 21658 raft_consensus.cc:2949] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:57:39.057783 21585 raft_consensus.cc:2949] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:57:39.069584 21411 catalog_manager.cc:5526] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 787ddcc824c2490a808fc5a53452dc5e reported cstate change: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New cstate: current_term: 1 leader_uuid: "8bf6e26e98b14566899578a990b27d47" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:57:39.070761 21658 consensus_queue.cc:237] T 20ea30dba4314290ae45d9b4c13bebdd P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:57:39.084769 21585 raft_consensus.cc:1270] T 20ea30dba4314290ae45d9b4c13bebdd P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20250109 08:57:39.085649 21627 consensus_peers.cc:487] T 20ea30dba4314290ae45d9b4c13bebdd P 787ddcc824c2490a808fc5a53452dc5e -> Peer 88935ab835544e9889d05a6587d8cf21 (127.19.226.196:44363): Couldn't send request to peer 88935ab835544e9889d05a6587d8cf21. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 20ea30dba4314290ae45d9b4c13bebdd. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:57:39.086468 21907 consensus_queue.cc:1035] T 20ea30dba4314290ae45d9b4c13bebdd P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:57:39.088548 21510 raft_consensus.cc:1270] T 20ea30dba4314290ae45d9b4c13bebdd P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250109 08:57:39.089756 21907 consensus_queue.cc:1035] T 20ea30dba4314290ae45d9b4c13bebdd P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:57:39.095340 21758 raft_consensus.cc:2949] T 20ea30dba4314290ae45d9b4c13bebdd P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:57:39.099668 21585 raft_consensus.cc:2949] T 20ea30dba4314290ae45d9b4c13bebdd P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:57:39.104547 21510 raft_consensus.cc:2949] T 20ea30dba4314290ae45d9b4c13bebdd P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:57:39.119886 21411 catalog_manager.cc:5526] T 20ea30dba4314290ae45d9b4c13bebdd P a2b1a692435241ee988e0bd1661cd4c3 reported cstate change: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New cstate: current_term: 1 leader_uuid: "787ddcc824c2490a808fc5a53452dc5e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:57:39.361999 22237 ts_tablet_manager.cc:927] T 90a4a2381cbe40f0b204be325ee9fea1 P 88935ab835544e9889d05a6587d8cf21: Initiating tablet copy from peer 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:39.364882 22237 tablet_copy_client.cc:323] T 90a4a2381cbe40f0b204be325ee9fea1 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Beginning tablet copy session from remote peer at address 127.19.226.195:35731
I20250109 08:57:39.376766 21673 tablet_copy_service.cc:140] P 787ddcc824c2490a808fc5a53452dc5e: Received BeginTabletCopySession request for tablet 90a4a2381cbe40f0b204be325ee9fea1 from peer 88935ab835544e9889d05a6587d8cf21 ({username='slave'} at 127.0.0.1:52376)
I20250109 08:57:39.377471 21673 tablet_copy_service.cc:161] P 787ddcc824c2490a808fc5a53452dc5e: Beginning new tablet copy session on tablet 90a4a2381cbe40f0b204be325ee9fea1 from peer 88935ab835544e9889d05a6587d8cf21 at {username='slave'} at 127.0.0.1:52376: session id = 88935ab835544e9889d05a6587d8cf21-90a4a2381cbe40f0b204be325ee9fea1
I20250109 08:57:39.384418 21673 tablet_copy_source_session.cc:215] T 90a4a2381cbe40f0b204be325ee9fea1 P 787ddcc824c2490a808fc5a53452dc5e: Tablet Copy: opened 0 blocks and 1 log segments
I20250109 08:57:39.388221 22237 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90a4a2381cbe40f0b204be325ee9fea1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:39.402513 22237 tablet_copy_client.cc:806] T 90a4a2381cbe40f0b204be325ee9fea1 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Starting download of 0 data blocks...
I20250109 08:57:39.403283 22237 tablet_copy_client.cc:670] T 90a4a2381cbe40f0b204be325ee9fea1 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Starting download of 1 WAL segments...
I20250109 08:57:39.412461 22237 tablet_copy_client.cc:538] T 90a4a2381cbe40f0b204be325ee9fea1 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250109 08:57:39.419359 22237 tablet_bootstrap.cc:492] T 90a4a2381cbe40f0b204be325ee9fea1 P 88935ab835544e9889d05a6587d8cf21: Bootstrap starting.
I20250109 08:57:39.438441 22237 tablet_bootstrap.cc:492] T 90a4a2381cbe40f0b204be325ee9fea1 P 88935ab835544e9889d05a6587d8cf21: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250109 08:57:39.439517 22237 tablet_bootstrap.cc:492] T 90a4a2381cbe40f0b204be325ee9fea1 P 88935ab835544e9889d05a6587d8cf21: Bootstrap complete.
I20250109 08:57:39.440248 22237 ts_tablet_manager.cc:1397] T 90a4a2381cbe40f0b204be325ee9fea1 P 88935ab835544e9889d05a6587d8cf21: Time spent bootstrapping tablet: real 0.021s user 0.021s sys 0.001s
I20250109 08:57:39.443156 22237 raft_consensus.cc:357] T 90a4a2381cbe40f0b204be325ee9fea1 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:57:39.444099 22237 raft_consensus.cc:738] T 90a4a2381cbe40f0b204be325ee9fea1 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 88935ab835544e9889d05a6587d8cf21, State: Initialized, Role: LEARNER
I20250109 08:57:39.444799 22237 consensus_queue.cc:260] T 90a4a2381cbe40f0b204be325ee9fea1 P 88935ab835544e9889d05a6587d8cf21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:57:39.449162 22237 ts_tablet_manager.cc:1428] T 90a4a2381cbe40f0b204be325ee9fea1 P 88935ab835544e9889d05a6587d8cf21: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20250109 08:57:39.450836 21673 tablet_copy_service.cc:342] P 787ddcc824c2490a808fc5a53452dc5e: Request end of tablet copy session 88935ab835544e9889d05a6587d8cf21-90a4a2381cbe40f0b204be325ee9fea1 received from {username='slave'} at 127.0.0.1:52376
I20250109 08:57:39.451344 21673 tablet_copy_service.cc:434] P 787ddcc824c2490a808fc5a53452dc5e: ending tablet copy session 88935ab835544e9889d05a6587d8cf21-90a4a2381cbe40f0b204be325ee9fea1 on tablet 90a4a2381cbe40f0b204be325ee9fea1 with peer 88935ab835544e9889d05a6587d8cf21
I20250109 08:57:39.451501 22241 ts_tablet_manager.cc:927] T edc9db96e0ff4aa5839a6922f3eb5f24 P 88935ab835544e9889d05a6587d8cf21: Initiating tablet copy from peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725)
I20250109 08:57:39.454156 22241 tablet_copy_client.cc:323] T edc9db96e0ff4aa5839a6922f3eb5f24 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Beginning tablet copy session from remote peer at address 127.19.226.193:42725
I20250109 08:57:39.466651 21524 tablet_copy_service.cc:140] P 8bf6e26e98b14566899578a990b27d47: Received BeginTabletCopySession request for tablet edc9db96e0ff4aa5839a6922f3eb5f24 from peer 88935ab835544e9889d05a6587d8cf21 ({username='slave'} at 127.0.0.1:41812)
I20250109 08:57:39.467214 21524 tablet_copy_service.cc:161] P 8bf6e26e98b14566899578a990b27d47: Beginning new tablet copy session on tablet edc9db96e0ff4aa5839a6922f3eb5f24 from peer 88935ab835544e9889d05a6587d8cf21 at {username='slave'} at 127.0.0.1:41812: session id = 88935ab835544e9889d05a6587d8cf21-edc9db96e0ff4aa5839a6922f3eb5f24
I20250109 08:57:39.474802 21524 tablet_copy_source_session.cc:215] T edc9db96e0ff4aa5839a6922f3eb5f24 P 8bf6e26e98b14566899578a990b27d47: Tablet Copy: opened 0 blocks and 1 log segments
I20250109 08:57:39.479076 22241 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet edc9db96e0ff4aa5839a6922f3eb5f24. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:39.491717 22241 tablet_copy_client.cc:806] T edc9db96e0ff4aa5839a6922f3eb5f24 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Starting download of 0 data blocks...
I20250109 08:57:39.492359 22241 tablet_copy_client.cc:670] T edc9db96e0ff4aa5839a6922f3eb5f24 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Starting download of 1 WAL segments...
I20250109 08:57:39.495869 22241 tablet_copy_client.cc:538] T edc9db96e0ff4aa5839a6922f3eb5f24 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250109 08:57:39.502509 22241 tablet_bootstrap.cc:492] T edc9db96e0ff4aa5839a6922f3eb5f24 P 88935ab835544e9889d05a6587d8cf21: Bootstrap starting.
I20250109 08:57:39.520507 22241 tablet_bootstrap.cc:492] T edc9db96e0ff4aa5839a6922f3eb5f24 P 88935ab835544e9889d05a6587d8cf21: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250109 08:57:39.521235 22241 tablet_bootstrap.cc:492] T edc9db96e0ff4aa5839a6922f3eb5f24 P 88935ab835544e9889d05a6587d8cf21: Bootstrap complete.
I20250109 08:57:39.521733 22241 ts_tablet_manager.cc:1397] T edc9db96e0ff4aa5839a6922f3eb5f24 P 88935ab835544e9889d05a6587d8cf21: Time spent bootstrapping tablet: real 0.019s user 0.016s sys 0.004s
I20250109 08:57:39.524080 22241 raft_consensus.cc:357] T edc9db96e0ff4aa5839a6922f3eb5f24 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:57:39.524986 22241 raft_consensus.cc:738] T edc9db96e0ff4aa5839a6922f3eb5f24 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 88935ab835544e9889d05a6587d8cf21, State: Initialized, Role: LEARNER
I20250109 08:57:39.525588 22241 consensus_queue.cc:260] T edc9db96e0ff4aa5839a6922f3eb5f24 P 88935ab835544e9889d05a6587d8cf21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:57:39.529371 22241 ts_tablet_manager.cc:1428] T edc9db96e0ff4aa5839a6922f3eb5f24 P 88935ab835544e9889d05a6587d8cf21: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20250109 08:57:39.531718 21524 tablet_copy_service.cc:342] P 8bf6e26e98b14566899578a990b27d47: Request end of tablet copy session 88935ab835544e9889d05a6587d8cf21-edc9db96e0ff4aa5839a6922f3eb5f24 received from {username='slave'} at 127.0.0.1:41812
I20250109 08:57:39.532159 21524 tablet_copy_service.cc:434] P 8bf6e26e98b14566899578a990b27d47: ending tablet copy session 88935ab835544e9889d05a6587d8cf21-edc9db96e0ff4aa5839a6922f3eb5f24 on tablet edc9db96e0ff4aa5839a6922f3eb5f24 with peer 88935ab835544e9889d05a6587d8cf21
I20250109 08:57:39.535636 22237 ts_tablet_manager.cc:927] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 88935ab835544e9889d05a6587d8cf21: Initiating tablet copy from peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725)
I20250109 08:57:39.542693 22237 tablet_copy_client.cc:323] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Beginning tablet copy session from remote peer at address 127.19.226.193:42725
I20250109 08:57:39.544539 21524 tablet_copy_service.cc:140] P 8bf6e26e98b14566899578a990b27d47: Received BeginTabletCopySession request for tablet 5d31dbfb2b4a4a6ca4500f92df9dbb98 from peer 88935ab835544e9889d05a6587d8cf21 ({username='slave'} at 127.0.0.1:41812)
I20250109 08:57:39.544967 21524 tablet_copy_service.cc:161] P 8bf6e26e98b14566899578a990b27d47: Beginning new tablet copy session on tablet 5d31dbfb2b4a4a6ca4500f92df9dbb98 from peer 88935ab835544e9889d05a6587d8cf21 at {username='slave'} at 127.0.0.1:41812: session id = 88935ab835544e9889d05a6587d8cf21-5d31dbfb2b4a4a6ca4500f92df9dbb98
I20250109 08:57:39.550061 21524 tablet_copy_source_session.cc:215] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 8bf6e26e98b14566899578a990b27d47: Tablet Copy: opened 0 blocks and 1 log segments
I20250109 08:57:39.553041 22237 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d31dbfb2b4a4a6ca4500f92df9dbb98. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:39.565891 22237 tablet_copy_client.cc:806] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Starting download of 0 data blocks...
I20250109 08:57:39.566336 22237 tablet_copy_client.cc:670] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Starting download of 1 WAL segments...
I20250109 08:57:39.569001 22237 tablet_copy_client.cc:538] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250109 08:57:39.575100 22237 tablet_bootstrap.cc:492] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 88935ab835544e9889d05a6587d8cf21: Bootstrap starting.
I20250109 08:57:39.595572 22237 tablet_bootstrap.cc:492] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 88935ab835544e9889d05a6587d8cf21: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250109 08:57:39.596426 22237 tablet_bootstrap.cc:492] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 88935ab835544e9889d05a6587d8cf21: Bootstrap complete.
I20250109 08:57:39.597020 22237 ts_tablet_manager.cc:1397] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 88935ab835544e9889d05a6587d8cf21: Time spent bootstrapping tablet: real 0.022s user 0.011s sys 0.011s
I20250109 08:57:39.599411 22237 raft_consensus.cc:357] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:57:39.600201 22237 raft_consensus.cc:738] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 88935ab835544e9889d05a6587d8cf21, State: Initialized, Role: LEARNER
I20250109 08:57:39.600754 22237 consensus_queue.cc:260] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 88935ab835544e9889d05a6587d8cf21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:57:39.602345 22237 ts_tablet_manager.cc:1428] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 88935ab835544e9889d05a6587d8cf21: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20250109 08:57:39.603976 21524 tablet_copy_service.cc:342] P 8bf6e26e98b14566899578a990b27d47: Request end of tablet copy session 88935ab835544e9889d05a6587d8cf21-5d31dbfb2b4a4a6ca4500f92df9dbb98 received from {username='slave'} at 127.0.0.1:41812
I20250109 08:57:39.604403 21524 tablet_copy_service.cc:434] P 8bf6e26e98b14566899578a990b27d47: ending tablet copy session 88935ab835544e9889d05a6587d8cf21-5d31dbfb2b4a4a6ca4500f92df9dbb98 on tablet 5d31dbfb2b4a4a6ca4500f92df9dbb98 with peer 88935ab835544e9889d05a6587d8cf21
I20250109 08:57:39.651198 22237 ts_tablet_manager.cc:927] T 20ea30dba4314290ae45d9b4c13bebdd P 88935ab835544e9889d05a6587d8cf21: Initiating tablet copy from peer 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:39.653018 22237 tablet_copy_client.cc:323] T 20ea30dba4314290ae45d9b4c13bebdd P 88935ab835544e9889d05a6587d8cf21: tablet copy: Beginning tablet copy session from remote peer at address 127.19.226.195:35731
I20250109 08:57:39.654384 21673 tablet_copy_service.cc:140] P 787ddcc824c2490a808fc5a53452dc5e: Received BeginTabletCopySession request for tablet 20ea30dba4314290ae45d9b4c13bebdd from peer 88935ab835544e9889d05a6587d8cf21 ({username='slave'} at 127.0.0.1:52376)
I20250109 08:57:39.654881 21673 tablet_copy_service.cc:161] P 787ddcc824c2490a808fc5a53452dc5e: Beginning new tablet copy session on tablet 20ea30dba4314290ae45d9b4c13bebdd from peer 88935ab835544e9889d05a6587d8cf21 at {username='slave'} at 127.0.0.1:52376: session id = 88935ab835544e9889d05a6587d8cf21-20ea30dba4314290ae45d9b4c13bebdd
I20250109 08:57:39.664696 21673 tablet_copy_source_session.cc:215] T 20ea30dba4314290ae45d9b4c13bebdd P 787ddcc824c2490a808fc5a53452dc5e: Tablet Copy: opened 0 blocks and 1 log segments
I20250109 08:57:39.667593 22237 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20ea30dba4314290ae45d9b4c13bebdd. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:39.685000 22237 tablet_copy_client.cc:806] T 20ea30dba4314290ae45d9b4c13bebdd P 88935ab835544e9889d05a6587d8cf21: tablet copy: Starting download of 0 data blocks...
I20250109 08:57:39.685516 22237 tablet_copy_client.cc:670] T 20ea30dba4314290ae45d9b4c13bebdd P 88935ab835544e9889d05a6587d8cf21: tablet copy: Starting download of 1 WAL segments...
I20250109 08:57:39.688652 22237 tablet_copy_client.cc:538] T 20ea30dba4314290ae45d9b4c13bebdd P 88935ab835544e9889d05a6587d8cf21: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250109 08:57:39.695501 22237 tablet_bootstrap.cc:492] T 20ea30dba4314290ae45d9b4c13bebdd P 88935ab835544e9889d05a6587d8cf21: Bootstrap starting.
I20250109 08:57:39.720005 22237 tablet_bootstrap.cc:492] T 20ea30dba4314290ae45d9b4c13bebdd P 88935ab835544e9889d05a6587d8cf21: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250109 08:57:39.720703 22237 tablet_bootstrap.cc:492] T 20ea30dba4314290ae45d9b4c13bebdd P 88935ab835544e9889d05a6587d8cf21: Bootstrap complete.
I20250109 08:57:39.721215 22237 ts_tablet_manager.cc:1397] T 20ea30dba4314290ae45d9b4c13bebdd P 88935ab835544e9889d05a6587d8cf21: Time spent bootstrapping tablet: real 0.026s user 0.024s sys 0.004s
I20250109 08:57:39.723747 22237 raft_consensus.cc:357] T 20ea30dba4314290ae45d9b4c13bebdd P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:57:39.724526 22237 raft_consensus.cc:738] T 20ea30dba4314290ae45d9b4c13bebdd P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 88935ab835544e9889d05a6587d8cf21, State: Initialized, Role: LEARNER
I20250109 08:57:39.724988 22237 consensus_queue.cc:260] T 20ea30dba4314290ae45d9b4c13bebdd P 88935ab835544e9889d05a6587d8cf21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:57:39.728353 22237 ts_tablet_manager.cc:1428] T 20ea30dba4314290ae45d9b4c13bebdd P 88935ab835544e9889d05a6587d8cf21: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20250109 08:57:39.729635 21673 tablet_copy_service.cc:342] P 787ddcc824c2490a808fc5a53452dc5e: Request end of tablet copy session 88935ab835544e9889d05a6587d8cf21-20ea30dba4314290ae45d9b4c13bebdd received from {username='slave'} at 127.0.0.1:52376
I20250109 08:57:39.729962 21673 tablet_copy_service.cc:434] P 787ddcc824c2490a808fc5a53452dc5e: ending tablet copy session 88935ab835544e9889d05a6587d8cf21-20ea30dba4314290ae45d9b4c13bebdd on tablet 20ea30dba4314290ae45d9b4c13bebdd with peer 88935ab835544e9889d05a6587d8cf21
I20250109 08:57:39.759593 22178 raft_consensus.cc:1212] T 90a4a2381cbe40f0b204be325ee9fea1 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20250109 08:57:39.935427 22178 raft_consensus.cc:1212] T edc9db96e0ff4aa5839a6922f3eb5f24 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20250109 08:57:40.017416 22178 raft_consensus.cc:1212] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20250109 08:57:40.235203 22178 raft_consensus.cc:1212] T 20ea30dba4314290ae45d9b4c13bebdd P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20250109 08:57:40.316522 22251 raft_consensus.cc:1059] T 90a4a2381cbe40f0b204be325ee9fea1 P 787ddcc824c2490a808fc5a53452dc5e: attempting to promote NON_VOTER 88935ab835544e9889d05a6587d8cf21 to VOTER
I20250109 08:57:40.318970 22251 consensus_queue.cc:237] T 90a4a2381cbe40f0b204be325ee9fea1 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:57:40.324152 21585 raft_consensus.cc:1270] T 90a4a2381cbe40f0b204be325ee9fea1 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:57:40.325045 22178 raft_consensus.cc:1270] T 90a4a2381cbe40f0b204be325ee9fea1 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:57:40.325233 21513 raft_consensus.cc:1270] T 90a4a2381cbe40f0b204be325ee9fea1 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:57:40.325577 22251 consensus_queue.cc:1035] T 90a4a2381cbe40f0b204be325ee9fea1 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20250109 08:57:40.327322 22208 consensus_queue.cc:1035] T 90a4a2381cbe40f0b204be325ee9fea1 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:57:40.328156 21907 consensus_queue.cc:1035] T 90a4a2381cbe40f0b204be325ee9fea1 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:57:40.340274 21907 raft_consensus.cc:2949] T 90a4a2381cbe40f0b204be325ee9fea1 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:40.344394 21585 raft_consensus.cc:2949] T 90a4a2381cbe40f0b204be325ee9fea1 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:40.353220 21513 raft_consensus.cc:2949] T 90a4a2381cbe40f0b204be325ee9fea1 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:40.353761 22178 raft_consensus.cc:2949] T 90a4a2381cbe40f0b204be325ee9fea1 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:40.364921 21412 catalog_manager.cc:5526] T 90a4a2381cbe40f0b204be325ee9fea1 P a2b1a692435241ee988e0bd1661cd4c3 reported cstate change: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "787ddcc824c2490a808fc5a53452dc5e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:40.448519 22221 raft_consensus.cc:1059] T edc9db96e0ff4aa5839a6922f3eb5f24 P 8bf6e26e98b14566899578a990b27d47: attempting to promote NON_VOTER 88935ab835544e9889d05a6587d8cf21 to VOTER
I20250109 08:57:40.450996 22221 consensus_queue.cc:237] T edc9db96e0ff4aa5839a6922f3eb5f24 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:57:40.454185 21658 consensus_queue.cc:237] T 90a4a2381cbe40f0b204be325ee9fea1 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:57:40.464470 22178 raft_consensus.cc:1270] T 90a4a2381cbe40f0b204be325ee9fea1 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250109 08:57:40.465598 21585 raft_consensus.cc:1270] T 90a4a2381cbe40f0b204be325ee9fea1 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250109 08:57:40.469182 22208 consensus_queue.cc:1035] T 90a4a2381cbe40f0b204be325ee9fea1 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250109 08:57:40.471562 22208 consensus_queue.cc:1035] T 90a4a2381cbe40f0b204be325ee9fea1 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250109 08:57:40.480949 21781 raft_consensus.cc:1059] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 8bf6e26e98b14566899578a990b27d47: attempting to promote NON_VOTER 88935ab835544e9889d05a6587d8cf21 to VOTER
I20250109 08:57:40.481184 22260 raft_consensus.cc:2949] T 90a4a2381cbe40f0b204be325ee9fea1 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8bf6e26e98b14566899578a990b27d47 (127.19.226.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:40.483043 21781 consensus_queue.cc:237] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:57:40.490880 21396 catalog_manager.cc:5039] ChangeConfig:REMOVE_PEER RPC for tablet 90a4a2381cbe40f0b204be325ee9fea1 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250109 08:57:40.493736 21585 raft_consensus.cc:2949] T 90a4a2381cbe40f0b204be325ee9fea1 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8bf6e26e98b14566899578a990b27d47 (127.19.226.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
W20250109 08:57:40.500989 21396 catalog_manager.cc:5049] ChangeConfig:REMOVE_PEER RPC for tablet 90a4a2381cbe40f0b204be325ee9fea1 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731) due to CAS failure; no further retry: Illegal state: Request specified cas_config_opid_index of 3 but the committed config has opid_index of 4
I20250109 08:57:40.504093 22177 raft_consensus.cc:1270] T edc9db96e0ff4aa5839a6922f3eb5f24 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Refusing update from remote peer 8bf6e26e98b14566899578a990b27d47: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:57:40.505375 22178 raft_consensus.cc:2949] T 90a4a2381cbe40f0b204be325ee9fea1 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8bf6e26e98b14566899578a990b27d47 (127.19.226.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:40.523650 22250 consensus_queue.cc:1035] T edc9db96e0ff4aa5839a6922f3eb5f24 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20250109 08:57:40.534895 21660 raft_consensus.cc:1270] T edc9db96e0ff4aa5839a6922f3eb5f24 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Refusing update from remote peer 8bf6e26e98b14566899578a990b27d47: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:57:40.549407 22177 raft_consensus.cc:1270] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Refusing update from remote peer 8bf6e26e98b14566899578a990b27d47: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:57:40.549533 21585 raft_consensus.cc:1270] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Refusing update from remote peer 8bf6e26e98b14566899578a990b27d47: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:57:40.563665 21408 catalog_manager.cc:5526] T 90a4a2381cbe40f0b204be325ee9fea1 P a2b1a692435241ee988e0bd1661cd4c3 reported cstate change: config changed from index 3 to 4, VOTER 8bf6e26e98b14566899578a990b27d47 (127.19.226.193) evicted. New cstate: current_term: 1 leader_uuid: "787ddcc824c2490a808fc5a53452dc5e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:40.568020 21585 raft_consensus.cc:1270] T edc9db96e0ff4aa5839a6922f3eb5f24 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Refusing update from remote peer 8bf6e26e98b14566899578a990b27d47: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:57:40.575853 22266 consensus_queue.cc:1035] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.008s
I20250109 08:57:40.577315 22264 consensus_queue.cc:1035] T edc9db96e0ff4aa5839a6922f3eb5f24 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20250109 08:57:40.580610 22264 consensus_queue.cc:1035] T edc9db96e0ff4aa5839a6922f3eb5f24 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:57:40.582245 21660 raft_consensus.cc:1270] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Refusing update from remote peer 8bf6e26e98b14566899578a990b27d47: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:57:40.587971 22265 consensus_queue.cc:1035] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20250109 08:57:40.588966 22264 consensus_queue.cc:1035] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:57:40.605037 22265 raft_consensus.cc:2949] T edc9db96e0ff4aa5839a6922f3eb5f24 P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:40.609887 21503 tablet_service.cc:1514] Processing DeleteTablet for tablet 90a4a2381cbe40f0b204be325ee9fea1 with delete_type TABLET_DATA_TOMBSTONED (TS 8bf6e26e98b14566899578a990b27d47 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:39242
I20250109 08:57:40.606451 21660 raft_consensus.cc:2949] T edc9db96e0ff4aa5839a6922f3eb5f24 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:40.611857 21584 raft_consensus.cc:2949] T edc9db96e0ff4aa5839a6922f3eb5f24 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:40.612964 22277 tablet_replica.cc:331] T 90a4a2381cbe40f0b204be325ee9fea1 P 8bf6e26e98b14566899578a990b27d47: stopping tablet replica
I20250109 08:57:40.614434 22277 raft_consensus.cc:2238] T 90a4a2381cbe40f0b204be325ee9fea1 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:57:40.615142 22277 raft_consensus.cc:2267] T 90a4a2381cbe40f0b204be325ee9fea1 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:57:40.621001 22277 ts_tablet_manager.cc:1905] T 90a4a2381cbe40f0b204be325ee9fea1 P 8bf6e26e98b14566899578a990b27d47: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250109 08:57:40.627172 22177 raft_consensus.cc:2949] T edc9db96e0ff4aa5839a6922f3eb5f24 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:40.629377 21408 catalog_manager.cc:5526] T edc9db96e0ff4aa5839a6922f3eb5f24 P a2b1a692435241ee988e0bd1661cd4c3 reported cstate change: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8bf6e26e98b14566899578a990b27d47" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:40.649601 22277 ts_tablet_manager.cc:1918] T 90a4a2381cbe40f0b204be325ee9fea1 P 8bf6e26e98b14566899578a990b27d47: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20250109 08:57:40.650190 22277 log.cc:1198] T 90a4a2381cbe40f0b204be325ee9fea1 P 8bf6e26e98b14566899578a990b27d47: Deleting WAL directory at /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736412987538469-20363-0/minicluster-data/ts-0-root/wals/90a4a2381cbe40f0b204be325ee9fea1
I20250109 08:57:40.654110 21396 catalog_manager.cc:4872] TS 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): tablet 90a4a2381cbe40f0b204be325ee9fea1 (table test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]) successfully deleted
I20250109 08:57:40.656322 22266 raft_consensus.cc:2949] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:40.658835 22178 raft_consensus.cc:2949] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:40.658849 21657 raft_consensus.cc:2949] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:40.662325 21511 consensus_queue.cc:237] T edc9db96e0ff4aa5839a6922f3eb5f24 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:57:40.673653 21658 raft_consensus.cc:1270] T edc9db96e0ff4aa5839a6922f3eb5f24 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Refusing update from remote peer 8bf6e26e98b14566899578a990b27d47: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250109 08:57:40.673316 21585 raft_consensus.cc:2949] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:40.675537 22272 consensus_queue.cc:1035] T edc9db96e0ff4aa5839a6922f3eb5f24 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20250109 08:57:40.684247 22221 raft_consensus.cc:2949] T edc9db96e0ff4aa5839a6922f3eb5f24 P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:40.689182 22178 raft_consensus.cc:1270] T edc9db96e0ff4aa5839a6922f3eb5f24 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Refusing update from remote peer 8bf6e26e98b14566899578a990b27d47: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250109 08:57:40.692693 21657 raft_consensus.cc:2949] T edc9db96e0ff4aa5839a6922f3eb5f24 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:40.695006 21412 catalog_manager.cc:5526] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 787ddcc824c2490a808fc5a53452dc5e reported cstate change: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8bf6e26e98b14566899578a990b27d47" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:40.698621 22266 consensus_queue.cc:1035] T edc9db96e0ff4aa5839a6922f3eb5f24 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250109 08:57:40.705925 22178 raft_consensus.cc:2949] T edc9db96e0ff4aa5839a6922f3eb5f24 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:40.709565 21396 catalog_manager.cc:5039] ChangeConfig:REMOVE_PEER RPC for tablet edc9db96e0ff4aa5839a6922f3eb5f24 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250109 08:57:40.719439 21511 consensus_queue.cc:237] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:57:40.727813 21586 raft_consensus.cc:1270] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Refusing update from remote peer 8bf6e26e98b14566899578a990b27d47: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250109 08:57:40.727389 21410 catalog_manager.cc:5526] T edc9db96e0ff4aa5839a6922f3eb5f24 P 8bf6e26e98b14566899578a990b27d47 reported cstate change: config changed from index 3 to 4, VOTER a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194) evicted. New cstate: current_term: 1 leader_uuid: "8bf6e26e98b14566899578a990b27d47" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20250109 08:57:40.729032 22177 raft_consensus.cc:1270] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Refusing update from remote peer 8bf6e26e98b14566899578a990b27d47: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250109 08:57:40.732736 22272 consensus_queue.cc:1035] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250109 08:57:40.744793 22265 raft_consensus.cc:2949] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:40.746798 22178 raft_consensus.cc:2949] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:40.747748 22264 consensus_queue.cc:1035] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250109 08:57:40.758620 21584 raft_consensus.cc:2949] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:40.760872 21570 tablet_service.cc:1514] Processing DeleteTablet for tablet edc9db96e0ff4aa5839a6922f3eb5f24 with delete_type TABLET_DATA_TOMBSTONED (TS a2b1a692435241ee988e0bd1661cd4c3 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:34732
I20250109 08:57:40.772622 21396 catalog_manager.cc:5039] ChangeConfig:REMOVE_PEER RPC for tablet 5d31dbfb2b4a4a6ca4500f92df9dbb98 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250109 08:57:40.777843 21410 catalog_manager.cc:5526] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 88935ab835544e9889d05a6587d8cf21 reported cstate change: config changed from index 3 to 4, VOTER 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195) evicted. New cstate: current_term: 1 leader_uuid: "8bf6e26e98b14566899578a990b27d47" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:40.791893 22279 tablet_replica.cc:331] T edc9db96e0ff4aa5839a6922f3eb5f24 P a2b1a692435241ee988e0bd1661cd4c3: stopping tablet replica
I20250109 08:57:40.794811 22279 raft_consensus.cc:2238] T edc9db96e0ff4aa5839a6922f3eb5f24 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:57:40.795543 22279 raft_consensus.cc:2267] T edc9db96e0ff4aa5839a6922f3eb5f24 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:57:40.798830 22279 ts_tablet_manager.cc:1905] T edc9db96e0ff4aa5839a6922f3eb5f24 P a2b1a692435241ee988e0bd1661cd4c3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250109 08:57:40.802948 21653 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d31dbfb2b4a4a6ca4500f92df9dbb98 with delete_type TABLET_DATA_TOMBSTONED (TS 787ddcc824c2490a808fc5a53452dc5e not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:58436
I20250109 08:57:40.813166 22279 ts_tablet_manager.cc:1918] T edc9db96e0ff4aa5839a6922f3eb5f24 P a2b1a692435241ee988e0bd1661cd4c3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20250109 08:57:40.813712 22279 log.cc:1198] T edc9db96e0ff4aa5839a6922f3eb5f24 P a2b1a692435241ee988e0bd1661cd4c3: Deleting WAL directory at /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736412987538469-20363-0/minicluster-data/ts-1-root/wals/edc9db96e0ff4aa5839a6922f3eb5f24
I20250109 08:57:40.815402 21397 catalog_manager.cc:4872] TS a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): tablet edc9db96e0ff4aa5839a6922f3eb5f24 (table test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]) successfully deleted
I20250109 08:57:40.819615 22280 tablet_replica.cc:331] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 787ddcc824c2490a808fc5a53452dc5e: stopping tablet replica
I20250109 08:57:40.820640 22280 raft_consensus.cc:2238] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:57:40.821271 22280 raft_consensus.cc:2267] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:57:40.825114 22280 ts_tablet_manager.cc:1905] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 787ddcc824c2490a808fc5a53452dc5e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250109 08:57:40.828290 21907 raft_consensus.cc:1059] T 20ea30dba4314290ae45d9b4c13bebdd P 787ddcc824c2490a808fc5a53452dc5e: attempting to promote NON_VOTER 88935ab835544e9889d05a6587d8cf21 to VOTER
I20250109 08:57:40.830518 21907 consensus_queue.cc:237] T 20ea30dba4314290ae45d9b4c13bebdd P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:57:40.837579 21585 raft_consensus.cc:1270] T 20ea30dba4314290ae45d9b4c13bebdd P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:57:40.839329 21511 raft_consensus.cc:1270] T 20ea30dba4314290ae45d9b4c13bebdd P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:57:40.841282 22178 raft_consensus.cc:1270] T 20ea30dba4314290ae45d9b4c13bebdd P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:57:40.841065 21907 consensus_queue.cc:1035] T 20ea30dba4314290ae45d9b4c13bebdd P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:57:40.839990 22280 ts_tablet_manager.cc:1918] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 787ddcc824c2490a808fc5a53452dc5e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20250109 08:57:40.841904 22280 log.cc:1198] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 787ddcc824c2490a808fc5a53452dc5e: Deleting WAL directory at /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736412987538469-20363-0/minicluster-data/ts-2-root/wals/5d31dbfb2b4a4a6ca4500f92df9dbb98
I20250109 08:57:40.842679 22208 consensus_queue.cc:1035] T 20ea30dba4314290ae45d9b4c13bebdd P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:57:40.843513 21907 consensus_queue.cc:1035] T 20ea30dba4314290ae45d9b4c13bebdd P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:57:40.843709 21396 catalog_manager.cc:4872] TS 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731): tablet 5d31dbfb2b4a4a6ca4500f92df9dbb98 (table test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]) successfully deleted
I20250109 08:57:40.857756 22251 raft_consensus.cc:2949] T 20ea30dba4314290ae45d9b4c13bebdd P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:40.860947 21511 raft_consensus.cc:2949] T 20ea30dba4314290ae45d9b4c13bebdd P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:40.862141 22178 raft_consensus.cc:2949] T 20ea30dba4314290ae45d9b4c13bebdd P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:40.874720 21585 raft_consensus.cc:2949] T 20ea30dba4314290ae45d9b4c13bebdd P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:40.877893 21408 catalog_manager.cc:5526] T 20ea30dba4314290ae45d9b4c13bebdd P 787ddcc824c2490a808fc5a53452dc5e reported cstate change: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "787ddcc824c2490a808fc5a53452dc5e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250109 08:57:40.892987 21658 consensus_queue.cc:237] T 20ea30dba4314290ae45d9b4c13bebdd P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:57:40.897249 21511 raft_consensus.cc:1270] T 20ea30dba4314290ae45d9b4c13bebdd P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250109 08:57:40.898531 22178 raft_consensus.cc:1270] T 20ea30dba4314290ae45d9b4c13bebdd P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250109 08:57:40.898432 22251 consensus_queue.cc:1035] T 20ea30dba4314290ae45d9b4c13bebdd P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250109 08:57:40.899749 21758 consensus_queue.cc:1035] T 20ea30dba4314290ae45d9b4c13bebdd P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250109 08:57:40.906625 22251 raft_consensus.cc:2949] T 20ea30dba4314290ae45d9b4c13bebdd P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:40.908552 22178 raft_consensus.cc:2949] T 20ea30dba4314290ae45d9b4c13bebdd P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:40.909840 21511 raft_consensus.cc:2949] T 20ea30dba4314290ae45d9b4c13bebdd P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:40.918483 21396 catalog_manager.cc:5039] ChangeConfig:REMOVE_PEER RPC for tablet 20ea30dba4314290ae45d9b4c13bebdd with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250109 08:57:40.924579 21411 catalog_manager.cc:5526] T 20ea30dba4314290ae45d9b4c13bebdd P 88935ab835544e9889d05a6587d8cf21 reported cstate change: config changed from index 3 to 4, VOTER a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194) evicted. New cstate: current_term: 1 leader_uuid: "787ddcc824c2490a808fc5a53452dc5e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:40.933140 21570 tablet_service.cc:1514] Processing DeleteTablet for tablet 20ea30dba4314290ae45d9b4c13bebdd with delete_type TABLET_DATA_TOMBSTONED (TS a2b1a692435241ee988e0bd1661cd4c3 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:34732
I20250109 08:57:40.934309 22279 tablet_replica.cc:331] T 20ea30dba4314290ae45d9b4c13bebdd P a2b1a692435241ee988e0bd1661cd4c3: stopping tablet replica
I20250109 08:57:40.935349 22279 raft_consensus.cc:2238] T 20ea30dba4314290ae45d9b4c13bebdd P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:57:40.936077 22279 raft_consensus.cc:2267] T 20ea30dba4314290ae45d9b4c13bebdd P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:57:40.939113 22279 ts_tablet_manager.cc:1905] T 20ea30dba4314290ae45d9b4c13bebdd P a2b1a692435241ee988e0bd1661cd4c3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250109 08:57:40.949705 22279 ts_tablet_manager.cc:1918] T 20ea30dba4314290ae45d9b4c13bebdd P a2b1a692435241ee988e0bd1661cd4c3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20250109 08:57:40.950011 22279 log.cc:1198] T 20ea30dba4314290ae45d9b4c13bebdd P a2b1a692435241ee988e0bd1661cd4c3: Deleting WAL directory at /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736412987538469-20363-0/minicluster-data/ts-1-root/wals/20ea30dba4314290ae45d9b4c13bebdd
I20250109 08:57:40.951032 21397 catalog_manager.cc:4872] TS a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): tablet 20ea30dba4314290ae45d9b4c13bebdd (table test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]) successfully deleted
I20250109 08:57:41.984233 21511 consensus_queue.cc:237] T db34a00c02b84ee988edc7d6e344c1f5 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:57:41.989651 21584 raft_consensus.cc:1270] T db34a00c02b84ee988edc7d6e344c1f5 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Refusing update from remote peer 8bf6e26e98b14566899578a990b27d47: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20250109 08:57:41.989892 21478 consensus_peers.cc:487] T db34a00c02b84ee988edc7d6e344c1f5 P 8bf6e26e98b14566899578a990b27d47 -> Peer 88935ab835544e9889d05a6587d8cf21 (127.19.226.196:44363): Couldn't send request to peer 88935ab835544e9889d05a6587d8cf21. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: db34a00c02b84ee988edc7d6e344c1f5. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:57:41.990571 21657 raft_consensus.cc:1270] T db34a00c02b84ee988edc7d6e344c1f5 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Refusing update from remote peer 8bf6e26e98b14566899578a990b27d47: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250109 08:57:41.991094 22286 consensus_queue.cc:1035] T db34a00c02b84ee988edc7d6e344c1f5 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20250109 08:57:41.992635 22265 consensus_queue.cc:1035] T db34a00c02b84ee988edc7d6e344c1f5 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:57:41.999933 22265 raft_consensus.cc:2949] T db34a00c02b84ee988edc7d6e344c1f5 P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:57:42.005066 21657 raft_consensus.cc:2949] T db34a00c02b84ee988edc7d6e344c1f5 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:57:42.013806 21585 raft_consensus.cc:2949] T db34a00c02b84ee988edc7d6e344c1f5 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:57:42.017427 21412 catalog_manager.cc:5526] T db34a00c02b84ee988edc7d6e344c1f5 P 8bf6e26e98b14566899578a990b27d47 reported cstate change: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New cstate: current_term: 1 leader_uuid: "8bf6e26e98b14566899578a990b27d47" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20250109 08:57:42.023054 21657 consensus_queue.cc:237] T 306c4892bbff4dbfbe5a0645294aa627 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:57:42.044174 21585 raft_consensus.cc:1270] T 306c4892bbff4dbfbe5a0645294aa627 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250109 08:57:42.044272 21511 raft_consensus.cc:1270] T 306c4892bbff4dbfbe5a0645294aa627 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20250109 08:57:42.045027 21627 consensus_peers.cc:487] T 306c4892bbff4dbfbe5a0645294aa627 P 787ddcc824c2490a808fc5a53452dc5e -> Peer 88935ab835544e9889d05a6587d8cf21 (127.19.226.196:44363): Couldn't send request to peer 88935ab835544e9889d05a6587d8cf21. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 306c4892bbff4dbfbe5a0645294aa627. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:57:42.045492 22287 consensus_queue.cc:1035] T 306c4892bbff4dbfbe5a0645294aa627 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:57:42.046350 22208 consensus_queue.cc:1035] T 306c4892bbff4dbfbe5a0645294aa627 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:57:42.054777 22208 raft_consensus.cc:2949] T 306c4892bbff4dbfbe5a0645294aa627 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:57:42.055765 21585 raft_consensus.cc:2949] T 306c4892bbff4dbfbe5a0645294aa627 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:57:42.062242 21513 raft_consensus.cc:2949] T 306c4892bbff4dbfbe5a0645294aa627 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:57:42.071166 21657 consensus_queue.cc:237] T 0f3bde1a1ccd4c359a2e80feca968453 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:57:42.077325 21410 catalog_manager.cc:5526] T 306c4892bbff4dbfbe5a0645294aa627 P 787ddcc824c2490a808fc5a53452dc5e reported cstate change: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New cstate: current_term: 1 leader_uuid: "787ddcc824c2490a808fc5a53452dc5e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20250109 08:57:42.089787 21585 raft_consensus.cc:1270] T 0f3bde1a1ccd4c359a2e80feca968453 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20250109 08:57:42.090374 21627 consensus_peers.cc:487] T 0f3bde1a1ccd4c359a2e80feca968453 P 787ddcc824c2490a808fc5a53452dc5e -> Peer 88935ab835544e9889d05a6587d8cf21 (127.19.226.196:44363): Couldn't send request to peer 88935ab835544e9889d05a6587d8cf21. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0f3bde1a1ccd4c359a2e80feca968453. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:57:42.091238 21513 raft_consensus.cc:1270] T 0f3bde1a1ccd4c359a2e80feca968453 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250109 08:57:42.091385 22287 consensus_queue.cc:1035] T 0f3bde1a1ccd4c359a2e80feca968453 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20250109 08:57:42.098219 22287 consensus_queue.cc:1035] T 0f3bde1a1ccd4c359a2e80feca968453 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:57:42.122215 22208 raft_consensus.cc:2949] T 0f3bde1a1ccd4c359a2e80feca968453 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:57:42.125610 21585 raft_consensus.cc:2949] T 0f3bde1a1ccd4c359a2e80feca968453 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:57:42.144743 21411 catalog_manager.cc:5526] T 0f3bde1a1ccd4c359a2e80feca968453 P a2b1a692435241ee988e0bd1661cd4c3 reported cstate change: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New cstate: current_term: 1 leader_uuid: "787ddcc824c2490a808fc5a53452dc5e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:57:42.145252 21511 raft_consensus.cc:2949] T 0f3bde1a1ccd4c359a2e80feca968453 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:57:42.158599 21585 consensus_queue.cc:237] T 97304b06c7fa40ef8345a72c796c47f8 P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:57:42.165612 21657 raft_consensus.cc:1270] T 97304b06c7fa40ef8345a72c796c47f8 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Refusing update from remote peer a2b1a692435241ee988e0bd1661cd4c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250109 08:57:42.167094 22285 consensus_queue.cc:1035] T 97304b06c7fa40ef8345a72c796c47f8 P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:57:42.168175 21511 raft_consensus.cc:1270] T 97304b06c7fa40ef8345a72c796c47f8 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Refusing update from remote peer a2b1a692435241ee988e0bd1661cd4c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250109 08:57:42.173313 22285 consensus_queue.cc:1035] T 97304b06c7fa40ef8345a72c796c47f8 P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:57:42.177273 22210 raft_consensus.cc:2949] T 97304b06c7fa40ef8345a72c796c47f8 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:57:42.178684 21657 raft_consensus.cc:2949] T 97304b06c7fa40ef8345a72c796c47f8 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:57:42.197428 21411 catalog_manager.cc:5526] T 97304b06c7fa40ef8345a72c796c47f8 P 787ddcc824c2490a808fc5a53452dc5e reported cstate change: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New cstate: current_term: 1 leader_uuid: "a2b1a692435241ee988e0bd1661cd4c3" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
W20250109 08:57:42.257908 21553 consensus_peers.cc:487] T 97304b06c7fa40ef8345a72c796c47f8 P a2b1a692435241ee988e0bd1661cd4c3 -> Peer 88935ab835544e9889d05a6587d8cf21 (127.19.226.196:44363): Couldn't send request to peer 88935ab835544e9889d05a6587d8cf21. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 97304b06c7fa40ef8345a72c796c47f8. This is attempt 1: this message will repeat every 5th retry.
W20250109 08:57:42.260592 21457 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.BulkChangeConfig -> {remote=127.19.226.194:45527, user_credentials={real_user=slave}} blocked reactor thread for 66895.2us
I20250109 08:57:42.263278 21513 raft_consensus.cc:2949] T 97304b06c7fa40ef8345a72c796c47f8 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:57:42.674211 22320 ts_tablet_manager.cc:927] T 306c4892bbff4dbfbe5a0645294aa627 P 88935ab835544e9889d05a6587d8cf21: Initiating tablet copy from peer 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:42.675721 22320 tablet_copy_client.cc:323] T 306c4892bbff4dbfbe5a0645294aa627 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Beginning tablet copy session from remote peer at address 127.19.226.195:35731
I20250109 08:57:42.678282 21673 tablet_copy_service.cc:140] P 787ddcc824c2490a808fc5a53452dc5e: Received BeginTabletCopySession request for tablet 306c4892bbff4dbfbe5a0645294aa627 from peer 88935ab835544e9889d05a6587d8cf21 ({username='slave'} at 127.0.0.1:52376)
I20250109 08:57:42.678869 21673 tablet_copy_service.cc:161] P 787ddcc824c2490a808fc5a53452dc5e: Beginning new tablet copy session on tablet 306c4892bbff4dbfbe5a0645294aa627 from peer 88935ab835544e9889d05a6587d8cf21 at {username='slave'} at 127.0.0.1:52376: session id = 88935ab835544e9889d05a6587d8cf21-306c4892bbff4dbfbe5a0645294aa627
I20250109 08:57:42.687402 21673 tablet_copy_source_session.cc:215] T 306c4892bbff4dbfbe5a0645294aa627 P 787ddcc824c2490a808fc5a53452dc5e: Tablet Copy: opened 0 blocks and 1 log segments
I20250109 08:57:42.690755 22320 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 306c4892bbff4dbfbe5a0645294aa627. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:42.720554 22319 ts_tablet_manager.cc:927] T db34a00c02b84ee988edc7d6e344c1f5 P 88935ab835544e9889d05a6587d8cf21: Initiating tablet copy from peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725)
I20250109 08:57:42.724941 22320 tablet_copy_client.cc:806] T 306c4892bbff4dbfbe5a0645294aa627 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Starting download of 0 data blocks...
I20250109 08:57:42.725556 22320 tablet_copy_client.cc:670] T 306c4892bbff4dbfbe5a0645294aa627 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Starting download of 1 WAL segments...
I20250109 08:57:42.727048 22318 ts_tablet_manager.cc:927] T 0f3bde1a1ccd4c359a2e80feca968453 P 88935ab835544e9889d05a6587d8cf21: Initiating tablet copy from peer 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:42.730159 22319 tablet_copy_client.cc:323] T db34a00c02b84ee988edc7d6e344c1f5 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Beginning tablet copy session from remote peer at address 127.19.226.193:42725
I20250109 08:57:42.732540 22320 tablet_copy_client.cc:538] T 306c4892bbff4dbfbe5a0645294aa627 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250109 08:57:42.732656 21524 tablet_copy_service.cc:140] P 8bf6e26e98b14566899578a990b27d47: Received BeginTabletCopySession request for tablet db34a00c02b84ee988edc7d6e344c1f5 from peer 88935ab835544e9889d05a6587d8cf21 ({username='slave'} at 127.0.0.1:41812)
I20250109 08:57:42.733239 21524 tablet_copy_service.cc:161] P 8bf6e26e98b14566899578a990b27d47: Beginning new tablet copy session on tablet db34a00c02b84ee988edc7d6e344c1f5 from peer 88935ab835544e9889d05a6587d8cf21 at {username='slave'} at 127.0.0.1:41812: session id = 88935ab835544e9889d05a6587d8cf21-db34a00c02b84ee988edc7d6e344c1f5
I20250109 08:57:42.733628 22318 tablet_copy_client.cc:323] T 0f3bde1a1ccd4c359a2e80feca968453 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Beginning tablet copy session from remote peer at address 127.19.226.195:35731
I20250109 08:57:42.735152 21673 tablet_copy_service.cc:140] P 787ddcc824c2490a808fc5a53452dc5e: Received BeginTabletCopySession request for tablet 0f3bde1a1ccd4c359a2e80feca968453 from peer 88935ab835544e9889d05a6587d8cf21 ({username='slave'} at 127.0.0.1:52376)
I20250109 08:57:42.735634 21673 tablet_copy_service.cc:161] P 787ddcc824c2490a808fc5a53452dc5e: Beginning new tablet copy session on tablet 0f3bde1a1ccd4c359a2e80feca968453 from peer 88935ab835544e9889d05a6587d8cf21 at {username='slave'} at 127.0.0.1:52376: session id = 88935ab835544e9889d05a6587d8cf21-0f3bde1a1ccd4c359a2e80feca968453
I20250109 08:57:42.740126 21524 tablet_copy_source_session.cc:215] T db34a00c02b84ee988edc7d6e344c1f5 P 8bf6e26e98b14566899578a990b27d47: Tablet Copy: opened 0 blocks and 1 log segments
I20250109 08:57:42.742820 22320 tablet_bootstrap.cc:492] T 306c4892bbff4dbfbe5a0645294aa627 P 88935ab835544e9889d05a6587d8cf21: Bootstrap starting.
I20250109 08:57:42.743937 22319 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db34a00c02b84ee988edc7d6e344c1f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:42.744474 21673 tablet_copy_source_session.cc:215] T 0f3bde1a1ccd4c359a2e80feca968453 P 787ddcc824c2490a808fc5a53452dc5e: Tablet Copy: opened 0 blocks and 1 log segments
I20250109 08:57:42.747942 22318 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f3bde1a1ccd4c359a2e80feca968453. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:42.758579 22319 tablet_copy_client.cc:806] T db34a00c02b84ee988edc7d6e344c1f5 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Starting download of 0 data blocks...
I20250109 08:57:42.759168 22319 tablet_copy_client.cc:670] T db34a00c02b84ee988edc7d6e344c1f5 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Starting download of 1 WAL segments...
I20250109 08:57:42.762774 22319 tablet_copy_client.cc:538] T db34a00c02b84ee988edc7d6e344c1f5 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250109 08:57:42.763787 22318 tablet_copy_client.cc:806] T 0f3bde1a1ccd4c359a2e80feca968453 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Starting download of 0 data blocks...
I20250109 08:57:42.764384 22318 tablet_copy_client.cc:670] T 0f3bde1a1ccd4c359a2e80feca968453 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Starting download of 1 WAL segments...
I20250109 08:57:42.767784 22318 tablet_copy_client.cc:538] T 0f3bde1a1ccd4c359a2e80feca968453 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250109 08:57:42.770655 22325 ts_tablet_manager.cc:927] T 97304b06c7fa40ef8345a72c796c47f8 P 88935ab835544e9889d05a6587d8cf21: Initiating tablet copy from peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
I20250109 08:57:42.771065 22319 tablet_bootstrap.cc:492] T db34a00c02b84ee988edc7d6e344c1f5 P 88935ab835544e9889d05a6587d8cf21: Bootstrap starting.
I20250109 08:57:42.784080 22320 tablet_bootstrap.cc:492] T 306c4892bbff4dbfbe5a0645294aa627 P 88935ab835544e9889d05a6587d8cf21: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250109 08:57:42.785149 22320 tablet_bootstrap.cc:492] T 306c4892bbff4dbfbe5a0645294aa627 P 88935ab835544e9889d05a6587d8cf21: Bootstrap complete.
I20250109 08:57:42.787639 22318 tablet_bootstrap.cc:492] T 0f3bde1a1ccd4c359a2e80feca968453 P 88935ab835544e9889d05a6587d8cf21: Bootstrap starting.
I20250109 08:57:42.792572 22325 tablet_copy_client.cc:323] T 97304b06c7fa40ef8345a72c796c47f8 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Beginning tablet copy session from remote peer at address 127.19.226.194:45527
I20250109 08:57:42.793679 22320 ts_tablet_manager.cc:1397] T 306c4892bbff4dbfbe5a0645294aa627 P 88935ab835544e9889d05a6587d8cf21: Time spent bootstrapping tablet: real 0.051s user 0.016s sys 0.010s
I20250109 08:57:42.819928 22320 raft_consensus.cc:357] T 306c4892bbff4dbfbe5a0645294aa627 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:57:42.820986 22320 raft_consensus.cc:738] T 306c4892bbff4dbfbe5a0645294aa627 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 88935ab835544e9889d05a6587d8cf21, State: Initialized, Role: LEARNER
I20250109 08:57:42.821691 22320 consensus_queue.cc:260] T 306c4892bbff4dbfbe5a0645294aa627 P 88935ab835544e9889d05a6587d8cf21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:57:42.902478 22319 tablet_bootstrap.cc:492] T db34a00c02b84ee988edc7d6e344c1f5 P 88935ab835544e9889d05a6587d8cf21: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250109 08:57:42.902700 22320 ts_tablet_manager.cc:1428] T 306c4892bbff4dbfbe5a0645294aa627 P 88935ab835544e9889d05a6587d8cf21: Time spent starting tablet: real 0.103s user 0.009s sys 0.023s
I20250109 08:57:42.903439 22319 tablet_bootstrap.cc:492] T db34a00c02b84ee988edc7d6e344c1f5 P 88935ab835544e9889d05a6587d8cf21: Bootstrap complete.
I20250109 08:57:42.904167 22319 ts_tablet_manager.cc:1397] T db34a00c02b84ee988edc7d6e344c1f5 P 88935ab835544e9889d05a6587d8cf21: Time spent bootstrapping tablet: real 0.133s user 0.067s sys 0.042s
I20250109 08:57:42.904738 21673 tablet_copy_service.cc:342] P 787ddcc824c2490a808fc5a53452dc5e: Request end of tablet copy session 88935ab835544e9889d05a6587d8cf21-306c4892bbff4dbfbe5a0645294aa627 received from {username='slave'} at 127.0.0.1:52376
I20250109 08:57:42.904945 21599 tablet_copy_service.cc:140] P a2b1a692435241ee988e0bd1661cd4c3: Received BeginTabletCopySession request for tablet 97304b06c7fa40ef8345a72c796c47f8 from peer 88935ab835544e9889d05a6587d8cf21 ({username='slave'} at 127.0.0.1:33652)
I20250109 08:57:42.905232 21673 tablet_copy_service.cc:434] P 787ddcc824c2490a808fc5a53452dc5e: ending tablet copy session 88935ab835544e9889d05a6587d8cf21-306c4892bbff4dbfbe5a0645294aa627 on tablet 306c4892bbff4dbfbe5a0645294aa627 with peer 88935ab835544e9889d05a6587d8cf21
I20250109 08:57:42.905551 21599 tablet_copy_service.cc:161] P a2b1a692435241ee988e0bd1661cd4c3: Beginning new tablet copy session on tablet 97304b06c7fa40ef8345a72c796c47f8 from peer 88935ab835544e9889d05a6587d8cf21 at {username='slave'} at 127.0.0.1:33652: session id = 88935ab835544e9889d05a6587d8cf21-97304b06c7fa40ef8345a72c796c47f8
I20250109 08:57:42.910789 22319 raft_consensus.cc:357] T db34a00c02b84ee988edc7d6e344c1f5 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:57:42.911828 22319 raft_consensus.cc:738] T db34a00c02b84ee988edc7d6e344c1f5 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 88935ab835544e9889d05a6587d8cf21, State: Initialized, Role: LEARNER
I20250109 08:57:42.913339 21599 tablet_copy_source_session.cc:215] T 97304b06c7fa40ef8345a72c796c47f8 P a2b1a692435241ee988e0bd1661cd4c3: Tablet Copy: opened 0 blocks and 1 log segments
I20250109 08:57:42.912565 22319 consensus_queue.cc:260] T db34a00c02b84ee988edc7d6e344c1f5 P 88935ab835544e9889d05a6587d8cf21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:57:42.915561 22318 tablet_bootstrap.cc:492] T 0f3bde1a1ccd4c359a2e80feca968453 P 88935ab835544e9889d05a6587d8cf21: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250109 08:57:42.916527 22318 tablet_bootstrap.cc:492] T 0f3bde1a1ccd4c359a2e80feca968453 P 88935ab835544e9889d05a6587d8cf21: Bootstrap complete.
I20250109 08:57:42.916427 22325 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97304b06c7fa40ef8345a72c796c47f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:42.916882 22319 ts_tablet_manager.cc:1428] T db34a00c02b84ee988edc7d6e344c1f5 P 88935ab835544e9889d05a6587d8cf21: Time spent starting tablet: real 0.012s user 0.007s sys 0.000s
I20250109 08:57:42.917292 22318 ts_tablet_manager.cc:1397] T 0f3bde1a1ccd4c359a2e80feca968453 P 88935ab835544e9889d05a6587d8cf21: Time spent bootstrapping tablet: real 0.130s user 0.047s sys 0.053s
I20250109 08:57:42.918401 21524 tablet_copy_service.cc:342] P 8bf6e26e98b14566899578a990b27d47: Request end of tablet copy session 88935ab835544e9889d05a6587d8cf21-db34a00c02b84ee988edc7d6e344c1f5 received from {username='slave'} at 127.0.0.1:41812
I20250109 08:57:42.918800 21524 tablet_copy_service.cc:434] P 8bf6e26e98b14566899578a990b27d47: ending tablet copy session 88935ab835544e9889d05a6587d8cf21-db34a00c02b84ee988edc7d6e344c1f5 on tablet db34a00c02b84ee988edc7d6e344c1f5 with peer 88935ab835544e9889d05a6587d8cf21
I20250109 08:57:42.919952 22318 raft_consensus.cc:357] T 0f3bde1a1ccd4c359a2e80feca968453 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:57:42.920830 22318 raft_consensus.cc:738] T 0f3bde1a1ccd4c359a2e80feca968453 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 88935ab835544e9889d05a6587d8cf21, State: Initialized, Role: LEARNER
I20250109 08:57:42.921468 22318 consensus_queue.cc:260] T 0f3bde1a1ccd4c359a2e80feca968453 P 88935ab835544e9889d05a6587d8cf21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:57:42.923810 22318 ts_tablet_manager.cc:1428] T 0f3bde1a1ccd4c359a2e80feca968453 P 88935ab835544e9889d05a6587d8cf21: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20250109 08:57:42.925175 21673 tablet_copy_service.cc:342] P 787ddcc824c2490a808fc5a53452dc5e: Request end of tablet copy session 88935ab835544e9889d05a6587d8cf21-0f3bde1a1ccd4c359a2e80feca968453 received from {username='slave'} at 127.0.0.1:52376
I20250109 08:57:42.925554 21673 tablet_copy_service.cc:434] P 787ddcc824c2490a808fc5a53452dc5e: ending tablet copy session 88935ab835544e9889d05a6587d8cf21-0f3bde1a1ccd4c359a2e80feca968453 on tablet 0f3bde1a1ccd4c359a2e80feca968453 with peer 88935ab835544e9889d05a6587d8cf21
I20250109 08:57:42.939864 22325 tablet_copy_client.cc:806] T 97304b06c7fa40ef8345a72c796c47f8 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Starting download of 0 data blocks...
I20250109 08:57:42.940357 22325 tablet_copy_client.cc:670] T 97304b06c7fa40ef8345a72c796c47f8 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Starting download of 1 WAL segments...
I20250109 08:57:42.943483 22325 tablet_copy_client.cc:538] T 97304b06c7fa40ef8345a72c796c47f8 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250109 08:57:42.951692 22325 tablet_bootstrap.cc:492] T 97304b06c7fa40ef8345a72c796c47f8 P 88935ab835544e9889d05a6587d8cf21: Bootstrap starting.
I20250109 08:57:43.007746 22325 tablet_bootstrap.cc:492] T 97304b06c7fa40ef8345a72c796c47f8 P 88935ab835544e9889d05a6587d8cf21: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250109 08:57:43.008657 22325 tablet_bootstrap.cc:492] T 97304b06c7fa40ef8345a72c796c47f8 P 88935ab835544e9889d05a6587d8cf21: Bootstrap complete.
I20250109 08:57:43.009212 22325 ts_tablet_manager.cc:1397] T 97304b06c7fa40ef8345a72c796c47f8 P 88935ab835544e9889d05a6587d8cf21: Time spent bootstrapping tablet: real 0.058s user 0.029s sys 0.027s
I20250109 08:57:43.011504 22325 raft_consensus.cc:357] T 97304b06c7fa40ef8345a72c796c47f8 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:57:43.012257 22325 raft_consensus.cc:738] T 97304b06c7fa40ef8345a72c796c47f8 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 88935ab835544e9889d05a6587d8cf21, State: Initialized, Role: LEARNER
I20250109 08:57:43.012802 22325 consensus_queue.cc:260] T 97304b06c7fa40ef8345a72c796c47f8 P 88935ab835544e9889d05a6587d8cf21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:57:43.015055 22325 ts_tablet_manager.cc:1428] T 97304b06c7fa40ef8345a72c796c47f8 P 88935ab835544e9889d05a6587d8cf21: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20250109 08:57:43.016598 21599 tablet_copy_service.cc:342] P a2b1a692435241ee988e0bd1661cd4c3: Request end of tablet copy session 88935ab835544e9889d05a6587d8cf21-97304b06c7fa40ef8345a72c796c47f8 received from {username='slave'} at 127.0.0.1:33652
I20250109 08:57:43.017012 21599 tablet_copy_service.cc:434] P a2b1a692435241ee988e0bd1661cd4c3: ending tablet copy session 88935ab835544e9889d05a6587d8cf21-97304b06c7fa40ef8345a72c796c47f8 on tablet 97304b06c7fa40ef8345a72c796c47f8 with peer 88935ab835544e9889d05a6587d8cf21
I20250109 08:57:43.061172 22178 raft_consensus.cc:1212] T db34a00c02b84ee988edc7d6e344c1f5 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20250109 08:57:43.098302 22178 raft_consensus.cc:1212] T 0f3bde1a1ccd4c359a2e80feca968453 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20250109 08:57:43.156123 22178 raft_consensus.cc:1212] T 97304b06c7fa40ef8345a72c796c47f8 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20250109 08:57:43.209394 22178 raft_consensus.cc:1212] T 306c4892bbff4dbfbe5a0645294aa627 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
W20250109 08:57:43.474553 21443 debug-util.cc:398] Leaking SignalData structure 0x7b0800041a00 after lost signal to thread 20366
W20250109 08:57:43.475437 21443 debug-util.cc:398] Leaking SignalData structure 0x7b0800036080 after lost signal to thread 21446
W20250109 08:57:43.476295 21443 debug-util.cc:398] Leaking SignalData structure 0x7b0800154900 after lost signal to thread 21538
W20250109 08:57:43.477718 21443 debug-util.cc:398] Leaking SignalData structure 0x7b080009fb00 after lost signal to thread 21687
W20250109 08:57:43.478473 21443 debug-util.cc:398] Leaking SignalData structure 0x7b08002930a0 after lost signal to thread 22202
I20250109 08:57:43.511198 22312 raft_consensus.cc:1059] T 0f3bde1a1ccd4c359a2e80feca968453 P 787ddcc824c2490a808fc5a53452dc5e: attempting to promote NON_VOTER 88935ab835544e9889d05a6587d8cf21 to VOTER
I20250109 08:57:43.513801 22312 consensus_queue.cc:237] T 0f3bde1a1ccd4c359a2e80feca968453 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:57:43.520677 21584 raft_consensus.cc:1270] T 0f3bde1a1ccd4c359a2e80feca968453 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:57:43.522948 22311 consensus_queue.cc:1035] T 0f3bde1a1ccd4c359a2e80feca968453 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:57:43.528342 21512 raft_consensus.cc:1270] T 0f3bde1a1ccd4c359a2e80feca968453 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:57:43.529953 22311 consensus_queue.cc:1035] T 0f3bde1a1ccd4c359a2e80feca968453 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:57:43.535274 22178 raft_consensus.cc:1270] T 0f3bde1a1ccd4c359a2e80feca968453 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:57:43.536655 22311 consensus_queue.cc:1035] T 0f3bde1a1ccd4c359a2e80feca968453 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:57:43.568552 22312 raft_consensus.cc:2949] T 0f3bde1a1ccd4c359a2e80feca968453 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:43.571677 21512 raft_consensus.cc:2949] T 0f3bde1a1ccd4c359a2e80feca968453 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:43.575429 22178 raft_consensus.cc:2949] T 0f3bde1a1ccd4c359a2e80feca968453 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:43.590999 21412 catalog_manager.cc:5526] T 0f3bde1a1ccd4c359a2e80feca968453 P 8bf6e26e98b14566899578a990b27d47 reported cstate change: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "787ddcc824c2490a808fc5a53452dc5e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:43.610359 21581 raft_consensus.cc:2949] T 0f3bde1a1ccd4c359a2e80feca968453 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:43.617535 22317 raft_consensus.cc:1059] T 97304b06c7fa40ef8345a72c796c47f8 P a2b1a692435241ee988e0bd1661cd4c3: attempting to promote NON_VOTER 88935ab835544e9889d05a6587d8cf21 to VOTER
I20250109 08:57:43.619449 22317 consensus_queue.cc:237] T 97304b06c7fa40ef8345a72c796c47f8 P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:57:43.662577 22344 raft_consensus.cc:1059] T db34a00c02b84ee988edc7d6e344c1f5 P 8bf6e26e98b14566899578a990b27d47: attempting to promote NON_VOTER 88935ab835544e9889d05a6587d8cf21 to VOTER
I20250109 08:57:43.670852 21663 raft_consensus.cc:1270] T 97304b06c7fa40ef8345a72c796c47f8 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Refusing update from remote peer a2b1a692435241ee988e0bd1661cd4c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:57:43.679116 22347 consensus_queue.cc:1035] T 97304b06c7fa40ef8345a72c796c47f8 P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20250109 08:57:43.685643 22178 raft_consensus.cc:1270] T 97304b06c7fa40ef8345a72c796c47f8 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Refusing update from remote peer a2b1a692435241ee988e0bd1661cd4c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:57:43.686393 21513 raft_consensus.cc:1270] T 97304b06c7fa40ef8345a72c796c47f8 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Refusing update from remote peer a2b1a692435241ee988e0bd1661cd4c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:57:43.688338 22309 consensus_queue.cc:1035] T 97304b06c7fa40ef8345a72c796c47f8 P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.004s
I20250109 08:57:43.695643 22210 consensus_queue.cc:1035] T 97304b06c7fa40ef8345a72c796c47f8 P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:57:43.728202 22344 consensus_queue.cc:237] T db34a00c02b84ee988edc7d6e344c1f5 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:57:43.740924 22311 raft_consensus.cc:1059] T 306c4892bbff4dbfbe5a0645294aa627 P 787ddcc824c2490a808fc5a53452dc5e: attempting to promote NON_VOTER 88935ab835544e9889d05a6587d8cf21 to VOTER
I20250109 08:57:43.745874 22347 raft_consensus.cc:2949] T 97304b06c7fa40ef8345a72c796c47f8 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:43.743394 22311 consensus_queue.cc:237] T 306c4892bbff4dbfbe5a0645294aa627 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:57:43.749814 22178 raft_consensus.cc:2949] T 97304b06c7fa40ef8345a72c796c47f8 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:43.754673 22177 raft_consensus.cc:1270] T db34a00c02b84ee988edc7d6e344c1f5 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Refusing update from remote peer 8bf6e26e98b14566899578a990b27d47: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:57:43.757398 22342 consensus_queue.cc:1035] T db34a00c02b84ee988edc7d6e344c1f5 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:57:43.757659 21663 raft_consensus.cc:1270] T db34a00c02b84ee988edc7d6e344c1f5 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Refusing update from remote peer 8bf6e26e98b14566899578a990b27d47: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:57:43.761684 22342 consensus_queue.cc:1035] T db34a00c02b84ee988edc7d6e344c1f5 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:57:43.764801 21581 raft_consensus.cc:1270] T db34a00c02b84ee988edc7d6e344c1f5 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Refusing update from remote peer 8bf6e26e98b14566899578a990b27d47: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:57:43.776502 22342 consensus_queue.cc:1035] T db34a00c02b84ee988edc7d6e344c1f5 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:57:43.775758 21663 consensus_queue.cc:237] T 0f3bde1a1ccd4c359a2e80feca968453 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:57:43.781744 21411 catalog_manager.cc:5526] T 97304b06c7fa40ef8345a72c796c47f8 P 88935ab835544e9889d05a6587d8cf21 reported cstate change: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a2b1a692435241ee988e0bd1661cd4c3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:43.790244 21512 raft_consensus.cc:2949] T 97304b06c7fa40ef8345a72c796c47f8 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:43.826066 21584 raft_consensus.cc:1270] T 0f3bde1a1ccd4c359a2e80feca968453 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250109 08:57:43.836117 22178 raft_consensus.cc:1270] T 0f3bde1a1ccd4c359a2e80feca968453 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250109 08:57:43.836431 22358 consensus_queue.cc:1035] T 0f3bde1a1ccd4c359a2e80feca968453 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20250109 08:57:43.845947 22360 consensus_queue.cc:1035] T 0f3bde1a1ccd4c359a2e80feca968453 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20250109 08:57:43.856096 22176 raft_consensus.cc:2949] T 0f3bde1a1ccd4c359a2e80feca968453 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8bf6e26e98b14566899578a990b27d47 (127.19.226.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:43.866395 22360 raft_consensus.cc:2949] T 0f3bde1a1ccd4c359a2e80feca968453 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8bf6e26e98b14566899578a990b27d47 (127.19.226.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:43.877498 21657 raft_consensus.cc:2949] T 97304b06c7fa40ef8345a72c796c47f8 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:43.887761 21588 raft_consensus.cc:1270] T 306c4892bbff4dbfbe5a0645294aa627 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:57:43.891680 22178 raft_consensus.cc:1270] T 306c4892bbff4dbfbe5a0645294aa627 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:57:43.896867 22311 consensus_queue.cc:1035] T 306c4892bbff4dbfbe5a0645294aa627 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:57:43.897792 22358 consensus_queue.cc:1035] T 306c4892bbff4dbfbe5a0645294aa627 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:57:43.901732 21584 consensus_queue.cc:237] T 97304b06c7fa40ef8345a72c796c47f8 P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:57:43.922720 22178 raft_consensus.cc:2949] T db34a00c02b84ee988edc7d6e344c1f5 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:43.925835 22177 raft_consensus.cc:1270] T 97304b06c7fa40ef8345a72c796c47f8 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Refusing update from remote peer a2b1a692435241ee988e0bd1661cd4c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250109 08:57:43.922281 21583 raft_consensus.cc:2949] T 0f3bde1a1ccd4c359a2e80feca968453 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8bf6e26e98b14566899578a990b27d47 (127.19.226.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:43.934139 22317 consensus_queue.cc:1035] T 97304b06c7fa40ef8345a72c796c47f8 P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20250109 08:57:43.940753 21626 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.226.193:42725, user_credentials={real_user=slave}} blocked reactor thread for 77805.3us
I20250109 08:57:43.945482 21511 raft_consensus.cc:1270] T 306c4892bbff4dbfbe5a0645294aa627 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:57:43.923760 22342 raft_consensus.cc:2949] T db34a00c02b84ee988edc7d6e344c1f5 P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:43.953253 21658 raft_consensus.cc:2949] T db34a00c02b84ee988edc7d6e344c1f5 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:43.956902 21512 raft_consensus.cc:1270] T 97304b06c7fa40ef8345a72c796c47f8 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Refusing update from remote peer a2b1a692435241ee988e0bd1661cd4c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250109 08:57:43.960822 21408 catalog_manager.cc:5526] T 0f3bde1a1ccd4c359a2e80feca968453 P 88935ab835544e9889d05a6587d8cf21 reported cstate change: config changed from index 3 to 4, VOTER 8bf6e26e98b14566899578a990b27d47 (127.19.226.193) evicted. New cstate: current_term: 1 leader_uuid: "787ddcc824c2490a808fc5a53452dc5e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:43.964581 22347 consensus_queue.cc:1035] T 97304b06c7fa40ef8345a72c796c47f8 P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20250109 08:57:43.974355 22317 raft_consensus.cc:2949] T 97304b06c7fa40ef8345a72c796c47f8 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:43.980176 22312 consensus_queue.cc:1035] T 306c4892bbff4dbfbe5a0645294aa627 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20250109 08:57:43.980070 21396 catalog_manager.cc:5039] ChangeConfig:REMOVE_PEER RPC for tablet 0f3bde1a1ccd4c359a2e80feca968453 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250109 08:57:43.980597 21503 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f3bde1a1ccd4c359a2e80feca968453 with delete_type TABLET_DATA_TOMBSTONED (TS 8bf6e26e98b14566899578a990b27d47 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:39242
I20250109 08:57:43.985502 22178 raft_consensus.cc:2949] T 97304b06c7fa40ef8345a72c796c47f8 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:43.994114 21509 raft_consensus.cc:2949] T 97304b06c7fa40ef8345a72c796c47f8 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:43.998271 21408 catalog_manager.cc:5526] T db34a00c02b84ee988edc7d6e344c1f5 P 88935ab835544e9889d05a6587d8cf21 reported cstate change: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8bf6e26e98b14566899578a990b27d47" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:44.019383 21397 catalog_manager.cc:5039] ChangeConfig:REMOVE_PEER RPC for tablet 97304b06c7fa40ef8345a72c796c47f8 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250109 08:57:44.023363 22367 tablet_replica.cc:331] T 0f3bde1a1ccd4c359a2e80feca968453 P 8bf6e26e98b14566899578a990b27d47: stopping tablet replica
I20250109 08:57:44.031422 22358 raft_consensus.cc:2949] T 306c4892bbff4dbfbe5a0645294aa627 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:44.045209 21410 catalog_manager.cc:5526] T 97304b06c7fa40ef8345a72c796c47f8 P 8bf6e26e98b14566899578a990b27d47 reported cstate change: config changed from index 3 to 4, VOTER 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195) evicted. New cstate: current_term: 1 leader_uuid: "a2b1a692435241ee988e0bd1661cd4c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:44.052585 22178 raft_consensus.cc:2949] T 306c4892bbff4dbfbe5a0645294aa627 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:44.054456 22367 raft_consensus.cc:2238] T 0f3bde1a1ccd4c359a2e80feca968453 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:57:44.055384 22367 raft_consensus.cc:2267] T 0f3bde1a1ccd4c359a2e80feca968453 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:57:44.067293 21653 tablet_service.cc:1514] Processing DeleteTablet for tablet 97304b06c7fa40ef8345a72c796c47f8 with delete_type TABLET_DATA_TOMBSTONED (TS 787ddcc824c2490a808fc5a53452dc5e not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:58436
I20250109 08:57:44.068519 21584 raft_consensus.cc:2949] T 306c4892bbff4dbfbe5a0645294aa627 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:44.075106 21588 raft_consensus.cc:2949] T db34a00c02b84ee988edc7d6e344c1f5 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:44.087879 21408 catalog_manager.cc:5526] T 306c4892bbff4dbfbe5a0645294aa627 P 88935ab835544e9889d05a6587d8cf21 reported cstate change: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "787ddcc824c2490a808fc5a53452dc5e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:44.117014 22367 ts_tablet_manager.cc:1905] T 0f3bde1a1ccd4c359a2e80feca968453 P 8bf6e26e98b14566899578a990b27d47: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250109 08:57:44.117484 22371 tablet_replica.cc:331] T 97304b06c7fa40ef8345a72c796c47f8 P 787ddcc824c2490a808fc5a53452dc5e: stopping tablet replica
I20250109 08:57:44.118506 22371 raft_consensus.cc:2238] T 97304b06c7fa40ef8345a72c796c47f8 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:57:44.119233 22371 raft_consensus.cc:2267] T 97304b06c7fa40ef8345a72c796c47f8 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:57:44.125766 21512 raft_consensus.cc:2949] T 306c4892bbff4dbfbe5a0645294aa627 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:44.136474 22367 ts_tablet_manager.cc:1918] T 0f3bde1a1ccd4c359a2e80feca968453 P 8bf6e26e98b14566899578a990b27d47: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20250109 08:57:44.137845 22367 log.cc:1198] T 0f3bde1a1ccd4c359a2e80feca968453 P 8bf6e26e98b14566899578a990b27d47: Deleting WAL directory at /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736412987538469-20363-0/minicluster-data/ts-0-root/wals/0f3bde1a1ccd4c359a2e80feca968453
I20250109 08:57:44.139640 21396 catalog_manager.cc:4872] TS 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): tablet 0f3bde1a1ccd4c359a2e80feca968453 (table test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]) successfully deleted
I20250109 08:57:44.141316 22371 ts_tablet_manager.cc:1905] T 97304b06c7fa40ef8345a72c796c47f8 P 787ddcc824c2490a808fc5a53452dc5e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250109 08:57:44.151111 21658 consensus_queue.cc:237] T 306c4892bbff4dbfbe5a0645294aa627 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:57:44.164552 22178 raft_consensus.cc:1270] T 306c4892bbff4dbfbe5a0645294aa627 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250109 08:57:44.166039 22364 consensus_queue.cc:1035] T 306c4892bbff4dbfbe5a0645294aa627 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20250109 08:57:44.175665 21588 raft_consensus.cc:1270] T 306c4892bbff4dbfbe5a0645294aa627 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250109 08:57:44.176767 22178 raft_consensus.cc:2949] T 306c4892bbff4dbfbe5a0645294aa627 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8bf6e26e98b14566899578a990b27d47 (127.19.226.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:44.173851 22315 raft_consensus.cc:2949] T 306c4892bbff4dbfbe5a0645294aa627 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8bf6e26e98b14566899578a990b27d47 (127.19.226.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:44.179759 22364 consensus_queue.cc:1035] T 306c4892bbff4dbfbe5a0645294aa627 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250109 08:57:44.183264 21509 consensus_queue.cc:237] T db34a00c02b84ee988edc7d6e344c1f5 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:57:44.186662 21588 raft_consensus.cc:2949] T 306c4892bbff4dbfbe5a0645294aa627 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8bf6e26e98b14566899578a990b27d47 (127.19.226.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:44.192484 22177 raft_consensus.cc:1270] T db34a00c02b84ee988edc7d6e344c1f5 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Refusing update from remote peer 8bf6e26e98b14566899578a990b27d47: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250109 08:57:44.192979 21583 raft_consensus.cc:1270] T db34a00c02b84ee988edc7d6e344c1f5 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Refusing update from remote peer 8bf6e26e98b14566899578a990b27d47: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250109 08:57:44.197432 22371 ts_tablet_manager.cc:1918] T 97304b06c7fa40ef8345a72c796c47f8 P 787ddcc824c2490a808fc5a53452dc5e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20250109 08:57:44.197880 22371 log.cc:1198] T 97304b06c7fa40ef8345a72c796c47f8 P 787ddcc824c2490a808fc5a53452dc5e: Deleting WAL directory at /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736412987538469-20363-0/minicluster-data/ts-2-root/wals/97304b06c7fa40ef8345a72c796c47f8
I20250109 08:57:44.196779 21408 catalog_manager.cc:5526] T 306c4892bbff4dbfbe5a0645294aa627 P 88935ab835544e9889d05a6587d8cf21 reported cstate change: config changed from index 3 to 4, VOTER 8bf6e26e98b14566899578a990b27d47 (127.19.226.193) evicted. New cstate: current_term: 1 leader_uuid: "787ddcc824c2490a808fc5a53452dc5e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:44.198297 21396 catalog_manager.cc:5039] ChangeConfig:REMOVE_PEER RPC for tablet 306c4892bbff4dbfbe5a0645294aa627 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250109 08:57:44.201874 22345 consensus_queue.cc:1035] T db34a00c02b84ee988edc7d6e344c1f5 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250109 08:57:44.203565 22345 consensus_queue.cc:1035] T db34a00c02b84ee988edc7d6e344c1f5 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250109 08:57:44.210696 21396 catalog_manager.cc:4872] TS 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731): tablet 97304b06c7fa40ef8345a72c796c47f8 (table test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]) successfully deleted
I20250109 08:57:44.214584 21503 tablet_service.cc:1514] Processing DeleteTablet for tablet 306c4892bbff4dbfbe5a0645294aa627 with delete_type TABLET_DATA_TOMBSTONED (TS 8bf6e26e98b14566899578a990b27d47 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:39242
I20250109 08:57:44.215471 22367 tablet_replica.cc:331] T 306c4892bbff4dbfbe5a0645294aa627 P 8bf6e26e98b14566899578a990b27d47: stopping tablet replica
I20250109 08:57:44.216473 22367 raft_consensus.cc:2238] T 306c4892bbff4dbfbe5a0645294aa627 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:57:44.217149 22367 raft_consensus.cc:2267] T 306c4892bbff4dbfbe5a0645294aa627 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:57:44.219955 22367 ts_tablet_manager.cc:1905] T 306c4892bbff4dbfbe5a0645294aa627 P 8bf6e26e98b14566899578a990b27d47: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250109 08:57:44.223956 22343 raft_consensus.cc:2949] T db34a00c02b84ee988edc7d6e344c1f5 P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:44.226639 21583 raft_consensus.cc:2949] T db34a00c02b84ee988edc7d6e344c1f5 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:44.228725 22177 raft_consensus.cc:2949] T db34a00c02b84ee988edc7d6e344c1f5 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:44.241135 21396 catalog_manager.cc:5039] ChangeConfig:REMOVE_PEER RPC for tablet db34a00c02b84ee988edc7d6e344c1f5 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250109 08:57:44.247372 21410 catalog_manager.cc:5526] T db34a00c02b84ee988edc7d6e344c1f5 P 8bf6e26e98b14566899578a990b27d47 reported cstate change: config changed from index 3 to 4, VOTER 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195) evicted. New cstate: current_term: 1 leader_uuid: "8bf6e26e98b14566899578a990b27d47" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250109 08:57:44.257520 22367 ts_tablet_manager.cc:1918] T 306c4892bbff4dbfbe5a0645294aa627 P 8bf6e26e98b14566899578a990b27d47: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20250109 08:57:44.258077 22367 log.cc:1198] T 306c4892bbff4dbfbe5a0645294aa627 P 8bf6e26e98b14566899578a990b27d47: Deleting WAL directory at /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736412987538469-20363-0/minicluster-data/ts-0-root/wals/306c4892bbff4dbfbe5a0645294aa627
I20250109 08:57:44.261135 21396 catalog_manager.cc:4872] TS 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): tablet 306c4892bbff4dbfbe5a0645294aa627 (table test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]) successfully deleted
I20250109 08:57:44.263841 21653 tablet_service.cc:1514] Processing DeleteTablet for tablet db34a00c02b84ee988edc7d6e344c1f5 with delete_type TABLET_DATA_TOMBSTONED (TS 787ddcc824c2490a808fc5a53452dc5e not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:58436
I20250109 08:57:44.264786 22371 tablet_replica.cc:331] T db34a00c02b84ee988edc7d6e344c1f5 P 787ddcc824c2490a808fc5a53452dc5e: stopping tablet replica
I20250109 08:57:44.265728 22371 raft_consensus.cc:2238] T db34a00c02b84ee988edc7d6e344c1f5 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:57:44.266381 22371 raft_consensus.cc:2267] T db34a00c02b84ee988edc7d6e344c1f5 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:57:44.269503 22371 ts_tablet_manager.cc:1905] T db34a00c02b84ee988edc7d6e344c1f5 P 787ddcc824c2490a808fc5a53452dc5e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250109 08:57:44.286090 22371 ts_tablet_manager.cc:1918] T db34a00c02b84ee988edc7d6e344c1f5 P 787ddcc824c2490a808fc5a53452dc5e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20250109 08:57:44.286497 22371 log.cc:1198] T db34a00c02b84ee988edc7d6e344c1f5 P 787ddcc824c2490a808fc5a53452dc5e: Deleting WAL directory at /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736412987538469-20363-0/minicluster-data/ts-2-root/wals/db34a00c02b84ee988edc7d6e344c1f5
I20250109 08:57:44.288049 21396 catalog_manager.cc:4872] TS 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731): tablet db34a00c02b84ee988edc7d6e344c1f5 (table test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]) successfully deleted
I20250109 08:57:45.333814 21509 consensus_queue.cc:237] T 37280454805d47f98a415a8f0c16d869 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:57:45.340728 21583 raft_consensus.cc:1270] T 37280454805d47f98a415a8f0c16d869 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Refusing update from remote peer 8bf6e26e98b14566899578a990b27d47: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20250109 08:57:45.343137 21478 consensus_peers.cc:487] T 37280454805d47f98a415a8f0c16d869 P 8bf6e26e98b14566899578a990b27d47 -> Peer 88935ab835544e9889d05a6587d8cf21 (127.19.226.196:44363): Couldn't send request to peer 88935ab835544e9889d05a6587d8cf21. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 37280454805d47f98a415a8f0c16d869. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:57:45.343412 22348 consensus_queue.cc:1035] T 37280454805d47f98a415a8f0c16d869 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:57:45.344899 21658 raft_consensus.cc:1270] T 37280454805d47f98a415a8f0c16d869 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Refusing update from remote peer 8bf6e26e98b14566899578a990b27d47: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250109 08:57:45.348299 22373 consensus_queue.cc:1035] T 37280454805d47f98a415a8f0c16d869 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:57:45.357473 22348 raft_consensus.cc:2949] T 37280454805d47f98a415a8f0c16d869 P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:57:45.362558 21663 raft_consensus.cc:2949] T 37280454805d47f98a415a8f0c16d869 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:57:45.366261 21583 raft_consensus.cc:2949] T 37280454805d47f98a415a8f0c16d869 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:57:45.374082 21512 consensus_queue.cc:237] T e8ff570914d74df8b87296010e9b864f P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:57:45.383013 21408 catalog_manager.cc:5526] T 37280454805d47f98a415a8f0c16d869 P 8bf6e26e98b14566899578a990b27d47 reported cstate change: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New cstate: current_term: 1 leader_uuid: "8bf6e26e98b14566899578a990b27d47" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20250109 08:57:45.408026 21478 consensus_peers.cc:487] T e8ff570914d74df8b87296010e9b864f P 8bf6e26e98b14566899578a990b27d47 -> Peer 88935ab835544e9889d05a6587d8cf21 (127.19.226.196:44363): Couldn't send request to peer 88935ab835544e9889d05a6587d8cf21. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e8ff570914d74df8b87296010e9b864f. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:57:45.408605 21663 raft_consensus.cc:1270] T e8ff570914d74df8b87296010e9b864f P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Refusing update from remote peer 8bf6e26e98b14566899578a990b27d47: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250109 08:57:45.409970 22348 consensus_queue.cc:1035] T e8ff570914d74df8b87296010e9b864f P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:57:45.409418 21583 raft_consensus.cc:1270] T e8ff570914d74df8b87296010e9b864f P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Refusing update from remote peer 8bf6e26e98b14566899578a990b27d47: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250109 08:57:45.413018 22373 consensus_queue.cc:1035] T e8ff570914d74df8b87296010e9b864f P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20250109 08:57:45.424645 21552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.226.195:35731, user_credentials={real_user=slave}} blocked reactor thread for 53854.8us
I20250109 08:57:45.434468 22373 raft_consensus.cc:2949] T e8ff570914d74df8b87296010e9b864f P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:57:45.437291 21658 raft_consensus.cc:2949] T e8ff570914d74df8b87296010e9b864f P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:57:45.466419 21412 catalog_manager.cc:5526] T e8ff570914d74df8b87296010e9b864f P 787ddcc824c2490a808fc5a53452dc5e reported cstate change: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New cstate: current_term: 1 leader_uuid: "8bf6e26e98b14566899578a990b27d47" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:57:45.466581 21658 consensus_queue.cc:237] T d015a10f95eb4f19b6d7c53cb201fa80 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:57:45.479094 21583 raft_consensus.cc:2949] T e8ff570914d74df8b87296010e9b864f P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:57:45.497905 21583 raft_consensus.cc:1270] T d015a10f95eb4f19b6d7c53cb201fa80 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250109 08:57:45.501993 21511 raft_consensus.cc:1270] T d015a10f95eb4f19b6d7c53cb201fa80 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20250109 08:57:45.499115 21627 consensus_peers.cc:487] T d015a10f95eb4f19b6d7c53cb201fa80 P 787ddcc824c2490a808fc5a53452dc5e -> Peer 88935ab835544e9889d05a6587d8cf21 (127.19.226.196:44363): Couldn't send request to peer 88935ab835544e9889d05a6587d8cf21. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d015a10f95eb4f19b6d7c53cb201fa80. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:57:45.503434 22315 consensus_queue.cc:1035] T d015a10f95eb4f19b6d7c53cb201fa80 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:57:45.504251 22358 consensus_queue.cc:1035] T d015a10f95eb4f19b6d7c53cb201fa80 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:57:45.518536 22358 raft_consensus.cc:2949] T d015a10f95eb4f19b6d7c53cb201fa80 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:57:45.528563 21512 raft_consensus.cc:2949] T d015a10f95eb4f19b6d7c53cb201fa80 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:57:45.536912 21583 raft_consensus.cc:2949] T d015a10f95eb4f19b6d7c53cb201fa80 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:57:45.538599 21588 consensus_queue.cc:237] T 90978f185a374feba3bc19b54f9d97f5 P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:57:45.552441 21410 catalog_manager.cc:5526] T d015a10f95eb4f19b6d7c53cb201fa80 P 787ddcc824c2490a808fc5a53452dc5e reported cstate change: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New cstate: current_term: 1 leader_uuid: "787ddcc824c2490a808fc5a53452dc5e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20250109 08:57:45.563872 21658 raft_consensus.cc:1270] T 90978f185a374feba3bc19b54f9d97f5 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Refusing update from remote peer a2b1a692435241ee988e0bd1661cd4c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250109 08:57:45.565637 22376 consensus_queue.cc:1035] T 90978f185a374feba3bc19b54f9d97f5 P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20250109 08:57:45.572513 21553 consensus_peers.cc:487] T 90978f185a374feba3bc19b54f9d97f5 P a2b1a692435241ee988e0bd1661cd4c3 -> Peer 88935ab835544e9889d05a6587d8cf21 (127.19.226.196:44363): Couldn't send request to peer 88935ab835544e9889d05a6587d8cf21. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 90978f185a374feba3bc19b54f9d97f5. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:57:45.576115 21512 raft_consensus.cc:1270] T 90978f185a374feba3bc19b54f9d97f5 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Refusing update from remote peer a2b1a692435241ee988e0bd1661cd4c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250109 08:57:45.580969 22376 consensus_queue.cc:1035] T 90978f185a374feba3bc19b54f9d97f5 P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:57:45.590122 22383 raft_consensus.cc:2949] T 90978f185a374feba3bc19b54f9d97f5 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:57:45.591825 21658 raft_consensus.cc:2949] T 90978f185a374feba3bc19b54f9d97f5 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:57:45.591995 21512 raft_consensus.cc:2949] T 90978f185a374feba3bc19b54f9d97f5 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:57:45.613466 21412 catalog_manager.cc:5526] T 90978f185a374feba3bc19b54f9d97f5 P 8bf6e26e98b14566899578a990b27d47 reported cstate change: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New cstate: current_term: 1 leader_uuid: "a2b1a692435241ee988e0bd1661cd4c3" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:57:45.851258 22392 ts_tablet_manager.cc:927] T e8ff570914d74df8b87296010e9b864f P 88935ab835544e9889d05a6587d8cf21: Initiating tablet copy from peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725)
I20250109 08:57:45.853497 22392 tablet_copy_client.cc:323] T e8ff570914d74df8b87296010e9b864f P 88935ab835544e9889d05a6587d8cf21: tablet copy: Beginning tablet copy session from remote peer at address 127.19.226.193:42725
I20250109 08:57:45.855075 21524 tablet_copy_service.cc:140] P 8bf6e26e98b14566899578a990b27d47: Received BeginTabletCopySession request for tablet e8ff570914d74df8b87296010e9b864f from peer 88935ab835544e9889d05a6587d8cf21 ({username='slave'} at 127.0.0.1:41812)
I20250109 08:57:45.855605 21524 tablet_copy_service.cc:161] P 8bf6e26e98b14566899578a990b27d47: Beginning new tablet copy session on tablet e8ff570914d74df8b87296010e9b864f from peer 88935ab835544e9889d05a6587d8cf21 at {username='slave'} at 127.0.0.1:41812: session id = 88935ab835544e9889d05a6587d8cf21-e8ff570914d74df8b87296010e9b864f
I20250109 08:57:45.863291 21524 tablet_copy_source_session.cc:215] T e8ff570914d74df8b87296010e9b864f P 8bf6e26e98b14566899578a990b27d47: Tablet Copy: opened 0 blocks and 1 log segments
I20250109 08:57:45.866292 22394 ts_tablet_manager.cc:927] T 37280454805d47f98a415a8f0c16d869 P 88935ab835544e9889d05a6587d8cf21: Initiating tablet copy from peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725)
I20250109 08:57:45.871989 22392 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8ff570914d74df8b87296010e9b864f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:45.884562 22394 tablet_copy_client.cc:323] T 37280454805d47f98a415a8f0c16d869 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Beginning tablet copy session from remote peer at address 127.19.226.193:42725
I20250109 08:57:45.887456 21524 tablet_copy_service.cc:140] P 8bf6e26e98b14566899578a990b27d47: Received BeginTabletCopySession request for tablet 37280454805d47f98a415a8f0c16d869 from peer 88935ab835544e9889d05a6587d8cf21 ({username='slave'} at 127.0.0.1:41812)
I20250109 08:57:45.888008 21524 tablet_copy_service.cc:161] P 8bf6e26e98b14566899578a990b27d47: Beginning new tablet copy session on tablet 37280454805d47f98a415a8f0c16d869 from peer 88935ab835544e9889d05a6587d8cf21 at {username='slave'} at 127.0.0.1:41812: session id = 88935ab835544e9889d05a6587d8cf21-37280454805d47f98a415a8f0c16d869
I20250109 08:57:45.890251 22392 tablet_copy_client.cc:806] T e8ff570914d74df8b87296010e9b864f P 88935ab835544e9889d05a6587d8cf21: tablet copy: Starting download of 0 data blocks...
I20250109 08:57:45.890769 22392 tablet_copy_client.cc:670] T e8ff570914d74df8b87296010e9b864f P 88935ab835544e9889d05a6587d8cf21: tablet copy: Starting download of 1 WAL segments...
I20250109 08:57:45.895241 21524 tablet_copy_source_session.cc:215] T 37280454805d47f98a415a8f0c16d869 P 8bf6e26e98b14566899578a990b27d47: Tablet Copy: opened 0 blocks and 1 log segments
I20250109 08:57:45.897159 22392 tablet_copy_client.cc:538] T e8ff570914d74df8b87296010e9b864f P 88935ab835544e9889d05a6587d8cf21: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250109 08:57:45.898319 22394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37280454805d47f98a415a8f0c16d869. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:45.913370 22394 tablet_copy_client.cc:806] T 37280454805d47f98a415a8f0c16d869 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Starting download of 0 data blocks...
I20250109 08:57:45.914058 22394 tablet_copy_client.cc:670] T 37280454805d47f98a415a8f0c16d869 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Starting download of 1 WAL segments...
I20250109 08:57:45.920250 22394 tablet_copy_client.cc:538] T 37280454805d47f98a415a8f0c16d869 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250109 08:57:45.926226 22392 tablet_bootstrap.cc:492] T e8ff570914d74df8b87296010e9b864f P 88935ab835544e9889d05a6587d8cf21: Bootstrap starting.
I20250109 08:57:45.932499 22394 tablet_bootstrap.cc:492] T 37280454805d47f98a415a8f0c16d869 P 88935ab835544e9889d05a6587d8cf21: Bootstrap starting.
W20250109 08:57:46.070529 21627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.226.194:45527, user_credentials={real_user=slave}} blocked reactor thread for 66412.7us
I20250109 08:57:46.116158 22400 ts_tablet_manager.cc:927] T d015a10f95eb4f19b6d7c53cb201fa80 P 88935ab835544e9889d05a6587d8cf21: Initiating tablet copy from peer 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
W20250109 08:57:46.132622 21478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.226.194:45527, user_credentials={real_user=slave}} blocked reactor thread for 65780.5us
I20250109 08:57:46.137365 22401 ts_tablet_manager.cc:927] T 90978f185a374feba3bc19b54f9d97f5 P 88935ab835544e9889d05a6587d8cf21: Initiating tablet copy from peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
I20250109 08:57:46.148485 22401 tablet_copy_client.cc:323] T 90978f185a374feba3bc19b54f9d97f5 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Beginning tablet copy session from remote peer at address 127.19.226.194:45527
I20250109 08:57:46.151146 21599 tablet_copy_service.cc:140] P a2b1a692435241ee988e0bd1661cd4c3: Received BeginTabletCopySession request for tablet 90978f185a374feba3bc19b54f9d97f5 from peer 88935ab835544e9889d05a6587d8cf21 ({username='slave'} at 127.0.0.1:33652)
I20250109 08:57:46.151685 21599 tablet_copy_service.cc:161] P a2b1a692435241ee988e0bd1661cd4c3: Beginning new tablet copy session on tablet 90978f185a374feba3bc19b54f9d97f5 from peer 88935ab835544e9889d05a6587d8cf21 at {username='slave'} at 127.0.0.1:33652: session id = 88935ab835544e9889d05a6587d8cf21-90978f185a374feba3bc19b54f9d97f5
I20250109 08:57:46.160809 21599 tablet_copy_source_session.cc:215] T 90978f185a374feba3bc19b54f9d97f5 P a2b1a692435241ee988e0bd1661cd4c3: Tablet Copy: opened 0 blocks and 1 log segments
I20250109 08:57:46.164012 22401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90978f185a374feba3bc19b54f9d97f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:46.174217 22400 tablet_copy_client.cc:323] T d015a10f95eb4f19b6d7c53cb201fa80 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Beginning tablet copy session from remote peer at address 127.19.226.195:35731
I20250109 08:57:46.175995 21673 tablet_copy_service.cc:140] P 787ddcc824c2490a808fc5a53452dc5e: Received BeginTabletCopySession request for tablet d015a10f95eb4f19b6d7c53cb201fa80 from peer 88935ab835544e9889d05a6587d8cf21 ({username='slave'} at 127.0.0.1:52376)
I20250109 08:57:46.176542 21673 tablet_copy_service.cc:161] P 787ddcc824c2490a808fc5a53452dc5e: Beginning new tablet copy session on tablet d015a10f95eb4f19b6d7c53cb201fa80 from peer 88935ab835544e9889d05a6587d8cf21 at {username='slave'} at 127.0.0.1:52376: session id = 88935ab835544e9889d05a6587d8cf21-d015a10f95eb4f19b6d7c53cb201fa80
I20250109 08:57:46.187865 21673 tablet_copy_source_session.cc:215] T d015a10f95eb4f19b6d7c53cb201fa80 P 787ddcc824c2490a808fc5a53452dc5e: Tablet Copy: opened 0 blocks and 1 log segments
I20250109 08:57:46.192508 22401 tablet_copy_client.cc:806] T 90978f185a374feba3bc19b54f9d97f5 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Starting download of 0 data blocks...
I20250109 08:57:46.193192 22401 tablet_copy_client.cc:670] T 90978f185a374feba3bc19b54f9d97f5 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Starting download of 1 WAL segments...
I20250109 08:57:46.197459 22401 tablet_copy_client.cc:538] T 90978f185a374feba3bc19b54f9d97f5 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250109 08:57:46.208992 22400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d015a10f95eb4f19b6d7c53cb201fa80. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:46.214550 22401 tablet_bootstrap.cc:492] T 90978f185a374feba3bc19b54f9d97f5 P 88935ab835544e9889d05a6587d8cf21: Bootstrap starting.
I20250109 08:57:46.237077 22392 tablet_bootstrap.cc:492] T e8ff570914d74df8b87296010e9b864f P 88935ab835544e9889d05a6587d8cf21: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250109 08:57:46.238232 22392 tablet_bootstrap.cc:492] T e8ff570914d74df8b87296010e9b864f P 88935ab835544e9889d05a6587d8cf21: Bootstrap complete.
I20250109 08:57:46.238967 22392 ts_tablet_manager.cc:1397] T e8ff570914d74df8b87296010e9b864f P 88935ab835544e9889d05a6587d8cf21: Time spent bootstrapping tablet: real 0.313s user 0.161s sys 0.137s
I20250109 08:57:46.241931 22392 raft_consensus.cc:357] T e8ff570914d74df8b87296010e9b864f P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:57:46.242897 22392 raft_consensus.cc:738] T e8ff570914d74df8b87296010e9b864f P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 88935ab835544e9889d05a6587d8cf21, State: Initialized, Role: LEARNER
I20250109 08:57:46.243572 22392 consensus_queue.cc:260] T e8ff570914d74df8b87296010e9b864f P 88935ab835544e9889d05a6587d8cf21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:57:46.253563 22392 ts_tablet_manager.cc:1428] T e8ff570914d74df8b87296010e9b864f P 88935ab835544e9889d05a6587d8cf21: Time spent starting tablet: real 0.014s user 0.008s sys 0.005s
I20250109 08:57:46.260445 22400 tablet_copy_client.cc:806] T d015a10f95eb4f19b6d7c53cb201fa80 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Starting download of 0 data blocks...
I20250109 08:57:46.261097 22400 tablet_copy_client.cc:670] T d015a10f95eb4f19b6d7c53cb201fa80 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Starting download of 1 WAL segments...
I20250109 08:57:46.266861 22400 tablet_copy_client.cc:538] T d015a10f95eb4f19b6d7c53cb201fa80 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250109 08:57:46.272711 21524 tablet_copy_service.cc:342] P 8bf6e26e98b14566899578a990b27d47: Request end of tablet copy session 88935ab835544e9889d05a6587d8cf21-e8ff570914d74df8b87296010e9b864f received from {username='slave'} at 127.0.0.1:41812
I20250109 08:57:46.273185 21524 tablet_copy_service.cc:434] P 8bf6e26e98b14566899578a990b27d47: ending tablet copy session 88935ab835544e9889d05a6587d8cf21-e8ff570914d74df8b87296010e9b864f on tablet e8ff570914d74df8b87296010e9b864f with peer 88935ab835544e9889d05a6587d8cf21
I20250109 08:57:46.277622 22400 tablet_bootstrap.cc:492] T d015a10f95eb4f19b6d7c53cb201fa80 P 88935ab835544e9889d05a6587d8cf21: Bootstrap starting.
I20250109 08:57:46.285931 22394 tablet_bootstrap.cc:492] T 37280454805d47f98a415a8f0c16d869 P 88935ab835544e9889d05a6587d8cf21: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250109 08:57:46.287858 22394 tablet_bootstrap.cc:492] T 37280454805d47f98a415a8f0c16d869 P 88935ab835544e9889d05a6587d8cf21: Bootstrap complete.
I20250109 08:57:46.279527 22401 tablet_bootstrap.cc:492] T 90978f185a374feba3bc19b54f9d97f5 P 88935ab835544e9889d05a6587d8cf21: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250109 08:57:46.288659 22394 ts_tablet_manager.cc:1397] T 37280454805d47f98a415a8f0c16d869 P 88935ab835544e9889d05a6587d8cf21: Time spent bootstrapping tablet: real 0.356s user 0.070s sys 0.095s
I20250109 08:57:46.289139 22401 tablet_bootstrap.cc:492] T 90978f185a374feba3bc19b54f9d97f5 P 88935ab835544e9889d05a6587d8cf21: Bootstrap complete.
I20250109 08:57:46.289904 22401 ts_tablet_manager.cc:1397] T 90978f185a374feba3bc19b54f9d97f5 P 88935ab835544e9889d05a6587d8cf21: Time spent bootstrapping tablet: real 0.076s user 0.017s sys 0.005s
I20250109 08:57:46.291491 22394 raft_consensus.cc:357] T 37280454805d47f98a415a8f0c16d869 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:57:46.292352 22394 raft_consensus.cc:738] T 37280454805d47f98a415a8f0c16d869 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 88935ab835544e9889d05a6587d8cf21, State: Initialized, Role: LEARNER
I20250109 08:57:46.292724 22401 raft_consensus.cc:357] T 90978f185a374feba3bc19b54f9d97f5 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:57:46.292989 22394 consensus_queue.cc:260] T 37280454805d47f98a415a8f0c16d869 P 88935ab835544e9889d05a6587d8cf21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:57:46.293685 22401 raft_consensus.cc:738] T 90978f185a374feba3bc19b54f9d97f5 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 88935ab835544e9889d05a6587d8cf21, State: Initialized, Role: LEARNER
I20250109 08:57:46.294463 22401 consensus_queue.cc:260] T 90978f185a374feba3bc19b54f9d97f5 P 88935ab835544e9889d05a6587d8cf21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:57:46.297567 22401 ts_tablet_manager.cc:1428] T 90978f185a374feba3bc19b54f9d97f5 P 88935ab835544e9889d05a6587d8cf21: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20250109 08:57:46.299870 22394 ts_tablet_manager.cc:1428] T 37280454805d47f98a415a8f0c16d869 P 88935ab835544e9889d05a6587d8cf21: Time spent starting tablet: real 0.011s user 0.004s sys 0.001s
I20250109 08:57:46.300988 21599 tablet_copy_service.cc:342] P a2b1a692435241ee988e0bd1661cd4c3: Request end of tablet copy session 88935ab835544e9889d05a6587d8cf21-90978f185a374feba3bc19b54f9d97f5 received from {username='slave'} at 127.0.0.1:33652
I20250109 08:57:46.302533 21599 tablet_copy_service.cc:434] P a2b1a692435241ee988e0bd1661cd4c3: ending tablet copy session 88935ab835544e9889d05a6587d8cf21-90978f185a374feba3bc19b54f9d97f5 on tablet 90978f185a374feba3bc19b54f9d97f5 with peer 88935ab835544e9889d05a6587d8cf21
I20250109 08:57:46.343505 22400 tablet_bootstrap.cc:492] T d015a10f95eb4f19b6d7c53cb201fa80 P 88935ab835544e9889d05a6587d8cf21: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250109 08:57:46.349542 22400 tablet_bootstrap.cc:492] T d015a10f95eb4f19b6d7c53cb201fa80 P 88935ab835544e9889d05a6587d8cf21: Bootstrap complete.
I20250109 08:57:46.350334 22400 ts_tablet_manager.cc:1397] T d015a10f95eb4f19b6d7c53cb201fa80 P 88935ab835544e9889d05a6587d8cf21: Time spent bootstrapping tablet: real 0.073s user 0.026s sys 0.004s
W20250109 08:57:46.353013 21478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.226.194:45527, user_credentials={real_user=slave}} blocked reactor thread for 79450.2us
I20250109 08:57:46.353170 22400 raft_consensus.cc:357] T d015a10f95eb4f19b6d7c53cb201fa80 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:57:46.354295 22400 raft_consensus.cc:738] T d015a10f95eb4f19b6d7c53cb201fa80 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 88935ab835544e9889d05a6587d8cf21, State: Initialized, Role: LEARNER
I20250109 08:57:46.355024 22400 consensus_queue.cc:260] T d015a10f95eb4f19b6d7c53cb201fa80 P 88935ab835544e9889d05a6587d8cf21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:57:46.358433 22400 ts_tablet_manager.cc:1428] T d015a10f95eb4f19b6d7c53cb201fa80 P 88935ab835544e9889d05a6587d8cf21: Time spent starting tablet: real 0.008s user 0.005s sys 0.000s
I20250109 08:57:46.360193 21673 tablet_copy_service.cc:342] P 787ddcc824c2490a808fc5a53452dc5e: Request end of tablet copy session 88935ab835544e9889d05a6587d8cf21-d015a10f95eb4f19b6d7c53cb201fa80 received from {username='slave'} at 127.0.0.1:52376
I20250109 08:57:46.360627 21673 tablet_copy_service.cc:434] P 787ddcc824c2490a808fc5a53452dc5e: ending tablet copy session 88935ab835544e9889d05a6587d8cf21-d015a10f95eb4f19b6d7c53cb201fa80 on tablet d015a10f95eb4f19b6d7c53cb201fa80 with peer 88935ab835544e9889d05a6587d8cf21
I20250109 08:57:46.362936 21524 tablet_copy_service.cc:342] P 8bf6e26e98b14566899578a990b27d47: Request end of tablet copy session 88935ab835544e9889d05a6587d8cf21-37280454805d47f98a415a8f0c16d869 received from {username='slave'} at 127.0.0.1:41812
I20250109 08:57:46.363392 21524 tablet_copy_service.cc:434] P 8bf6e26e98b14566899578a990b27d47: ending tablet copy session 88935ab835544e9889d05a6587d8cf21-37280454805d47f98a415a8f0c16d869 on tablet 37280454805d47f98a415a8f0c16d869 with peer 88935ab835544e9889d05a6587d8cf21
I20250109 08:57:46.462986 22177 raft_consensus.cc:1212] T 37280454805d47f98a415a8f0c16d869 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20250109 08:57:46.535589 22177 raft_consensus.cc:1212] T e8ff570914d74df8b87296010e9b864f P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20250109 08:57:46.633692 22177 raft_consensus.cc:1212] T 90978f185a374feba3bc19b54f9d97f5 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20250109 08:57:46.662422 22177 raft_consensus.cc:1212] T d015a10f95eb4f19b6d7c53cb201fa80 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20250109 08:57:46.858489 22410 raft_consensus.cc:1059] T 37280454805d47f98a415a8f0c16d869 P 8bf6e26e98b14566899578a990b27d47: attempting to promote NON_VOTER 88935ab835544e9889d05a6587d8cf21 to VOTER
I20250109 08:57:46.860734 22410 consensus_queue.cc:237] T 37280454805d47f98a415a8f0c16d869 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:57:46.866605 22177 raft_consensus.cc:1270] T 37280454805d47f98a415a8f0c16d869 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Refusing update from remote peer 8bf6e26e98b14566899578a990b27d47: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:57:46.867610 21663 raft_consensus.cc:1270] T 37280454805d47f98a415a8f0c16d869 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Refusing update from remote peer 8bf6e26e98b14566899578a990b27d47: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:57:46.868770 22408 consensus_queue.cc:1035] T 37280454805d47f98a415a8f0c16d869 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:57:46.869690 22385 consensus_queue.cc:1035] T 37280454805d47f98a415a8f0c16d869 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:57:46.878182 21588 raft_consensus.cc:1270] T 37280454805d47f98a415a8f0c16d869 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Refusing update from remote peer 8bf6e26e98b14566899578a990b27d47: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:57:46.880857 22410 consensus_queue.cc:1035] T 37280454805d47f98a415a8f0c16d869 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:57:46.900635 22385 raft_consensus.cc:2949] T 37280454805d47f98a415a8f0c16d869 P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:46.915704 21656 raft_consensus.cc:2949] T 37280454805d47f98a415a8f0c16d869 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:46.917896 21412 catalog_manager.cc:5526] T 37280454805d47f98a415a8f0c16d869 P 8bf6e26e98b14566899578a990b27d47 reported cstate change: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8bf6e26e98b14566899578a990b27d47" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250109 08:57:46.918475 21581 raft_consensus.cc:2949] T 37280454805d47f98a415a8f0c16d869 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:46.920908 22177 raft_consensus.cc:2949] T 37280454805d47f98a415a8f0c16d869 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:46.983909 21512 consensus_queue.cc:237] T 37280454805d47f98a415a8f0c16d869 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:57:46.992559 22178 raft_consensus.cc:1270] T 37280454805d47f98a415a8f0c16d869 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Refusing update from remote peer 8bf6e26e98b14566899578a990b27d47: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250109 08:57:46.994788 22410 raft_consensus.cc:1059] T e8ff570914d74df8b87296010e9b864f P 8bf6e26e98b14566899578a990b27d47: attempting to promote NON_VOTER 88935ab835544e9889d05a6587d8cf21 to VOTER
I20250109 08:57:46.995280 22385 consensus_queue.cc:1035] T 37280454805d47f98a415a8f0c16d869 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250109 08:57:46.999563 21656 raft_consensus.cc:1270] T 37280454805d47f98a415a8f0c16d869 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Refusing update from remote peer 8bf6e26e98b14566899578a990b27d47: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250109 08:57:47.002552 22402 raft_consensus.cc:2949] T 37280454805d47f98a415a8f0c16d869 P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:47.003305 22408 consensus_queue.cc:1035] T 37280454805d47f98a415a8f0c16d869 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250109 08:57:47.006783 22178 raft_consensus.cc:2949] T 37280454805d47f98a415a8f0c16d869 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:47.009806 22410 consensus_queue.cc:237] T e8ff570914d74df8b87296010e9b864f P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:57:47.012557 21396 catalog_manager.cc:5039] ChangeConfig:REMOVE_PEER RPC for tablet 37280454805d47f98a415a8f0c16d869 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250109 08:57:47.021960 21656 raft_consensus.cc:2949] T 37280454805d47f98a415a8f0c16d869 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:47.023169 21411 catalog_manager.cc:5526] T 37280454805d47f98a415a8f0c16d869 P 88935ab835544e9889d05a6587d8cf21 reported cstate change: config changed from index 3 to 4, VOTER a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194) evicted. New cstate: current_term: 1 leader_uuid: "8bf6e26e98b14566899578a990b27d47" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:47.029343 22178 raft_consensus.cc:1270] T e8ff570914d74df8b87296010e9b864f P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Refusing update from remote peer 8bf6e26e98b14566899578a990b27d47: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:57:47.035933 22408 consensus_queue.cc:1035] T e8ff570914d74df8b87296010e9b864f P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:57:47.043004 21658 raft_consensus.cc:1270] T e8ff570914d74df8b87296010e9b864f P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Refusing update from remote peer 8bf6e26e98b14566899578a990b27d47: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:57:47.045667 21570 tablet_service.cc:1514] Processing DeleteTablet for tablet 37280454805d47f98a415a8f0c16d869 with delete_type TABLET_DATA_TOMBSTONED (TS a2b1a692435241ee988e0bd1661cd4c3 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:34732
I20250109 08:57:47.047848 22402 consensus_queue.cc:1035] T e8ff570914d74df8b87296010e9b864f P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:57:47.050763 21583 raft_consensus.cc:1270] T e8ff570914d74df8b87296010e9b864f P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Refusing update from remote peer 8bf6e26e98b14566899578a990b27d47: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:57:47.061939 22410 consensus_queue.cc:1035] T e8ff570914d74df8b87296010e9b864f P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20250109 08:57:47.102242 22408 raft_consensus.cc:2949] T e8ff570914d74df8b87296010e9b864f P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:47.112110 21588 raft_consensus.cc:2949] T e8ff570914d74df8b87296010e9b864f P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:47.122619 22178 raft_consensus.cc:2949] T e8ff570914d74df8b87296010e9b864f P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:47.138258 21410 catalog_manager.cc:5526] T e8ff570914d74df8b87296010e9b864f P 8bf6e26e98b14566899578a990b27d47 reported cstate change: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8bf6e26e98b14566899578a990b27d47" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20250109 08:57:47.176163 21627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.226.194:45527, user_credentials={real_user=slave}} blocked reactor thread for 44412.2us
I20250109 08:57:47.186618 22424 raft_consensus.cc:1059] T 90978f185a374feba3bc19b54f9d97f5 P a2b1a692435241ee988e0bd1661cd4c3: attempting to promote NON_VOTER 88935ab835544e9889d05a6587d8cf21 to VOTER
I20250109 08:57:47.190253 22424 consensus_queue.cc:237] T 90978f185a374feba3bc19b54f9d97f5 P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:57:47.212352 21658 raft_consensus.cc:1270] T 90978f185a374feba3bc19b54f9d97f5 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Refusing update from remote peer a2b1a692435241ee988e0bd1661cd4c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:57:47.217901 22383 consensus_queue.cc:1035] T 90978f185a374feba3bc19b54f9d97f5 P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:57:47.233196 22420 tablet_replica.cc:331] T 37280454805d47f98a415a8f0c16d869 P a2b1a692435241ee988e0bd1661cd4c3: stopping tablet replica
I20250109 08:57:47.236006 22420 raft_consensus.cc:2238] T 37280454805d47f98a415a8f0c16d869 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:57:47.238304 22420 raft_consensus.cc:2267] T 37280454805d47f98a415a8f0c16d869 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:57:47.240798 21509 raft_consensus.cc:1270] T 90978f185a374feba3bc19b54f9d97f5 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Refusing update from remote peer a2b1a692435241ee988e0bd1661cd4c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:57:47.242455 22178 raft_consensus.cc:1270] T 90978f185a374feba3bc19b54f9d97f5 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Refusing update from remote peer a2b1a692435241ee988e0bd1661cd4c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:57:47.244596 22376 consensus_queue.cc:1035] T 90978f185a374feba3bc19b54f9d97f5 P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:57:47.245999 22429 consensus_queue.cc:1035] T 90978f185a374feba3bc19b54f9d97f5 P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:57:47.256574 21512 consensus_queue.cc:237] T e8ff570914d74df8b87296010e9b864f P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:57:47.257841 21658 raft_consensus.cc:2949] T e8ff570914d74df8b87296010e9b864f P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:47.253679 22420 ts_tablet_manager.cc:1905] T 37280454805d47f98a415a8f0c16d869 P a2b1a692435241ee988e0bd1661cd4c3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250109 08:57:47.267799 22177 raft_consensus.cc:1270] T e8ff570914d74df8b87296010e9b864f P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Refusing update from remote peer 8bf6e26e98b14566899578a990b27d47: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250109 08:57:47.271149 22427 consensus_queue.cc:1035] T e8ff570914d74df8b87296010e9b864f P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20250109 08:57:47.282377 21656 raft_consensus.cc:1270] T e8ff570914d74df8b87296010e9b864f P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Refusing update from remote peer 8bf6e26e98b14566899578a990b27d47: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250109 08:57:47.286065 22373 raft_consensus.cc:2949] T e8ff570914d74df8b87296010e9b864f P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:47.290777 22177 raft_consensus.cc:2949] T e8ff570914d74df8b87296010e9b864f P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:47.294531 22358 raft_consensus.cc:1059] T d015a10f95eb4f19b6d7c53cb201fa80 P 787ddcc824c2490a808fc5a53452dc5e: attempting to promote NON_VOTER 88935ab835544e9889d05a6587d8cf21 to VOTER
I20250109 08:57:47.296784 22358 consensus_queue.cc:237] T d015a10f95eb4f19b6d7c53cb201fa80 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:57:47.303133 22410 consensus_queue.cc:1035] T e8ff570914d74df8b87296010e9b864f P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250109 08:57:47.318956 21656 raft_consensus.cc:2949] T e8ff570914d74df8b87296010e9b864f P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:47.320016 21511 raft_consensus.cc:1270] T d015a10f95eb4f19b6d7c53cb201fa80 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:57:47.324306 22358 consensus_queue.cc:1035] T d015a10f95eb4f19b6d7c53cb201fa80 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:57:47.326843 21396 catalog_manager.cc:5039] ChangeConfig:REMOVE_PEER RPC for tablet e8ff570914d74df8b87296010e9b864f with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250109 08:57:47.345988 21408 catalog_manager.cc:5526] T e8ff570914d74df8b87296010e9b864f P 8bf6e26e98b14566899578a990b27d47 reported cstate change: config changed from index 3 to 4, VOTER a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194) evicted. New cstate: current_term: 1 leader_uuid: "8bf6e26e98b14566899578a990b27d47" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250109 08:57:47.351090 21581 raft_consensus.cc:1270] T d015a10f95eb4f19b6d7c53cb201fa80 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:57:47.354346 22178 raft_consensus.cc:1270] T d015a10f95eb4f19b6d7c53cb201fa80 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:57:47.353731 22428 raft_consensus.cc:2949] T 90978f185a374feba3bc19b54f9d97f5 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:47.359181 22398 consensus_queue.cc:1035] T d015a10f95eb4f19b6d7c53cb201fa80 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:57:47.362684 21663 raft_consensus.cc:2949] T 90978f185a374feba3bc19b54f9d97f5 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:47.363000 21509 raft_consensus.cc:2949] T 90978f185a374feba3bc19b54f9d97f5 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:47.382128 22358 consensus_queue.cc:1035] T d015a10f95eb4f19b6d7c53cb201fa80 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:57:47.408455 22176 raft_consensus.cc:2949] T 90978f185a374feba3bc19b54f9d97f5 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:47.425704 21411 catalog_manager.cc:5526] T 90978f185a374feba3bc19b54f9d97f5 P a2b1a692435241ee988e0bd1661cd4c3 reported cstate change: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a2b1a692435241ee988e0bd1661cd4c3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250109 08:57:47.435009 21570 tablet_service.cc:1514] Processing DeleteTablet for tablet e8ff570914d74df8b87296010e9b864f with delete_type TABLET_DATA_TOMBSTONED (TS a2b1a692435241ee988e0bd1661cd4c3 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:34732
I20250109 08:57:47.454914 21584 consensus_queue.cc:237] T 90978f185a374feba3bc19b54f9d97f5 P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:57:47.499538 21509 raft_consensus.cc:1270] T 90978f185a374feba3bc19b54f9d97f5 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Refusing update from remote peer a2b1a692435241ee988e0bd1661cd4c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250109 08:57:47.504073 22176 raft_consensus.cc:1270] T 90978f185a374feba3bc19b54f9d97f5 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Refusing update from remote peer a2b1a692435241ee988e0bd1661cd4c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250109 08:57:47.514869 22429 consensus_queue.cc:1035] T 90978f185a374feba3bc19b54f9d97f5 P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250109 08:57:47.515765 22376 consensus_queue.cc:1035] T 90978f185a374feba3bc19b54f9d97f5 P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250109 08:57:47.518143 22420 ts_tablet_manager.cc:1918] T 37280454805d47f98a415a8f0c16d869 P a2b1a692435241ee988e0bd1661cd4c3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20250109 08:57:47.519418 22358 raft_consensus.cc:2949] T d015a10f95eb4f19b6d7c53cb201fa80 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:47.525733 22428 raft_consensus.cc:2949] T 90978f185a374feba3bc19b54f9d97f5 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:47.529028 22178 raft_consensus.cc:2949] T 90978f185a374feba3bc19b54f9d97f5 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:47.538166 21511 raft_consensus.cc:2949] T 90978f185a374feba3bc19b54f9d97f5 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:47.530995 22177 raft_consensus.cc:2949] T d015a10f95eb4f19b6d7c53cb201fa80 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:47.569285 21509 raft_consensus.cc:2949] T d015a10f95eb4f19b6d7c53cb201fa80 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:47.543216 22420 log.cc:1198] T 37280454805d47f98a415a8f0c16d869 P a2b1a692435241ee988e0bd1661cd4c3: Deleting WAL directory at /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736412987538469-20363-0/minicluster-data/ts-1-root/wals/37280454805d47f98a415a8f0c16d869
I20250109 08:57:47.574267 22420 tablet_replica.cc:331] T e8ff570914d74df8b87296010e9b864f P a2b1a692435241ee988e0bd1661cd4c3: stopping tablet replica
I20250109 08:57:47.575152 22420 raft_consensus.cc:2238] T e8ff570914d74df8b87296010e9b864f P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:57:47.575951 21397 catalog_manager.cc:5039] ChangeConfig:REMOVE_PEER RPC for tablet 90978f185a374feba3bc19b54f9d97f5 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250109 08:57:47.579450 22420 raft_consensus.cc:2267] T e8ff570914d74df8b87296010e9b864f P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:57:47.587426 22420 ts_tablet_manager.cc:1905] T e8ff570914d74df8b87296010e9b864f P a2b1a692435241ee988e0bd1661cd4c3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250109 08:57:47.606395 21410 catalog_manager.cc:5526] T 90978f185a374feba3bc19b54f9d97f5 P 88935ab835544e9889d05a6587d8cf21 reported cstate change: config changed from index 3 to 4, VOTER 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195) evicted. New cstate: current_term: 1 leader_uuid: "a2b1a692435241ee988e0bd1661cd4c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:47.608206 21412 catalog_manager.cc:5526] T d015a10f95eb4f19b6d7c53cb201fa80 P 787ddcc824c2490a808fc5a53452dc5e reported cstate change: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "787ddcc824c2490a808fc5a53452dc5e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250109 08:57:47.577189 21397 catalog_manager.cc:4872] TS a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): tablet 37280454805d47f98a415a8f0c16d869 (table test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]) successfully deleted
I20250109 08:57:47.618139 22420 ts_tablet_manager.cc:1918] T e8ff570914d74df8b87296010e9b864f P a2b1a692435241ee988e0bd1661cd4c3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20250109 08:57:47.618759 22420 log.cc:1198] T e8ff570914d74df8b87296010e9b864f P a2b1a692435241ee988e0bd1661cd4c3: Deleting WAL directory at /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736412987538469-20363-0/minicluster-data/ts-1-root/wals/e8ff570914d74df8b87296010e9b864f
I20250109 08:57:47.622406 21397 catalog_manager.cc:4872] TS a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): tablet e8ff570914d74df8b87296010e9b864f (table test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]) successfully deleted
I20250109 08:57:47.625770 21653 tablet_service.cc:1514] Processing DeleteTablet for tablet 90978f185a374feba3bc19b54f9d97f5 with delete_type TABLET_DATA_TOMBSTONED (TS 787ddcc824c2490a808fc5a53452dc5e not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:58436
I20250109 08:57:47.635599 22437 tablet_replica.cc:331] T 90978f185a374feba3bc19b54f9d97f5 P 787ddcc824c2490a808fc5a53452dc5e: stopping tablet replica
I20250109 08:57:47.639008 22437 raft_consensus.cc:2238] T 90978f185a374feba3bc19b54f9d97f5 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:57:47.640334 22437 raft_consensus.cc:2267] T 90978f185a374feba3bc19b54f9d97f5 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:57:47.606176 21583 raft_consensus.cc:2949] T d015a10f95eb4f19b6d7c53cb201fa80 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:47.644018 22437 ts_tablet_manager.cc:1905] T 90978f185a374feba3bc19b54f9d97f5 P 787ddcc824c2490a808fc5a53452dc5e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250109 08:57:47.682750 22437 ts_tablet_manager.cc:1918] T 90978f185a374feba3bc19b54f9d97f5 P 787ddcc824c2490a808fc5a53452dc5e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20250109 08:57:47.683287 22437 log.cc:1198] T 90978f185a374feba3bc19b54f9d97f5 P 787ddcc824c2490a808fc5a53452dc5e: Deleting WAL directory at /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736412987538469-20363-0/minicluster-data/ts-2-root/wals/90978f185a374feba3bc19b54f9d97f5
I20250109 08:57:47.682857 21663 consensus_queue.cc:237] T d015a10f95eb4f19b6d7c53cb201fa80 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:57:47.685017 21396 catalog_manager.cc:4872] TS 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731): tablet 90978f185a374feba3bc19b54f9d97f5 (table test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]) successfully deleted
I20250109 08:57:47.689175 22177 raft_consensus.cc:1270] T d015a10f95eb4f19b6d7c53cb201fa80 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250109 08:57:47.690959 22315 consensus_queue.cc:1035] T d015a10f95eb4f19b6d7c53cb201fa80 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250109 08:57:47.692595 21583 raft_consensus.cc:1270] T d015a10f95eb4f19b6d7c53cb201fa80 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250109 08:57:47.698858 22398 consensus_queue.cc:1035] T d015a10f95eb4f19b6d7c53cb201fa80 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250109 08:57:47.702874 22358 raft_consensus.cc:2949] T d015a10f95eb4f19b6d7c53cb201fa80 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8bf6e26e98b14566899578a990b27d47 (127.19.226.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:47.711205 21584 raft_consensus.cc:2949] T d015a10f95eb4f19b6d7c53cb201fa80 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8bf6e26e98b14566899578a990b27d47 (127.19.226.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:47.711616 22177 raft_consensus.cc:2949] T d015a10f95eb4f19b6d7c53cb201fa80 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8bf6e26e98b14566899578a990b27d47 (127.19.226.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:47.726089 21412 catalog_manager.cc:5526] T d015a10f95eb4f19b6d7c53cb201fa80 P 88935ab835544e9889d05a6587d8cf21 reported cstate change: config changed from index 3 to 4, VOTER 8bf6e26e98b14566899578a990b27d47 (127.19.226.193) evicted. New cstate: current_term: 1 leader_uuid: "787ddcc824c2490a808fc5a53452dc5e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:47.729501 21396 catalog_manager.cc:5039] ChangeConfig:REMOVE_PEER RPC for tablet d015a10f95eb4f19b6d7c53cb201fa80 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250109 08:57:47.744550 21503 tablet_service.cc:1514] Processing DeleteTablet for tablet d015a10f95eb4f19b6d7c53cb201fa80 with delete_type TABLET_DATA_TOMBSTONED (TS 8bf6e26e98b14566899578a990b27d47 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:39242
I20250109 08:57:47.747431 22440 tablet_replica.cc:331] T d015a10f95eb4f19b6d7c53cb201fa80 P 8bf6e26e98b14566899578a990b27d47: stopping tablet replica
I20250109 08:57:47.751586 22440 raft_consensus.cc:2238] T d015a10f95eb4f19b6d7c53cb201fa80 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:57:47.752940 22440 raft_consensus.cc:2267] T d015a10f95eb4f19b6d7c53cb201fa80 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:57:47.756872 22440 ts_tablet_manager.cc:1905] T d015a10f95eb4f19b6d7c53cb201fa80 P 8bf6e26e98b14566899578a990b27d47: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250109 08:57:47.772198 22440 ts_tablet_manager.cc:1918] T d015a10f95eb4f19b6d7c53cb201fa80 P 8bf6e26e98b14566899578a990b27d47: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20250109 08:57:47.772634 22440 log.cc:1198] T d015a10f95eb4f19b6d7c53cb201fa80 P 8bf6e26e98b14566899578a990b27d47: Deleting WAL directory at /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736412987538469-20363-0/minicluster-data/ts-0-root/wals/d015a10f95eb4f19b6d7c53cb201fa80
I20250109 08:57:47.774060 21396 catalog_manager.cc:4872] TS 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): tablet d015a10f95eb4f19b6d7c53cb201fa80 (table test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]) successfully deleted
I20250109 08:57:48.828272 21663 consensus_queue.cc:237] T d52f79ecf037457cbcc09d5fa43456cf P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:57:48.835099 21511 raft_consensus.cc:1270] T d52f79ecf037457cbcc09d5fa43456cf P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250109 08:57:48.835064 21589 raft_consensus.cc:1270] T d52f79ecf037457cbcc09d5fa43456cf P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20250109 08:57:48.835626 21627 consensus_peers.cc:487] T d52f79ecf037457cbcc09d5fa43456cf P 787ddcc824c2490a808fc5a53452dc5e -> Peer 88935ab835544e9889d05a6587d8cf21 (127.19.226.196:44363): Couldn't send request to peer 88935ab835544e9889d05a6587d8cf21. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d52f79ecf037457cbcc09d5fa43456cf. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:57:48.836519 22441 consensus_queue.cc:1035] T d52f79ecf037457cbcc09d5fa43456cf P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:57:48.837384 22439 consensus_queue.cc:1035] T d52f79ecf037457cbcc09d5fa43456cf P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:57:48.849052 22439 raft_consensus.cc:2949] T d52f79ecf037457cbcc09d5fa43456cf P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:57:48.852101 21509 raft_consensus.cc:2949] T d52f79ecf037457cbcc09d5fa43456cf P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:57:48.854352 21589 raft_consensus.cc:2949] T d52f79ecf037457cbcc09d5fa43456cf P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:57:48.869616 21411 catalog_manager.cc:5526] T d52f79ecf037457cbcc09d5fa43456cf P 787ddcc824c2490a808fc5a53452dc5e reported cstate change: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New cstate: current_term: 1 leader_uuid: "787ddcc824c2490a808fc5a53452dc5e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20250109 08:57:48.870733 21509 consensus_queue.cc:237] T 0cc4fcda876242c396b804edebebb01f P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:57:48.879886 21663 raft_consensus.cc:1270] T 0cc4fcda876242c396b804edebebb01f P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Refusing update from remote peer 8bf6e26e98b14566899578a990b27d47: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20250109 08:57:48.880563 21478 consensus_peers.cc:487] T 0cc4fcda876242c396b804edebebb01f P 8bf6e26e98b14566899578a990b27d47 -> Peer 88935ab835544e9889d05a6587d8cf21 (127.19.226.196:44363): Couldn't send request to peer 88935ab835544e9889d05a6587d8cf21. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0cc4fcda876242c396b804edebebb01f. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:57:48.881599 21589 raft_consensus.cc:1270] T 0cc4fcda876242c396b804edebebb01f P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Refusing update from remote peer 8bf6e26e98b14566899578a990b27d47: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250109 08:57:48.881686 22410 consensus_queue.cc:1035] T 0cc4fcda876242c396b804edebebb01f P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20250109 08:57:48.885229 22427 consensus_queue.cc:1035] T 0cc4fcda876242c396b804edebebb01f P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20250109 08:57:48.891009 22410 raft_consensus.cc:2949] T 0cc4fcda876242c396b804edebebb01f P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:57:48.895884 21663 raft_consensus.cc:2949] T 0cc4fcda876242c396b804edebebb01f P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:57:48.912211 21658 consensus_queue.cc:237] T 5717a02ef53a4b22a96f431ca6c20d93 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:57:48.916997 21412 catalog_manager.cc:5526] T 0cc4fcda876242c396b804edebebb01f P 8bf6e26e98b14566899578a990b27d47 reported cstate change: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New cstate: current_term: 1 leader_uuid: "8bf6e26e98b14566899578a990b27d47" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20250109 08:57:48.895884 21589 raft_consensus.cc:2949] T 0cc4fcda876242c396b804edebebb01f P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:57:48.943848 21581 raft_consensus.cc:1270] T 5717a02ef53a4b22a96f431ca6c20d93 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20250109 08:57:48.945191 21627 consensus_peers.cc:487] T 5717a02ef53a4b22a96f431ca6c20d93 P 787ddcc824c2490a808fc5a53452dc5e -> Peer 88935ab835544e9889d05a6587d8cf21 (127.19.226.196:44363): Couldn't send request to peer 88935ab835544e9889d05a6587d8cf21. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5717a02ef53a4b22a96f431ca6c20d93. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:57:48.945859 21509 raft_consensus.cc:1270] T 5717a02ef53a4b22a96f431ca6c20d93 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250109 08:57:48.947273 22441 consensus_queue.cc:1035] T 5717a02ef53a4b22a96f431ca6c20d93 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:57:48.949877 22449 consensus_queue.cc:1035] T 5717a02ef53a4b22a96f431ca6c20d93 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20250109 08:57:48.957954 22439 raft_consensus.cc:2949] T 5717a02ef53a4b22a96f431ca6c20d93 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:57:48.959447 21511 raft_consensus.cc:2949] T 5717a02ef53a4b22a96f431ca6c20d93 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:57:48.959561 21581 raft_consensus.cc:2949] T 5717a02ef53a4b22a96f431ca6c20d93 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:57:48.975162 21663 consensus_queue.cc:237] T b00671c87aac405a81c6543f4defd722 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:57:48.976076 21409 catalog_manager.cc:5526] T 5717a02ef53a4b22a96f431ca6c20d93 P 787ddcc824c2490a808fc5a53452dc5e reported cstate change: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New cstate: current_term: 1 leader_uuid: "787ddcc824c2490a808fc5a53452dc5e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20250109 08:57:48.994325 21589 raft_consensus.cc:1270] T b00671c87aac405a81c6543f4defd722 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250109 08:57:48.995551 22449 consensus_queue.cc:1035] T b00671c87aac405a81c6543f4defd722 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:57:48.996734 21511 raft_consensus.cc:1270] T b00671c87aac405a81c6543f4defd722 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20250109 08:57:48.997371 21627 consensus_peers.cc:487] T b00671c87aac405a81c6543f4defd722 P 787ddcc824c2490a808fc5a53452dc5e -> Peer 88935ab835544e9889d05a6587d8cf21 (127.19.226.196:44363): Couldn't send request to peer 88935ab835544e9889d05a6587d8cf21. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b00671c87aac405a81c6543f4defd722. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:57:48.998147 22449 consensus_queue.cc:1035] T b00671c87aac405a81c6543f4defd722 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:57:49.003624 22439 raft_consensus.cc:2949] T b00671c87aac405a81c6543f4defd722 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:57:49.005249 21589 raft_consensus.cc:2949] T b00671c87aac405a81c6543f4defd722 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:57:49.007289 21511 raft_consensus.cc:2949] T b00671c87aac405a81c6543f4defd722 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:57:49.025985 21412 catalog_manager.cc:5526] T b00671c87aac405a81c6543f4defd722 P a2b1a692435241ee988e0bd1661cd4c3 reported cstate change: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New cstate: current_term: 1 leader_uuid: "787ddcc824c2490a808fc5a53452dc5e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:57:49.377300 22464 ts_tablet_manager.cc:927] T d52f79ecf037457cbcc09d5fa43456cf P 88935ab835544e9889d05a6587d8cf21: Initiating tablet copy from peer 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:49.378988 22464 tablet_copy_client.cc:323] T d52f79ecf037457cbcc09d5fa43456cf P 88935ab835544e9889d05a6587d8cf21: tablet copy: Beginning tablet copy session from remote peer at address 127.19.226.195:35731
I20250109 08:57:49.380626 21673 tablet_copy_service.cc:140] P 787ddcc824c2490a808fc5a53452dc5e: Received BeginTabletCopySession request for tablet d52f79ecf037457cbcc09d5fa43456cf from peer 88935ab835544e9889d05a6587d8cf21 ({username='slave'} at 127.0.0.1:52376)
I20250109 08:57:49.381196 21673 tablet_copy_service.cc:161] P 787ddcc824c2490a808fc5a53452dc5e: Beginning new tablet copy session on tablet d52f79ecf037457cbcc09d5fa43456cf from peer 88935ab835544e9889d05a6587d8cf21 at {username='slave'} at 127.0.0.1:52376: session id = 88935ab835544e9889d05a6587d8cf21-d52f79ecf037457cbcc09d5fa43456cf
I20250109 08:57:49.388273 21673 tablet_copy_source_session.cc:215] T d52f79ecf037457cbcc09d5fa43456cf P 787ddcc824c2490a808fc5a53452dc5e: Tablet Copy: opened 0 blocks and 1 log segments
I20250109 08:57:49.391053 22464 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d52f79ecf037457cbcc09d5fa43456cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:49.403971 22464 tablet_copy_client.cc:806] T d52f79ecf037457cbcc09d5fa43456cf P 88935ab835544e9889d05a6587d8cf21: tablet copy: Starting download of 0 data blocks...
I20250109 08:57:49.404697 22464 tablet_copy_client.cc:670] T d52f79ecf037457cbcc09d5fa43456cf P 88935ab835544e9889d05a6587d8cf21: tablet copy: Starting download of 1 WAL segments...
I20250109 08:57:49.408219 22464 tablet_copy_client.cc:538] T d52f79ecf037457cbcc09d5fa43456cf P 88935ab835544e9889d05a6587d8cf21: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250109 08:57:49.415764 22464 tablet_bootstrap.cc:492] T d52f79ecf037457cbcc09d5fa43456cf P 88935ab835544e9889d05a6587d8cf21: Bootstrap starting.
I20250109 08:57:49.433869 22467 ts_tablet_manager.cc:927] T 0cc4fcda876242c396b804edebebb01f P 88935ab835544e9889d05a6587d8cf21: Initiating tablet copy from peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725)
I20250109 08:57:49.433974 22464 tablet_bootstrap.cc:492] T d52f79ecf037457cbcc09d5fa43456cf P 88935ab835544e9889d05a6587d8cf21: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250109 08:57:49.435276 22464 tablet_bootstrap.cc:492] T d52f79ecf037457cbcc09d5fa43456cf P 88935ab835544e9889d05a6587d8cf21: Bootstrap complete.
I20250109 08:57:49.436511 22464 ts_tablet_manager.cc:1397] T d52f79ecf037457cbcc09d5fa43456cf P 88935ab835544e9889d05a6587d8cf21: Time spent bootstrapping tablet: real 0.021s user 0.014s sys 0.008s
I20250109 08:57:49.436443 22467 tablet_copy_client.cc:323] T 0cc4fcda876242c396b804edebebb01f P 88935ab835544e9889d05a6587d8cf21: tablet copy: Beginning tablet copy session from remote peer at address 127.19.226.193:42725
I20250109 08:57:49.439834 22464 raft_consensus.cc:357] T d52f79ecf037457cbcc09d5fa43456cf P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:57:49.440634 22464 raft_consensus.cc:738] T d52f79ecf037457cbcc09d5fa43456cf P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 88935ab835544e9889d05a6587d8cf21, State: Initialized, Role: LEARNER
I20250109 08:57:49.441164 22464 consensus_queue.cc:260] T d52f79ecf037457cbcc09d5fa43456cf P 88935ab835544e9889d05a6587d8cf21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:57:49.444453 22464 ts_tablet_manager.cc:1428] T d52f79ecf037457cbcc09d5fa43456cf P 88935ab835544e9889d05a6587d8cf21: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20250109 08:57:49.445832 21524 tablet_copy_service.cc:140] P 8bf6e26e98b14566899578a990b27d47: Received BeginTabletCopySession request for tablet 0cc4fcda876242c396b804edebebb01f from peer 88935ab835544e9889d05a6587d8cf21 ({username='slave'} at 127.0.0.1:41812)
I20250109 08:57:49.446365 21524 tablet_copy_service.cc:161] P 8bf6e26e98b14566899578a990b27d47: Beginning new tablet copy session on tablet 0cc4fcda876242c396b804edebebb01f from peer 88935ab835544e9889d05a6587d8cf21 at {username='slave'} at 127.0.0.1:41812: session id = 88935ab835544e9889d05a6587d8cf21-0cc4fcda876242c396b804edebebb01f
I20250109 08:57:49.449165 21673 tablet_copy_service.cc:342] P 787ddcc824c2490a808fc5a53452dc5e: Request end of tablet copy session 88935ab835544e9889d05a6587d8cf21-d52f79ecf037457cbcc09d5fa43456cf received from {username='slave'} at 127.0.0.1:52376
I20250109 08:57:49.449641 21673 tablet_copy_service.cc:434] P 787ddcc824c2490a808fc5a53452dc5e: ending tablet copy session 88935ab835544e9889d05a6587d8cf21-d52f79ecf037457cbcc09d5fa43456cf on tablet d52f79ecf037457cbcc09d5fa43456cf with peer 88935ab835544e9889d05a6587d8cf21
I20250109 08:57:49.455375 21524 tablet_copy_source_session.cc:215] T 0cc4fcda876242c396b804edebebb01f P 8bf6e26e98b14566899578a990b27d47: Tablet Copy: opened 0 blocks and 1 log segments
I20250109 08:57:49.458254 22467 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0cc4fcda876242c396b804edebebb01f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:49.473273 22467 tablet_copy_client.cc:806] T 0cc4fcda876242c396b804edebebb01f P 88935ab835544e9889d05a6587d8cf21: tablet copy: Starting download of 0 data blocks...
I20250109 08:57:49.473881 22467 tablet_copy_client.cc:670] T 0cc4fcda876242c396b804edebebb01f P 88935ab835544e9889d05a6587d8cf21: tablet copy: Starting download of 1 WAL segments...
I20250109 08:57:49.477105 22467 tablet_copy_client.cc:538] T 0cc4fcda876242c396b804edebebb01f P 88935ab835544e9889d05a6587d8cf21: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250109 08:57:49.479267 22464 ts_tablet_manager.cc:927] T b00671c87aac405a81c6543f4defd722 P 88935ab835544e9889d05a6587d8cf21: Initiating tablet copy from peer 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:49.481894 22464 tablet_copy_client.cc:323] T b00671c87aac405a81c6543f4defd722 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Beginning tablet copy session from remote peer at address 127.19.226.195:35731
I20250109 08:57:49.483335 21673 tablet_copy_service.cc:140] P 787ddcc824c2490a808fc5a53452dc5e: Received BeginTabletCopySession request for tablet b00671c87aac405a81c6543f4defd722 from peer 88935ab835544e9889d05a6587d8cf21 ({username='slave'} at 127.0.0.1:52376)
I20250109 08:57:49.483832 21673 tablet_copy_service.cc:161] P 787ddcc824c2490a808fc5a53452dc5e: Beginning new tablet copy session on tablet b00671c87aac405a81c6543f4defd722 from peer 88935ab835544e9889d05a6587d8cf21 at {username='slave'} at 127.0.0.1:52376: session id = 88935ab835544e9889d05a6587d8cf21-b00671c87aac405a81c6543f4defd722
I20250109 08:57:49.488998 22467 tablet_bootstrap.cc:492] T 0cc4fcda876242c396b804edebebb01f P 88935ab835544e9889d05a6587d8cf21: Bootstrap starting.
I20250109 08:57:49.492170 21673 tablet_copy_source_session.cc:215] T b00671c87aac405a81c6543f4defd722 P 787ddcc824c2490a808fc5a53452dc5e: Tablet Copy: opened 0 blocks and 1 log segments
I20250109 08:57:49.495213 22464 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b00671c87aac405a81c6543f4defd722. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:49.514704 22464 tablet_copy_client.cc:806] T b00671c87aac405a81c6543f4defd722 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Starting download of 0 data blocks...
I20250109 08:57:49.515393 22464 tablet_copy_client.cc:670] T b00671c87aac405a81c6543f4defd722 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Starting download of 1 WAL segments...
I20250109 08:57:49.517261 22467 tablet_bootstrap.cc:492] T 0cc4fcda876242c396b804edebebb01f P 88935ab835544e9889d05a6587d8cf21: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250109 08:57:49.518203 22467 tablet_bootstrap.cc:492] T 0cc4fcda876242c396b804edebebb01f P 88935ab835544e9889d05a6587d8cf21: Bootstrap complete.
I20250109 08:57:49.518924 22467 ts_tablet_manager.cc:1397] T 0cc4fcda876242c396b804edebebb01f P 88935ab835544e9889d05a6587d8cf21: Time spent bootstrapping tablet: real 0.030s user 0.012s sys 0.009s
I20250109 08:57:49.522686 22464 tablet_copy_client.cc:538] T b00671c87aac405a81c6543f4defd722 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250109 08:57:49.522567 22467 raft_consensus.cc:357] T 0cc4fcda876242c396b804edebebb01f P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:57:49.524880 22467 raft_consensus.cc:738] T 0cc4fcda876242c396b804edebebb01f P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 88935ab835544e9889d05a6587d8cf21, State: Initialized, Role: LEARNER
I20250109 08:57:49.525712 22467 consensus_queue.cc:260] T 0cc4fcda876242c396b804edebebb01f P 88935ab835544e9889d05a6587d8cf21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:57:49.529034 22467 ts_tablet_manager.cc:1428] T 0cc4fcda876242c396b804edebebb01f P 88935ab835544e9889d05a6587d8cf21: Time spent starting tablet: real 0.010s user 0.007s sys 0.000s
I20250109 08:57:49.530989 21524 tablet_copy_service.cc:342] P 8bf6e26e98b14566899578a990b27d47: Request end of tablet copy session 88935ab835544e9889d05a6587d8cf21-0cc4fcda876242c396b804edebebb01f received from {username='slave'} at 127.0.0.1:41812
I20250109 08:57:49.531445 21524 tablet_copy_service.cc:434] P 8bf6e26e98b14566899578a990b27d47: ending tablet copy session 88935ab835544e9889d05a6587d8cf21-0cc4fcda876242c396b804edebebb01f on tablet 0cc4fcda876242c396b804edebebb01f with peer 88935ab835544e9889d05a6587d8cf21
I20250109 08:57:49.532002 22464 tablet_bootstrap.cc:492] T b00671c87aac405a81c6543f4defd722 P 88935ab835544e9889d05a6587d8cf21: Bootstrap starting.
I20250109 08:57:49.546362 22467 ts_tablet_manager.cc:927] T 5717a02ef53a4b22a96f431ca6c20d93 P 88935ab835544e9889d05a6587d8cf21: Initiating tablet copy from peer 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:49.550015 22467 tablet_copy_client.cc:323] T 5717a02ef53a4b22a96f431ca6c20d93 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Beginning tablet copy session from remote peer at address 127.19.226.195:35731
I20250109 08:57:49.551513 21673 tablet_copy_service.cc:140] P 787ddcc824c2490a808fc5a53452dc5e: Received BeginTabletCopySession request for tablet 5717a02ef53a4b22a96f431ca6c20d93 from peer 88935ab835544e9889d05a6587d8cf21 ({username='slave'} at 127.0.0.1:52376)
I20250109 08:57:49.552021 21673 tablet_copy_service.cc:161] P 787ddcc824c2490a808fc5a53452dc5e: Beginning new tablet copy session on tablet 5717a02ef53a4b22a96f431ca6c20d93 from peer 88935ab835544e9889d05a6587d8cf21 at {username='slave'} at 127.0.0.1:52376: session id = 88935ab835544e9889d05a6587d8cf21-5717a02ef53a4b22a96f431ca6c20d93
I20250109 08:57:49.554795 22464 tablet_bootstrap.cc:492] T b00671c87aac405a81c6543f4defd722 P 88935ab835544e9889d05a6587d8cf21: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250109 08:57:49.555697 22464 tablet_bootstrap.cc:492] T b00671c87aac405a81c6543f4defd722 P 88935ab835544e9889d05a6587d8cf21: Bootstrap complete.
I20250109 08:57:49.556346 22464 ts_tablet_manager.cc:1397] T b00671c87aac405a81c6543f4defd722 P 88935ab835544e9889d05a6587d8cf21: Time spent bootstrapping tablet: real 0.025s user 0.020s sys 0.002s
I20250109 08:57:49.560724 22464 raft_consensus.cc:357] T b00671c87aac405a81c6543f4defd722 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:57:49.561622 22464 raft_consensus.cc:738] T b00671c87aac405a81c6543f4defd722 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 88935ab835544e9889d05a6587d8cf21, State: Initialized, Role: LEARNER
I20250109 08:57:49.562306 22464 consensus_queue.cc:260] T b00671c87aac405a81c6543f4defd722 P 88935ab835544e9889d05a6587d8cf21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:57:49.564709 22464 ts_tablet_manager.cc:1428] T b00671c87aac405a81c6543f4defd722 P 88935ab835544e9889d05a6587d8cf21: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20250109 08:57:49.566308 21672 tablet_copy_service.cc:342] P 787ddcc824c2490a808fc5a53452dc5e: Request end of tablet copy session 88935ab835544e9889d05a6587d8cf21-b00671c87aac405a81c6543f4defd722 received from {username='slave'} at 127.0.0.1:52376
I20250109 08:57:49.566673 21672 tablet_copy_service.cc:434] P 787ddcc824c2490a808fc5a53452dc5e: ending tablet copy session 88935ab835544e9889d05a6587d8cf21-b00671c87aac405a81c6543f4defd722 on tablet b00671c87aac405a81c6543f4defd722 with peer 88935ab835544e9889d05a6587d8cf21
I20250109 08:57:49.568020 21673 tablet_copy_source_session.cc:215] T 5717a02ef53a4b22a96f431ca6c20d93 P 787ddcc824c2490a808fc5a53452dc5e: Tablet Copy: opened 0 blocks and 1 log segments
I20250109 08:57:49.574201 22467 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5717a02ef53a4b22a96f431ca6c20d93. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:49.587040 22467 tablet_copy_client.cc:806] T 5717a02ef53a4b22a96f431ca6c20d93 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Starting download of 0 data blocks...
I20250109 08:57:49.587577 22467 tablet_copy_client.cc:670] T 5717a02ef53a4b22a96f431ca6c20d93 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Starting download of 1 WAL segments...
I20250109 08:57:49.594656 22467 tablet_copy_client.cc:538] T 5717a02ef53a4b22a96f431ca6c20d93 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250109 08:57:49.602277 22467 tablet_bootstrap.cc:492] T 5717a02ef53a4b22a96f431ca6c20d93 P 88935ab835544e9889d05a6587d8cf21: Bootstrap starting.
I20250109 08:57:49.618031 22467 tablet_bootstrap.cc:492] T 5717a02ef53a4b22a96f431ca6c20d93 P 88935ab835544e9889d05a6587d8cf21: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250109 08:57:49.618857 22467 tablet_bootstrap.cc:492] T 5717a02ef53a4b22a96f431ca6c20d93 P 88935ab835544e9889d05a6587d8cf21: Bootstrap complete.
I20250109 08:57:49.619439 22467 ts_tablet_manager.cc:1397] T 5717a02ef53a4b22a96f431ca6c20d93 P 88935ab835544e9889d05a6587d8cf21: Time spent bootstrapping tablet: real 0.017s user 0.008s sys 0.007s
I20250109 08:57:49.622185 22467 raft_consensus.cc:357] T 5717a02ef53a4b22a96f431ca6c20d93 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:57:49.623013 22467 raft_consensus.cc:738] T 5717a02ef53a4b22a96f431ca6c20d93 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 88935ab835544e9889d05a6587d8cf21, State: Initialized, Role: LEARNER
I20250109 08:57:49.623598 22467 consensus_queue.cc:260] T 5717a02ef53a4b22a96f431ca6c20d93 P 88935ab835544e9889d05a6587d8cf21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:57:49.625962 22467 ts_tablet_manager.cc:1428] T 5717a02ef53a4b22a96f431ca6c20d93 P 88935ab835544e9889d05a6587d8cf21: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20250109 08:57:49.627453 21673 tablet_copy_service.cc:342] P 787ddcc824c2490a808fc5a53452dc5e: Request end of tablet copy session 88935ab835544e9889d05a6587d8cf21-5717a02ef53a4b22a96f431ca6c20d93 received from {username='slave'} at 127.0.0.1:52376
I20250109 08:57:49.627848 21673 tablet_copy_service.cc:434] P 787ddcc824c2490a808fc5a53452dc5e: ending tablet copy session 88935ab835544e9889d05a6587d8cf21-5717a02ef53a4b22a96f431ca6c20d93 on tablet 5717a02ef53a4b22a96f431ca6c20d93 with peer 88935ab835544e9889d05a6587d8cf21
I20250109 08:57:49.903460 22177 raft_consensus.cc:1212] T d52f79ecf037457cbcc09d5fa43456cf P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20250109 08:57:49.980813 22177 raft_consensus.cc:1212] T 5717a02ef53a4b22a96f431ca6c20d93 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20250109 08:57:50.013020 22177 raft_consensus.cc:1212] T b00671c87aac405a81c6543f4defd722 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20250109 08:57:50.035185 22177 raft_consensus.cc:1212] T 0cc4fcda876242c396b804edebebb01f P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20250109 08:57:50.331965 22463 raft_consensus.cc:1059] T d52f79ecf037457cbcc09d5fa43456cf P 787ddcc824c2490a808fc5a53452dc5e: attempting to promote NON_VOTER 88935ab835544e9889d05a6587d8cf21 to VOTER
I20250109 08:57:50.334151 22463 consensus_queue.cc:237] T d52f79ecf037457cbcc09d5fa43456cf P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:57:50.340219 21581 raft_consensus.cc:1270] T d52f79ecf037457cbcc09d5fa43456cf P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:57:50.341869 21509 raft_consensus.cc:1270] T d52f79ecf037457cbcc09d5fa43456cf P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:57:50.342108 22438 consensus_queue.cc:1035] T d52f79ecf037457cbcc09d5fa43456cf P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:57:50.342845 22177 raft_consensus.cc:1270] T d52f79ecf037457cbcc09d5fa43456cf P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:57:50.344527 22463 consensus_queue.cc:1035] T d52f79ecf037457cbcc09d5fa43456cf P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:57:50.348634 22438 consensus_queue.cc:1035] T d52f79ecf037457cbcc09d5fa43456cf P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:57:50.358883 22463 raft_consensus.cc:2949] T d52f79ecf037457cbcc09d5fa43456cf P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:50.361516 22177 raft_consensus.cc:2949] T d52f79ecf037457cbcc09d5fa43456cf P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:50.365897 21509 raft_consensus.cc:2949] T d52f79ecf037457cbcc09d5fa43456cf P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:50.373719 22438 raft_consensus.cc:1059] T 5717a02ef53a4b22a96f431ca6c20d93 P 787ddcc824c2490a808fc5a53452dc5e: attempting to promote NON_VOTER 88935ab835544e9889d05a6587d8cf21 to VOTER
I20250109 08:57:50.375811 22438 consensus_queue.cc:237] T 5717a02ef53a4b22a96f431ca6c20d93 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:57:50.377943 21589 raft_consensus.cc:2949] T d52f79ecf037457cbcc09d5fa43456cf P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:50.382490 21411 catalog_manager.cc:5526] T d52f79ecf037457cbcc09d5fa43456cf P 88935ab835544e9889d05a6587d8cf21 reported cstate change: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "787ddcc824c2490a808fc5a53452dc5e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:50.398631 21581 raft_consensus.cc:1270] T 5717a02ef53a4b22a96f431ca6c20d93 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:57:50.399070 22177 raft_consensus.cc:1270] T 5717a02ef53a4b22a96f431ca6c20d93 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:57:50.401146 22483 consensus_queue.cc:1035] T 5717a02ef53a4b22a96f431ca6c20d93 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20250109 08:57:50.424163 21512 raft_consensus.cc:1270] T 5717a02ef53a4b22a96f431ca6c20d93 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:57:50.427529 22481 consensus_queue.cc:1035] T 5717a02ef53a4b22a96f431ca6c20d93 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:57:50.435361 22481 consensus_queue.cc:1035] T 5717a02ef53a4b22a96f431ca6c20d93 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20250109 08:57:50.449326 22483 raft_consensus.cc:2949] T 5717a02ef53a4b22a96f431ca6c20d93 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:50.450060 22177 raft_consensus.cc:2949] T 5717a02ef53a4b22a96f431ca6c20d93 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:50.463120 21658 consensus_queue.cc:237] T d52f79ecf037457cbcc09d5fa43456cf P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:57:50.465624 21509 raft_consensus.cc:2949] T 5717a02ef53a4b22a96f431ca6c20d93 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:50.470494 22178 raft_consensus.cc:1270] T d52f79ecf037457cbcc09d5fa43456cf P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250109 08:57:50.472220 22481 consensus_queue.cc:1035] T d52f79ecf037457cbcc09d5fa43456cf P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250109 08:57:50.472957 21589 raft_consensus.cc:2949] T 5717a02ef53a4b22a96f431ca6c20d93 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:50.476217 21581 raft_consensus.cc:1270] T d52f79ecf037457cbcc09d5fa43456cf P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250109 08:57:50.477881 22485 consensus_queue.cc:1035] T d52f79ecf037457cbcc09d5fa43456cf P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250109 08:57:50.484714 21410 catalog_manager.cc:5526] T 5717a02ef53a4b22a96f431ca6c20d93 P 88935ab835544e9889d05a6587d8cf21 reported cstate change: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "787ddcc824c2490a808fc5a53452dc5e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:50.489465 22480 raft_consensus.cc:2949] T d52f79ecf037457cbcc09d5fa43456cf P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8bf6e26e98b14566899578a990b27d47 (127.19.226.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:50.495210 22178 raft_consensus.cc:2949] T d52f79ecf037457cbcc09d5fa43456cf P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8bf6e26e98b14566899578a990b27d47 (127.19.226.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:50.497888 21396 catalog_manager.cc:5039] ChangeConfig:REMOVE_PEER RPC for tablet d52f79ecf037457cbcc09d5fa43456cf with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250109 08:57:50.496456 21581 raft_consensus.cc:2949] T d52f79ecf037457cbcc09d5fa43456cf P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8bf6e26e98b14566899578a990b27d47 (127.19.226.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:50.533001 21656 consensus_queue.cc:237] T 5717a02ef53a4b22a96f431ca6c20d93 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:57:50.521273 21412 catalog_manager.cc:5526] T d52f79ecf037457cbcc09d5fa43456cf P 88935ab835544e9889d05a6587d8cf21 reported cstate change: config changed from index 3 to 4, VOTER 8bf6e26e98b14566899578a990b27d47 (127.19.226.193) evicted. New cstate: current_term: 1 leader_uuid: "787ddcc824c2490a808fc5a53452dc5e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:50.555954 21509 raft_consensus.cc:1270] T 5717a02ef53a4b22a96f431ca6c20d93 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250109 08:57:50.557386 22483 consensus_queue.cc:1035] T 5717a02ef53a4b22a96f431ca6c20d93 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20250109 08:57:50.561518 21503 tablet_service.cc:1514] Processing DeleteTablet for tablet d52f79ecf037457cbcc09d5fa43456cf with delete_type TABLET_DATA_TOMBSTONED (TS 8bf6e26e98b14566899578a990b27d47 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:39242
I20250109 08:57:50.558539 22178 raft_consensus.cc:1270] T 5717a02ef53a4b22a96f431ca6c20d93 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250109 08:57:50.566766 22481 raft_consensus.cc:2949] T 5717a02ef53a4b22a96f431ca6c20d93 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:50.568164 22493 tablet_replica.cc:331] T d52f79ecf037457cbcc09d5fa43456cf P 8bf6e26e98b14566899578a990b27d47: stopping tablet replica
I20250109 08:57:50.569911 22493 raft_consensus.cc:2238] T d52f79ecf037457cbcc09d5fa43456cf P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:57:50.569281 21512 raft_consensus.cc:2949] T 5717a02ef53a4b22a96f431ca6c20d93 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:50.570452 22480 consensus_queue.cc:1035] T 5717a02ef53a4b22a96f431ca6c20d93 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250109 08:57:50.570686 22493 raft_consensus.cc:2267] T d52f79ecf037457cbcc09d5fa43456cf P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:57:50.576854 22178 raft_consensus.cc:2949] T 5717a02ef53a4b22a96f431ca6c20d93 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:50.577834 22493 ts_tablet_manager.cc:1905] T d52f79ecf037457cbcc09d5fa43456cf P 8bf6e26e98b14566899578a990b27d47: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250109 08:57:50.581789 22445 raft_consensus.cc:1059] T 0cc4fcda876242c396b804edebebb01f P 8bf6e26e98b14566899578a990b27d47: attempting to promote NON_VOTER 88935ab835544e9889d05a6587d8cf21 to VOTER
I20250109 08:57:50.581913 21396 catalog_manager.cc:5039] ChangeConfig:REMOVE_PEER RPC for tablet 5717a02ef53a4b22a96f431ca6c20d93 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250109 08:57:50.585047 22445 consensus_queue.cc:237] T 0cc4fcda876242c396b804edebebb01f P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:57:50.596688 21409 catalog_manager.cc:5526] T 5717a02ef53a4b22a96f431ca6c20d93 P 88935ab835544e9889d05a6587d8cf21 reported cstate change: config changed from index 3 to 4, VOTER a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194) evicted. New cstate: current_term: 1 leader_uuid: "787ddcc824c2490a808fc5a53452dc5e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:50.597550 22178 raft_consensus.cc:1270] T 0cc4fcda876242c396b804edebebb01f P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Refusing update from remote peer 8bf6e26e98b14566899578a990b27d47: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:57:50.598640 21589 raft_consensus.cc:1270] T 0cc4fcda876242c396b804edebebb01f P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Refusing update from remote peer 8bf6e26e98b14566899578a990b27d47: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:57:50.598670 22445 consensus_queue.cc:1035] T 0cc4fcda876242c396b804edebebb01f P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20250109 08:57:50.600873 22445 consensus_queue.cc:1035] T 0cc4fcda876242c396b804edebebb01f P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.006s
I20250109 08:57:50.607889 21656 raft_consensus.cc:1270] T 0cc4fcda876242c396b804edebebb01f P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Refusing update from remote peer 8bf6e26e98b14566899578a990b27d47: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:57:50.612775 21570 tablet_service.cc:1514] Processing DeleteTablet for tablet 5717a02ef53a4b22a96f431ca6c20d93 with delete_type TABLET_DATA_TOMBSTONED (TS a2b1a692435241ee988e0bd1661cd4c3 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:34732
I20250109 08:57:50.615449 22445 consensus_queue.cc:1035] T 0cc4fcda876242c396b804edebebb01f P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20250109 08:57:50.621506 22483 raft_consensus.cc:1059] T b00671c87aac405a81c6543f4defd722 P 787ddcc824c2490a808fc5a53452dc5e: attempting to promote NON_VOTER 88935ab835544e9889d05a6587d8cf21 to VOTER
I20250109 08:57:50.627562 22483 consensus_queue.cc:237] T b00671c87aac405a81c6543f4defd722 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:57:50.635120 22493 ts_tablet_manager.cc:1918] T d52f79ecf037457cbcc09d5fa43456cf P 8bf6e26e98b14566899578a990b27d47: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20250109 08:57:50.635681 22493 log.cc:1198] T d52f79ecf037457cbcc09d5fa43456cf P 8bf6e26e98b14566899578a990b27d47: Deleting WAL directory at /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736412987538469-20363-0/minicluster-data/ts-0-root/wals/d52f79ecf037457cbcc09d5fa43456cf
I20250109 08:57:50.639048 21396 catalog_manager.cc:4872] TS 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): tablet d52f79ecf037457cbcc09d5fa43456cf (table test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]) successfully deleted
I20250109 08:57:50.644223 22410 raft_consensus.cc:2949] T 0cc4fcda876242c396b804edebebb01f P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:50.651602 21581 raft_consensus.cc:1270] T b00671c87aac405a81c6543f4defd722 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:57:50.650447 22497 tablet_replica.cc:331] T 5717a02ef53a4b22a96f431ca6c20d93 P a2b1a692435241ee988e0bd1661cd4c3: stopping tablet replica
I20250109 08:57:50.653247 22177 raft_consensus.cc:1270] T b00671c87aac405a81c6543f4defd722 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:57:50.653461 22497 raft_consensus.cc:2238] T 5717a02ef53a4b22a96f431ca6c20d93 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:57:50.653537 21509 raft_consensus.cc:1270] T b00671c87aac405a81c6543f4defd722 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:57:50.654251 22480 consensus_queue.cc:1035] T b00671c87aac405a81c6543f4defd722 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:57:50.655442 22481 consensus_queue.cc:1035] T b00671c87aac405a81c6543f4defd722 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:57:50.656453 22485 consensus_queue.cc:1035] T b00671c87aac405a81c6543f4defd722 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:57:50.656560 21656 raft_consensus.cc:2949] T 0cc4fcda876242c396b804edebebb01f P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:50.661800 21581 raft_consensus.cc:2949] T 0cc4fcda876242c396b804edebebb01f P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:50.664952 22497 raft_consensus.cc:2267] T 5717a02ef53a4b22a96f431ca6c20d93 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:57:50.665750 22178 raft_consensus.cc:2949] T 0cc4fcda876242c396b804edebebb01f P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:50.678283 22497 ts_tablet_manager.cc:1905] T 5717a02ef53a4b22a96f431ca6c20d93 P a2b1a692435241ee988e0bd1661cd4c3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250109 08:57:50.686280 21410 catalog_manager.cc:5526] T 0cc4fcda876242c396b804edebebb01f P 8bf6e26e98b14566899578a990b27d47 reported cstate change: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8bf6e26e98b14566899578a990b27d47" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250109 08:57:50.702486 21512 consensus_queue.cc:237] T 0cc4fcda876242c396b804edebebb01f P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:57:50.717634 22483 raft_consensus.cc:2949] T b00671c87aac405a81c6543f4defd722 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:50.727056 21509 raft_consensus.cc:2949] T b00671c87aac405a81c6543f4defd722 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:50.733177 21583 raft_consensus.cc:2949] T b00671c87aac405a81c6543f4defd722 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:50.736078 22177 raft_consensus.cc:2949] T b00671c87aac405a81c6543f4defd722 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:50.743219 21409 catalog_manager.cc:5526] T b00671c87aac405a81c6543f4defd722 P 787ddcc824c2490a808fc5a53452dc5e reported cstate change: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "787ddcc824c2490a808fc5a53452dc5e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250109 08:57:50.743257 21589 raft_consensus.cc:1270] T 0cc4fcda876242c396b804edebebb01f P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Refusing update from remote peer 8bf6e26e98b14566899578a990b27d47: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250109 08:57:50.750643 22500 consensus_queue.cc:1035] T 0cc4fcda876242c396b804edebebb01f P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20250109 08:57:50.753829 22176 raft_consensus.cc:1270] T 0cc4fcda876242c396b804edebebb01f P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Refusing update from remote peer 8bf6e26e98b14566899578a990b27d47: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250109 08:57:50.766171 22500 consensus_queue.cc:1035] T 0cc4fcda876242c396b804edebebb01f P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20250109 08:57:50.774497 21663 consensus_queue.cc:237] T b00671c87aac405a81c6543f4defd722 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:57:50.777031 22410 raft_consensus.cc:2949] T 0cc4fcda876242c396b804edebebb01f P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:50.796046 21589 raft_consensus.cc:1270] T b00671c87aac405a81c6543f4defd722 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250109 08:57:50.781674 21588 raft_consensus.cc:2949] T 0cc4fcda876242c396b804edebebb01f P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:50.782909 22176 raft_consensus.cc:2949] T 0cc4fcda876242c396b804edebebb01f P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:50.801836 22497 ts_tablet_manager.cc:1918] T 5717a02ef53a4b22a96f431ca6c20d93 P a2b1a692435241ee988e0bd1661cd4c3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20250109 08:57:50.802913 22497 log.cc:1198] T 5717a02ef53a4b22a96f431ca6c20d93 P a2b1a692435241ee988e0bd1661cd4c3: Deleting WAL directory at /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736412987538469-20363-0/minicluster-data/ts-1-root/wals/5717a02ef53a4b22a96f431ca6c20d93
I20250109 08:57:50.803324 22481 consensus_queue.cc:1035] T b00671c87aac405a81c6543f4defd722 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250109 08:57:50.805526 21397 catalog_manager.cc:4872] TS a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): tablet 5717a02ef53a4b22a96f431ca6c20d93 (table test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]) successfully deleted
I20250109 08:57:50.814741 22177 raft_consensus.cc:1270] T b00671c87aac405a81c6543f4defd722 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250109 08:57:50.817476 22485 raft_consensus.cc:2949] T b00671c87aac405a81c6543f4defd722 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8bf6e26e98b14566899578a990b27d47 (127.19.226.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:50.819952 21589 raft_consensus.cc:2949] T b00671c87aac405a81c6543f4defd722 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8bf6e26e98b14566899578a990b27d47 (127.19.226.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:50.826033 21411 catalog_manager.cc:5526] T 0cc4fcda876242c396b804edebebb01f P 88935ab835544e9889d05a6587d8cf21 reported cstate change: config changed from index 3 to 4, VOTER 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195) evicted. New cstate: current_term: 1 leader_uuid: "8bf6e26e98b14566899578a990b27d47" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:50.829268 22480 consensus_queue.cc:1035] T b00671c87aac405a81c6543f4defd722 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250109 08:57:50.829922 21396 catalog_manager.cc:5039] ChangeConfig:REMOVE_PEER RPC for tablet 0cc4fcda876242c396b804edebebb01f with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250109 08:57:50.841539 21653 tablet_service.cc:1514] Processing DeleteTablet for tablet 0cc4fcda876242c396b804edebebb01f with delete_type TABLET_DATA_TOMBSTONED (TS 787ddcc824c2490a808fc5a53452dc5e not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:58436
I20250109 08:57:50.849300 21411 catalog_manager.cc:5526] T b00671c87aac405a81c6543f4defd722 P a2b1a692435241ee988e0bd1661cd4c3 reported cstate change: config changed from index 3 to 4, VOTER 8bf6e26e98b14566899578a990b27d47 (127.19.226.193) evicted. New cstate: current_term: 1 leader_uuid: "787ddcc824c2490a808fc5a53452dc5e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:50.849560 22176 raft_consensus.cc:2949] T b00671c87aac405a81c6543f4defd722 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8bf6e26e98b14566899578a990b27d47 (127.19.226.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:50.859997 21503 tablet_service.cc:1514] Processing DeleteTablet for tablet b00671c87aac405a81c6543f4defd722 with delete_type TABLET_DATA_TOMBSTONED (TS 8bf6e26e98b14566899578a990b27d47 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:39242
I20250109 08:57:50.861063 22493 tablet_replica.cc:331] T b00671c87aac405a81c6543f4defd722 P 8bf6e26e98b14566899578a990b27d47: stopping tablet replica
I20250109 08:57:50.862039 22493 raft_consensus.cc:2238] T b00671c87aac405a81c6543f4defd722 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:57:50.862687 22493 raft_consensus.cc:2267] T b00671c87aac405a81c6543f4defd722 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:57:50.866243 22493 ts_tablet_manager.cc:1905] T b00671c87aac405a81c6543f4defd722 P 8bf6e26e98b14566899578a990b27d47: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250109 08:57:50.868269 21396 catalog_manager.cc:5039] ChangeConfig:REMOVE_PEER RPC for tablet b00671c87aac405a81c6543f4defd722 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250109 08:57:50.869109 22505 tablet_replica.cc:331] T 0cc4fcda876242c396b804edebebb01f P 787ddcc824c2490a808fc5a53452dc5e: stopping tablet replica
I20250109 08:57:50.870229 22505 raft_consensus.cc:2238] T 0cc4fcda876242c396b804edebebb01f P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:57:50.870869 22505 raft_consensus.cc:2267] T 0cc4fcda876242c396b804edebebb01f P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:57:50.873849 22505 ts_tablet_manager.cc:1905] T 0cc4fcda876242c396b804edebebb01f P 787ddcc824c2490a808fc5a53452dc5e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250109 08:57:50.884874 22493 ts_tablet_manager.cc:1918] T b00671c87aac405a81c6543f4defd722 P 8bf6e26e98b14566899578a990b27d47: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20250109 08:57:50.885251 22493 log.cc:1198] T b00671c87aac405a81c6543f4defd722 P 8bf6e26e98b14566899578a990b27d47: Deleting WAL directory at /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736412987538469-20363-0/minicluster-data/ts-0-root/wals/b00671c87aac405a81c6543f4defd722
I20250109 08:57:50.886353 21396 catalog_manager.cc:4872] TS 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): tablet b00671c87aac405a81c6543f4defd722 (table test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]) successfully deleted
I20250109 08:57:50.892843 22505 ts_tablet_manager.cc:1918] T 0cc4fcda876242c396b804edebebb01f P 787ddcc824c2490a808fc5a53452dc5e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20250109 08:57:50.893304 22505 log.cc:1198] T 0cc4fcda876242c396b804edebebb01f P 787ddcc824c2490a808fc5a53452dc5e: Deleting WAL directory at /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736412987538469-20363-0/minicluster-data/ts-2-root/wals/0cc4fcda876242c396b804edebebb01f
I20250109 08:57:50.894582 21396 catalog_manager.cc:4872] TS 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731): tablet 0cc4fcda876242c396b804edebebb01f (table test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]) successfully deleted
I20250109 08:57:51.932530 21656 consensus_queue.cc:237] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:57:51.938074 21588 raft_consensus.cc:1270] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20250109 08:57:51.938611 21627 consensus_peers.cc:487] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P 787ddcc824c2490a808fc5a53452dc5e -> Peer 88935ab835544e9889d05a6587d8cf21 (127.19.226.196:44363): Couldn't send request to peer 88935ab835544e9889d05a6587d8cf21. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b54d6381a2cc41dfb59b1f6b99c0c5d4. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:57:51.938972 21512 raft_consensus.cc:1270] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250109 08:57:51.940127 22485 consensus_queue.cc:1035] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:57:51.941035 22480 consensus_queue.cc:1035] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:57:51.949198 22480 raft_consensus.cc:2949] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:57:51.950475 21588 raft_consensus.cc:2949] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:57:51.952342 21512 raft_consensus.cc:2949] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:57:51.965795 21410 catalog_manager.cc:5526] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P 8bf6e26e98b14566899578a990b27d47 reported cstate change: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New cstate: current_term: 1 leader_uuid: "787ddcc824c2490a808fc5a53452dc5e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:57:51.967489 21588 consensus_queue.cc:237] T 126a163a89c448be89ccb9490e9f0168 P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:57:51.975929 21656 raft_consensus.cc:1270] T 126a163a89c448be89ccb9490e9f0168 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Refusing update from remote peer a2b1a692435241ee988e0bd1661cd4c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20250109 08:57:51.977227 21553 consensus_peers.cc:487] T 126a163a89c448be89ccb9490e9f0168 P a2b1a692435241ee988e0bd1661cd4c3 -> Peer 88935ab835544e9889d05a6587d8cf21 (127.19.226.196:44363): Couldn't send request to peer 88935ab835544e9889d05a6587d8cf21. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 126a163a89c448be89ccb9490e9f0168. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:57:51.977486 21512 raft_consensus.cc:1270] T 126a163a89c448be89ccb9490e9f0168 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Refusing update from remote peer a2b1a692435241ee988e0bd1661cd4c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250109 08:57:51.978842 22507 consensus_queue.cc:1035] T 126a163a89c448be89ccb9490e9f0168 P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:57:51.980048 22475 consensus_queue.cc:1035] T 126a163a89c448be89ccb9490e9f0168 P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:57:52.054030 22443 raft_consensus.cc:2949] T 126a163a89c448be89ccb9490e9f0168 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:57:52.066565 21512 raft_consensus.cc:2949] T 126a163a89c448be89ccb9490e9f0168 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:57:52.071272 21656 raft_consensus.cc:2949] T 126a163a89c448be89ccb9490e9f0168 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:57:52.076097 21509 consensus_queue.cc:237] T 2935d69446cd458e92046d2b2f1b74ab P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:57:52.077571 21411 catalog_manager.cc:5526] T 126a163a89c448be89ccb9490e9f0168 P a2b1a692435241ee988e0bd1661cd4c3 reported cstate change: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New cstate: current_term: 1 leader_uuid: "a2b1a692435241ee988e0bd1661cd4c3" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20250109 08:57:52.100330 21478 consensus_peers.cc:487] T 2935d69446cd458e92046d2b2f1b74ab P 8bf6e26e98b14566899578a990b27d47 -> Peer 88935ab835544e9889d05a6587d8cf21 (127.19.226.196:44363): Couldn't send request to peer 88935ab835544e9889d05a6587d8cf21. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2935d69446cd458e92046d2b2f1b74ab. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:57:52.101943 21588 raft_consensus.cc:1270] T 2935d69446cd458e92046d2b2f1b74ab P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Refusing update from remote peer 8bf6e26e98b14566899578a990b27d47: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250109 08:57:52.102509 21656 raft_consensus.cc:1270] T 2935d69446cd458e92046d2b2f1b74ab P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Refusing update from remote peer 8bf6e26e98b14566899578a990b27d47: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250109 08:57:52.103152 22410 consensus_queue.cc:1035] T 2935d69446cd458e92046d2b2f1b74ab P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:57:52.104868 22504 consensus_queue.cc:1035] T 2935d69446cd458e92046d2b2f1b74ab P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:57:52.127883 22522 raft_consensus.cc:2949] T 2935d69446cd458e92046d2b2f1b74ab P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:57:52.129962 21589 raft_consensus.cc:2949] T 2935d69446cd458e92046d2b2f1b74ab P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:57:52.135370 21656 raft_consensus.cc:2949] T 2935d69446cd458e92046d2b2f1b74ab P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:57:52.148911 21411 catalog_manager.cc:5526] T 2935d69446cd458e92046d2b2f1b74ab P a2b1a692435241ee988e0bd1661cd4c3 reported cstate change: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New cstate: current_term: 1 leader_uuid: "8bf6e26e98b14566899578a990b27d47" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:57:52.154512 21663 consensus_queue.cc:237] T 239ca74a4c0a455a9db2652d7debb989 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:57:52.166795 21589 raft_consensus.cc:1270] T 239ca74a4c0a455a9db2652d7debb989 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20250109 08:57:52.168200 21627 consensus_peers.cc:487] T 239ca74a4c0a455a9db2652d7debb989 P 787ddcc824c2490a808fc5a53452dc5e -> Peer 88935ab835544e9889d05a6587d8cf21 (127.19.226.196:44363): Couldn't send request to peer 88935ab835544e9889d05a6587d8cf21. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 239ca74a4c0a455a9db2652d7debb989. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:57:52.169631 22485 consensus_queue.cc:1035] T 239ca74a4c0a455a9db2652d7debb989 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:57:52.170589 21509 raft_consensus.cc:1270] T 239ca74a4c0a455a9db2652d7debb989 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250109 08:57:52.172535 22485 consensus_queue.cc:1035] T 239ca74a4c0a455a9db2652d7debb989 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:57:52.178208 22480 raft_consensus.cc:2949] T 239ca74a4c0a455a9db2652d7debb989 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:57:52.179808 21589 raft_consensus.cc:2949] T 239ca74a4c0a455a9db2652d7debb989 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:57:52.182893 21509 raft_consensus.cc:2949] T 239ca74a4c0a455a9db2652d7debb989 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:57:52.195063 21409 catalog_manager.cc:5526] T 239ca74a4c0a455a9db2652d7debb989 P a2b1a692435241ee988e0bd1661cd4c3 reported cstate change: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New cstate: current_term: 1 leader_uuid: "787ddcc824c2490a808fc5a53452dc5e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:57:52.536157 22527 ts_tablet_manager.cc:927] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P 88935ab835544e9889d05a6587d8cf21: Initiating tablet copy from peer 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:52.538015 22527 tablet_copy_client.cc:323] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Beginning tablet copy session from remote peer at address 127.19.226.195:35731
I20250109 08:57:52.539302 21673 tablet_copy_service.cc:140] P 787ddcc824c2490a808fc5a53452dc5e: Received BeginTabletCopySession request for tablet b54d6381a2cc41dfb59b1f6b99c0c5d4 from peer 88935ab835544e9889d05a6587d8cf21 ({username='slave'} at 127.0.0.1:52376)
I20250109 08:57:52.539834 21673 tablet_copy_service.cc:161] P 787ddcc824c2490a808fc5a53452dc5e: Beginning new tablet copy session on tablet b54d6381a2cc41dfb59b1f6b99c0c5d4 from peer 88935ab835544e9889d05a6587d8cf21 at {username='slave'} at 127.0.0.1:52376: session id = 88935ab835544e9889d05a6587d8cf21-b54d6381a2cc41dfb59b1f6b99c0c5d4
I20250109 08:57:52.549680 21673 tablet_copy_source_session.cc:215] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P 787ddcc824c2490a808fc5a53452dc5e: Tablet Copy: opened 0 blocks and 1 log segments
I20250109 08:57:52.555485 22527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b54d6381a2cc41dfb59b1f6b99c0c5d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:52.578358 22527 tablet_copy_client.cc:806] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Starting download of 0 data blocks...
I20250109 08:57:52.579344 22527 tablet_copy_client.cc:670] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Starting download of 1 WAL segments...
I20250109 08:57:52.582396 22527 tablet_copy_client.cc:538] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250109 08:57:52.588862 22527 tablet_bootstrap.cc:492] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P 88935ab835544e9889d05a6587d8cf21: Bootstrap starting.
I20250109 08:57:52.605700 22529 ts_tablet_manager.cc:927] T 126a163a89c448be89ccb9490e9f0168 P 88935ab835544e9889d05a6587d8cf21: Initiating tablet copy from peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
I20250109 08:57:52.610903 22527 tablet_bootstrap.cc:492] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P 88935ab835544e9889d05a6587d8cf21: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250109 08:57:52.611905 22527 tablet_bootstrap.cc:492] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P 88935ab835544e9889d05a6587d8cf21: Bootstrap complete.
I20250109 08:57:52.612639 22527 ts_tablet_manager.cc:1397] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P 88935ab835544e9889d05a6587d8cf21: Time spent bootstrapping tablet: real 0.024s user 0.014s sys 0.008s
I20250109 08:57:52.615965 22529 tablet_copy_client.cc:323] T 126a163a89c448be89ccb9490e9f0168 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Beginning tablet copy session from remote peer at address 127.19.226.194:45527
I20250109 08:57:52.617709 21599 tablet_copy_service.cc:140] P a2b1a692435241ee988e0bd1661cd4c3: Received BeginTabletCopySession request for tablet 126a163a89c448be89ccb9490e9f0168 from peer 88935ab835544e9889d05a6587d8cf21 ({username='slave'} at 127.0.0.1:33652)
I20250109 08:57:52.618216 21599 tablet_copy_service.cc:161] P a2b1a692435241ee988e0bd1661cd4c3: Beginning new tablet copy session on tablet 126a163a89c448be89ccb9490e9f0168 from peer 88935ab835544e9889d05a6587d8cf21 at {username='slave'} at 127.0.0.1:33652: session id = 88935ab835544e9889d05a6587d8cf21-126a163a89c448be89ccb9490e9f0168
I20250109 08:57:52.618821 22527 raft_consensus.cc:357] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:57:52.619683 22527 raft_consensus.cc:738] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 88935ab835544e9889d05a6587d8cf21, State: Initialized, Role: LEARNER
I20250109 08:57:52.620296 22527 consensus_queue.cc:260] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P 88935ab835544e9889d05a6587d8cf21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:57:52.623837 22527 ts_tablet_manager.cc:1428] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P 88935ab835544e9889d05a6587d8cf21: Time spent starting tablet: real 0.011s user 0.005s sys 0.000s
I20250109 08:57:52.625715 21599 tablet_copy_source_session.cc:215] T 126a163a89c448be89ccb9490e9f0168 P a2b1a692435241ee988e0bd1661cd4c3: Tablet Copy: opened 0 blocks and 1 log segments
I20250109 08:57:52.626641 21673 tablet_copy_service.cc:342] P 787ddcc824c2490a808fc5a53452dc5e: Request end of tablet copy session 88935ab835544e9889d05a6587d8cf21-b54d6381a2cc41dfb59b1f6b99c0c5d4 received from {username='slave'} at 127.0.0.1:52376
I20250109 08:57:52.627068 21673 tablet_copy_service.cc:434] P 787ddcc824c2490a808fc5a53452dc5e: ending tablet copy session 88935ab835544e9889d05a6587d8cf21-b54d6381a2cc41dfb59b1f6b99c0c5d4 on tablet b54d6381a2cc41dfb59b1f6b99c0c5d4 with peer 88935ab835544e9889d05a6587d8cf21
I20250109 08:57:52.629410 22529 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 126a163a89c448be89ccb9490e9f0168. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:52.642410 22529 tablet_copy_client.cc:806] T 126a163a89c448be89ccb9490e9f0168 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Starting download of 0 data blocks...
I20250109 08:57:52.643033 22529 tablet_copy_client.cc:670] T 126a163a89c448be89ccb9490e9f0168 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Starting download of 1 WAL segments...
I20250109 08:57:52.646247 22529 tablet_copy_client.cc:538] T 126a163a89c448be89ccb9490e9f0168 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250109 08:57:52.653743 22527 ts_tablet_manager.cc:927] T 2935d69446cd458e92046d2b2f1b74ab P 88935ab835544e9889d05a6587d8cf21: Initiating tablet copy from peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725)
I20250109 08:57:52.654680 22529 tablet_bootstrap.cc:492] T 126a163a89c448be89ccb9490e9f0168 P 88935ab835544e9889d05a6587d8cf21: Bootstrap starting.
I20250109 08:57:52.658571 22527 tablet_copy_client.cc:323] T 2935d69446cd458e92046d2b2f1b74ab P 88935ab835544e9889d05a6587d8cf21: tablet copy: Beginning tablet copy session from remote peer at address 127.19.226.193:42725
I20250109 08:57:52.660013 21524 tablet_copy_service.cc:140] P 8bf6e26e98b14566899578a990b27d47: Received BeginTabletCopySession request for tablet 2935d69446cd458e92046d2b2f1b74ab from peer 88935ab835544e9889d05a6587d8cf21 ({username='slave'} at 127.0.0.1:41812)
I20250109 08:57:52.660553 21524 tablet_copy_service.cc:161] P 8bf6e26e98b14566899578a990b27d47: Beginning new tablet copy session on tablet 2935d69446cd458e92046d2b2f1b74ab from peer 88935ab835544e9889d05a6587d8cf21 at {username='slave'} at 127.0.0.1:41812: session id = 88935ab835544e9889d05a6587d8cf21-2935d69446cd458e92046d2b2f1b74ab
I20250109 08:57:52.666872 21524 tablet_copy_source_session.cc:215] T 2935d69446cd458e92046d2b2f1b74ab P 8bf6e26e98b14566899578a990b27d47: Tablet Copy: opened 0 blocks and 1 log segments
I20250109 08:57:52.669927 22527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2935d69446cd458e92046d2b2f1b74ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:52.673434 22529 tablet_bootstrap.cc:492] T 126a163a89c448be89ccb9490e9f0168 P 88935ab835544e9889d05a6587d8cf21: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250109 08:57:52.674230 22529 tablet_bootstrap.cc:492] T 126a163a89c448be89ccb9490e9f0168 P 88935ab835544e9889d05a6587d8cf21: Bootstrap complete.
I20250109 08:57:52.674916 22529 ts_tablet_manager.cc:1397] T 126a163a89c448be89ccb9490e9f0168 P 88935ab835544e9889d05a6587d8cf21: Time spent bootstrapping tablet: real 0.020s user 0.017s sys 0.003s
I20250109 08:57:52.678121 22529 raft_consensus.cc:357] T 126a163a89c448be89ccb9490e9f0168 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:57:52.679057 22529 raft_consensus.cc:738] T 126a163a89c448be89ccb9490e9f0168 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 88935ab835544e9889d05a6587d8cf21, State: Initialized, Role: LEARNER
I20250109 08:57:52.679725 22529 consensus_queue.cc:260] T 126a163a89c448be89ccb9490e9f0168 P 88935ab835544e9889d05a6587d8cf21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:57:52.682265 22529 ts_tablet_manager.cc:1428] T 126a163a89c448be89ccb9490e9f0168 P 88935ab835544e9889d05a6587d8cf21: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20250109 08:57:52.683815 22527 tablet_copy_client.cc:806] T 2935d69446cd458e92046d2b2f1b74ab P 88935ab835544e9889d05a6587d8cf21: tablet copy: Starting download of 0 data blocks...
I20250109 08:57:52.684039 21599 tablet_copy_service.cc:342] P a2b1a692435241ee988e0bd1661cd4c3: Request end of tablet copy session 88935ab835544e9889d05a6587d8cf21-126a163a89c448be89ccb9490e9f0168 received from {username='slave'} at 127.0.0.1:33652
I20250109 08:57:52.684645 21599 tablet_copy_service.cc:434] P a2b1a692435241ee988e0bd1661cd4c3: ending tablet copy session 88935ab835544e9889d05a6587d8cf21-126a163a89c448be89ccb9490e9f0168 on tablet 126a163a89c448be89ccb9490e9f0168 with peer 88935ab835544e9889d05a6587d8cf21
I20250109 08:57:52.685032 22535 ts_tablet_manager.cc:927] T 239ca74a4c0a455a9db2652d7debb989 P 88935ab835544e9889d05a6587d8cf21: Initiating tablet copy from peer 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:52.688684 22527 tablet_copy_client.cc:670] T 2935d69446cd458e92046d2b2f1b74ab P 88935ab835544e9889d05a6587d8cf21: tablet copy: Starting download of 1 WAL segments...
I20250109 08:57:52.689455 22535 tablet_copy_client.cc:323] T 239ca74a4c0a455a9db2652d7debb989 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Beginning tablet copy session from remote peer at address 127.19.226.195:35731
I20250109 08:57:52.690970 21673 tablet_copy_service.cc:140] P 787ddcc824c2490a808fc5a53452dc5e: Received BeginTabletCopySession request for tablet 239ca74a4c0a455a9db2652d7debb989 from peer 88935ab835544e9889d05a6587d8cf21 ({username='slave'} at 127.0.0.1:52376)
I20250109 08:57:52.691471 21673 tablet_copy_service.cc:161] P 787ddcc824c2490a808fc5a53452dc5e: Beginning new tablet copy session on tablet 239ca74a4c0a455a9db2652d7debb989 from peer 88935ab835544e9889d05a6587d8cf21 at {username='slave'} at 127.0.0.1:52376: session id = 88935ab835544e9889d05a6587d8cf21-239ca74a4c0a455a9db2652d7debb989
I20250109 08:57:52.692162 22527 tablet_copy_client.cc:538] T 2935d69446cd458e92046d2b2f1b74ab P 88935ab835544e9889d05a6587d8cf21: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250109 08:57:52.700819 22527 tablet_bootstrap.cc:492] T 2935d69446cd458e92046d2b2f1b74ab P 88935ab835544e9889d05a6587d8cf21: Bootstrap starting.
I20250109 08:57:52.703299 21673 tablet_copy_source_session.cc:215] T 239ca74a4c0a455a9db2652d7debb989 P 787ddcc824c2490a808fc5a53452dc5e: Tablet Copy: opened 0 blocks and 1 log segments
I20250109 08:57:52.706148 22535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 239ca74a4c0a455a9db2652d7debb989. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:52.724992 22527 tablet_bootstrap.cc:492] T 2935d69446cd458e92046d2b2f1b74ab P 88935ab835544e9889d05a6587d8cf21: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250109 08:57:52.725999 22527 tablet_bootstrap.cc:492] T 2935d69446cd458e92046d2b2f1b74ab P 88935ab835544e9889d05a6587d8cf21: Bootstrap complete.
I20250109 08:57:52.726749 22527 ts_tablet_manager.cc:1397] T 2935d69446cd458e92046d2b2f1b74ab P 88935ab835544e9889d05a6587d8cf21: Time spent bootstrapping tablet: real 0.026s user 0.017s sys 0.005s
I20250109 08:57:52.727267 22535 tablet_copy_client.cc:806] T 239ca74a4c0a455a9db2652d7debb989 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Starting download of 0 data blocks...
I20250109 08:57:52.727838 22535 tablet_copy_client.cc:670] T 239ca74a4c0a455a9db2652d7debb989 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Starting download of 1 WAL segments...
I20250109 08:57:52.729557 22527 raft_consensus.cc:357] T 2935d69446cd458e92046d2b2f1b74ab P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:57:52.730316 22527 raft_consensus.cc:738] T 2935d69446cd458e92046d2b2f1b74ab P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 88935ab835544e9889d05a6587d8cf21, State: Initialized, Role: LEARNER
I20250109 08:57:52.730811 22527 consensus_queue.cc:260] T 2935d69446cd458e92046d2b2f1b74ab P 88935ab835544e9889d05a6587d8cf21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:57:52.733170 22527 ts_tablet_manager.cc:1428] T 2935d69446cd458e92046d2b2f1b74ab P 88935ab835544e9889d05a6587d8cf21: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20250109 08:57:52.734916 21524 tablet_copy_service.cc:342] P 8bf6e26e98b14566899578a990b27d47: Request end of tablet copy session 88935ab835544e9889d05a6587d8cf21-2935d69446cd458e92046d2b2f1b74ab received from {username='slave'} at 127.0.0.1:41812
I20250109 08:57:52.731763 22535 tablet_copy_client.cc:538] T 239ca74a4c0a455a9db2652d7debb989 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250109 08:57:52.735320 21524 tablet_copy_service.cc:434] P 8bf6e26e98b14566899578a990b27d47: ending tablet copy session 88935ab835544e9889d05a6587d8cf21-2935d69446cd458e92046d2b2f1b74ab on tablet 2935d69446cd458e92046d2b2f1b74ab with peer 88935ab835544e9889d05a6587d8cf21
I20250109 08:57:52.749148 22535 tablet_bootstrap.cc:492] T 239ca74a4c0a455a9db2652d7debb989 P 88935ab835544e9889d05a6587d8cf21: Bootstrap starting.
I20250109 08:57:52.766907 22535 tablet_bootstrap.cc:492] T 239ca74a4c0a455a9db2652d7debb989 P 88935ab835544e9889d05a6587d8cf21: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250109 08:57:52.767781 22535 tablet_bootstrap.cc:492] T 239ca74a4c0a455a9db2652d7debb989 P 88935ab835544e9889d05a6587d8cf21: Bootstrap complete.
I20250109 08:57:52.768509 22535 ts_tablet_manager.cc:1397] T 239ca74a4c0a455a9db2652d7debb989 P 88935ab835544e9889d05a6587d8cf21: Time spent bootstrapping tablet: real 0.020s user 0.018s sys 0.001s
I20250109 08:57:52.772630 22535 raft_consensus.cc:357] T 239ca74a4c0a455a9db2652d7debb989 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:57:52.773316 22535 raft_consensus.cc:738] T 239ca74a4c0a455a9db2652d7debb989 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 88935ab835544e9889d05a6587d8cf21, State: Initialized, Role: LEARNER
I20250109 08:57:52.773789 22535 consensus_queue.cc:260] T 239ca74a4c0a455a9db2652d7debb989 P 88935ab835544e9889d05a6587d8cf21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:57:52.775693 22535 ts_tablet_manager.cc:1428] T 239ca74a4c0a455a9db2652d7debb989 P 88935ab835544e9889d05a6587d8cf21: Time spent starting tablet: real 0.006s user 0.000s sys 0.002s
I20250109 08:57:52.777429 21673 tablet_copy_service.cc:342] P 787ddcc824c2490a808fc5a53452dc5e: Request end of tablet copy session 88935ab835544e9889d05a6587d8cf21-239ca74a4c0a455a9db2652d7debb989 received from {username='slave'} at 127.0.0.1:52376
I20250109 08:57:52.777861 21673 tablet_copy_service.cc:434] P 787ddcc824c2490a808fc5a53452dc5e: ending tablet copy session 88935ab835544e9889d05a6587d8cf21-239ca74a4c0a455a9db2652d7debb989 on tablet 239ca74a4c0a455a9db2652d7debb989 with peer 88935ab835544e9889d05a6587d8cf21
I20250109 08:57:53.029446 22176 raft_consensus.cc:1212] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20250109 08:57:53.030952 22177 raft_consensus.cc:1212] T 126a163a89c448be89ccb9490e9f0168 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20250109 08:57:53.039913 22177 raft_consensus.cc:1212] T 2935d69446cd458e92046d2b2f1b74ab P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20250109 08:57:53.126199 22177 raft_consensus.cc:1212] T 239ca74a4c0a455a9db2652d7debb989 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20250109 08:57:53.520511 22507 raft_consensus.cc:1059] T 126a163a89c448be89ccb9490e9f0168 P a2b1a692435241ee988e0bd1661cd4c3: attempting to promote NON_VOTER 88935ab835544e9889d05a6587d8cf21 to VOTER
I20250109 08:57:53.522683 22507 consensus_queue.cc:237] T 126a163a89c448be89ccb9490e9f0168 P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:57:53.529413 21656 raft_consensus.cc:1270] T 126a163a89c448be89ccb9490e9f0168 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Refusing update from remote peer a2b1a692435241ee988e0bd1661cd4c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:57:53.531270 21512 raft_consensus.cc:1270] T 126a163a89c448be89ccb9490e9f0168 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Refusing update from remote peer a2b1a692435241ee988e0bd1661cd4c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:57:53.532065 22541 consensus_queue.cc:1035] T 126a163a89c448be89ccb9490e9f0168 P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20250109 08:57:53.533061 22507 consensus_queue.cc:1035] T 126a163a89c448be89ccb9490e9f0168 P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:57:53.533782 22177 raft_consensus.cc:1270] T 126a163a89c448be89ccb9490e9f0168 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Refusing update from remote peer a2b1a692435241ee988e0bd1661cd4c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:57:53.534992 22443 consensus_queue.cc:1035] T 126a163a89c448be89ccb9490e9f0168 P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:57:53.550138 22443 raft_consensus.cc:2949] T 126a163a89c448be89ccb9490e9f0168 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:53.551890 22480 raft_consensus.cc:1059] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P 787ddcc824c2490a808fc5a53452dc5e: attempting to promote NON_VOTER 88935ab835544e9889d05a6587d8cf21 to VOTER
I20250109 08:57:53.554905 22480 consensus_queue.cc:237] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:57:53.554668 22177 raft_consensus.cc:2949] T 126a163a89c448be89ccb9490e9f0168 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:53.560029 21663 raft_consensus.cc:2949] T 126a163a89c448be89ccb9490e9f0168 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:53.561733 21512 raft_consensus.cc:2949] T 126a163a89c448be89ccb9490e9f0168 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:53.567770 21411 catalog_manager.cc:5526] T 126a163a89c448be89ccb9490e9f0168 P a2b1a692435241ee988e0bd1661cd4c3 reported cstate change: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a2b1a692435241ee988e0bd1661cd4c3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250109 08:57:53.573431 22539 raft_consensus.cc:1059] T 239ca74a4c0a455a9db2652d7debb989 P 787ddcc824c2490a808fc5a53452dc5e: attempting to promote NON_VOTER 88935ab835544e9889d05a6587d8cf21 to VOTER
I20250109 08:57:53.575798 22539 consensus_queue.cc:237] T 239ca74a4c0a455a9db2652d7debb989 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:57:53.580859 21589 raft_consensus.cc:1270] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:57:53.582674 22485 consensus_queue.cc:1035] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:57:53.586041 22177 raft_consensus.cc:1270] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:57:53.590229 22485 consensus_queue.cc:1035] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:57:53.590741 21509 raft_consensus.cc:1270] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:57:53.593961 22485 consensus_queue.cc:1035] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.009s
I20250109 08:57:53.607415 22522 raft_consensus.cc:1059] T 2935d69446cd458e92046d2b2f1b74ab P 8bf6e26e98b14566899578a990b27d47: attempting to promote NON_VOTER 88935ab835544e9889d05a6587d8cf21 to VOTER
I20250109 08:57:53.612778 22522 consensus_queue.cc:237] T 2935d69446cd458e92046d2b2f1b74ab P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:57:53.620213 21588 raft_consensus.cc:1270] T 239ca74a4c0a455a9db2652d7debb989 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:57:53.620709 21512 raft_consensus.cc:1270] T 239ca74a4c0a455a9db2652d7debb989 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:57:53.621767 22539 consensus_queue.cc:1035] T 239ca74a4c0a455a9db2652d7debb989 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20250109 08:57:53.622787 22480 consensus_queue.cc:1035] T 239ca74a4c0a455a9db2652d7debb989 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:57:53.623669 22176 raft_consensus.cc:1270] T 239ca74a4c0a455a9db2652d7debb989 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:57:53.625553 22551 consensus_queue.cc:1035] T 239ca74a4c0a455a9db2652d7debb989 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:57:53.627082 21589 raft_consensus.cc:2949] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:53.642239 22485 raft_consensus.cc:2949] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:53.645598 21509 raft_consensus.cc:2949] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:53.661437 21656 raft_consensus.cc:1270] T 2935d69446cd458e92046d2b2f1b74ab P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Refusing update from remote peer 8bf6e26e98b14566899578a990b27d47: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:57:53.665192 22410 consensus_queue.cc:1035] T 2935d69446cd458e92046d2b2f1b74ab P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:57:53.666610 21588 raft_consensus.cc:1270] T 2935d69446cd458e92046d2b2f1b74ab P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Refusing update from remote peer 8bf6e26e98b14566899578a990b27d47: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:57:53.666611 22176 raft_consensus.cc:1270] T 2935d69446cd458e92046d2b2f1b74ab P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Refusing update from remote peer 8bf6e26e98b14566899578a990b27d47: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:57:53.668448 22410 consensus_queue.cc:1035] T 2935d69446cd458e92046d2b2f1b74ab P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:57:53.669872 22559 consensus_queue.cc:1035] T 2935d69446cd458e92046d2b2f1b74ab P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:57:53.659600 22480 raft_consensus.cc:2949] T 239ca74a4c0a455a9db2652d7debb989 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:53.687327 21589 raft_consensus.cc:2949] T 239ca74a4c0a455a9db2652d7debb989 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:53.693635 22177 raft_consensus.cc:2949] T 239ca74a4c0a455a9db2652d7debb989 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:53.710052 22560 raft_consensus.cc:2949] T 2935d69446cd458e92046d2b2f1b74ab P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:53.723110 21409 catalog_manager.cc:5526] T 239ca74a4c0a455a9db2652d7debb989 P 88935ab835544e9889d05a6587d8cf21 reported cstate change: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "787ddcc824c2490a808fc5a53452dc5e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:53.729920 21412 catalog_manager.cc:5526] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P a2b1a692435241ee988e0bd1661cd4c3 reported cstate change: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "787ddcc824c2490a808fc5a53452dc5e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:53.825714 21583 consensus_queue.cc:237] T 126a163a89c448be89ccb9490e9f0168 P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
W20250109 08:57:53.833698 22556 log.cc:926] Time spent T 239ca74a4c0a455a9db2652d7debb989 P 8bf6e26e98b14566899578a990b27d47: Append to log took a long time: real 0.133s user 0.000s sys 0.001s
I20250109 08:57:53.840723 22177 raft_consensus.cc:2949] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:53.848531 21588 raft_consensus.cc:2949] T 2935d69446cd458e92046d2b2f1b74ab P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:53.861668 21511 raft_consensus.cc:2949] T 239ca74a4c0a455a9db2652d7debb989 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:53.877628 21656 raft_consensus.cc:2949] T 2935d69446cd458e92046d2b2f1b74ab P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:53.891197 21412 catalog_manager.cc:5526] T 2935d69446cd458e92046d2b2f1b74ab P 8bf6e26e98b14566899578a990b27d47 reported cstate change: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8bf6e26e98b14566899578a990b27d47" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20250109 08:57:53.897917 22176 raft_consensus.cc:1270] T 126a163a89c448be89ccb9490e9f0168 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Refusing update from remote peer a2b1a692435241ee988e0bd1661cd4c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250109 08:57:53.900348 21513 raft_consensus.cc:1270] T 126a163a89c448be89ccb9490e9f0168 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Refusing update from remote peer a2b1a692435241ee988e0bd1661cd4c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250109 08:57:53.900312 22177 raft_consensus.cc:2949] T 2935d69446cd458e92046d2b2f1b74ab P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:53.919890 22572 consensus_queue.cc:1035] T 126a163a89c448be89ccb9490e9f0168 P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250109 08:57:53.925076 22570 consensus_queue.cc:1035] T 126a163a89c448be89ccb9490e9f0168 P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250109 08:57:53.930608 21660 consensus_queue.cc:237] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:57:53.940145 21513 raft_consensus.cc:1270] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250109 08:57:53.941095 22176 raft_consensus.cc:1270] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250109 08:57:53.942018 22485 consensus_queue.cc:1035] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250109 08:57:53.942840 21511 consensus_queue.cc:237] T 2935d69446cd458e92046d2b2f1b74ab P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:57:53.948923 22551 consensus_queue.cc:1035] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250109 08:57:53.948402 22176 raft_consensus.cc:2949] T 126a163a89c448be89ccb9490e9f0168 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:53.950003 21509 raft_consensus.cc:2949] T 126a163a89c448be89ccb9490e9f0168 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:53.957656 22178 raft_consensus.cc:1270] T 2935d69446cd458e92046d2b2f1b74ab P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Refusing update from remote peer 8bf6e26e98b14566899578a990b27d47: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250109 08:57:53.958725 22480 raft_consensus.cc:2949] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:53.946662 22568 raft_consensus.cc:2949] T 126a163a89c448be89ccb9490e9f0168 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:53.963394 21511 raft_consensus.cc:2949] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:53.968247 21396 catalog_manager.cc:5039] ChangeConfig:REMOVE_PEER RPC for tablet b54d6381a2cc41dfb59b1f6b99c0c5d4 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250109 08:57:53.971716 22569 consensus_queue.cc:1035] T 2935d69446cd458e92046d2b2f1b74ab P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20250109 08:57:53.973755 22178 raft_consensus.cc:2949] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:53.986778 21410 catalog_manager.cc:5526] T 126a163a89c448be89ccb9490e9f0168 P 8bf6e26e98b14566899578a990b27d47 reported cstate change: config changed from index 3 to 4, VOTER 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195) evicted. New cstate: current_term: 1 leader_uuid: "a2b1a692435241ee988e0bd1661cd4c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:53.975939 21588 raft_consensus.cc:1270] T 2935d69446cd458e92046d2b2f1b74ab P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Refusing update from remote peer 8bf6e26e98b14566899578a990b27d47: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250109 08:57:53.979629 21412 catalog_manager.cc:5526] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P 787ddcc824c2490a808fc5a53452dc5e reported cstate change: config changed from index 3 to 4, VOTER a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194) evicted. New cstate: current_term: 1 leader_uuid: "787ddcc824c2490a808fc5a53452dc5e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250109 08:57:53.987752 21660 consensus_queue.cc:237] T 239ca74a4c0a455a9db2652d7debb989 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:57:53.999778 21397 catalog_manager.cc:5039] ChangeConfig:REMOVE_PEER RPC for tablet 126a163a89c448be89ccb9490e9f0168 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250109 08:57:54.003329 22569 consensus_queue.cc:1035] T 2935d69446cd458e92046d2b2f1b74ab P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250109 08:57:54.004678 22560 raft_consensus.cc:2949] T 2935d69446cd458e92046d2b2f1b74ab P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:54.009815 21511 raft_consensus.cc:1270] T 239ca74a4c0a455a9db2652d7debb989 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250109 08:57:54.010742 21570 tablet_service.cc:1514] Processing DeleteTablet for tablet b54d6381a2cc41dfb59b1f6b99c0c5d4 with delete_type TABLET_DATA_TOMBSTONED (TS a2b1a692435241ee988e0bd1661cd4c3 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:34732
I20250109 08:57:54.011983 22547 consensus_queue.cc:1035] T 239ca74a4c0a455a9db2652d7debb989 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20250109 08:57:54.018914 22178 raft_consensus.cc:1270] T 239ca74a4c0a455a9db2652d7debb989 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250109 08:57:54.018949 22175 raft_consensus.cc:2949] T 2935d69446cd458e92046d2b2f1b74ab P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:54.023169 21396 catalog_manager.cc:5039] ChangeConfig:REMOVE_PEER RPC for tablet 2935d69446cd458e92046d2b2f1b74ab with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250109 08:57:54.024637 21653 tablet_service.cc:1514] Processing DeleteTablet for tablet 126a163a89c448be89ccb9490e9f0168 with delete_type TABLET_DATA_TOMBSTONED (TS 787ddcc824c2490a808fc5a53452dc5e not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:58436
I20250109 08:57:54.026592 22539 consensus_queue.cc:1035] T 239ca74a4c0a455a9db2652d7debb989 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250109 08:57:54.029455 21513 raft_consensus.cc:2949] T 239ca74a4c0a455a9db2652d7debb989 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:54.031148 21588 raft_consensus.cc:2949] T 2935d69446cd458e92046d2b2f1b74ab P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:54.035087 22176 raft_consensus.cc:2949] T 239ca74a4c0a455a9db2652d7debb989 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:54.025115 22547 raft_consensus.cc:2949] T 239ca74a4c0a455a9db2652d7debb989 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:54.042500 21412 catalog_manager.cc:5526] T 2935d69446cd458e92046d2b2f1b74ab P 8bf6e26e98b14566899578a990b27d47 reported cstate change: config changed from index 3 to 4, VOTER 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195) evicted. New cstate: current_term: 1 leader_uuid: "8bf6e26e98b14566899578a990b27d47" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250109 08:57:54.051987 22573 tablet_replica.cc:331] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P a2b1a692435241ee988e0bd1661cd4c3: stopping tablet replica
I20250109 08:57:54.061287 21649 tablet_service.cc:1514] Processing DeleteTablet for tablet 2935d69446cd458e92046d2b2f1b74ab with delete_type TABLET_DATA_TOMBSTONED (TS 787ddcc824c2490a808fc5a53452dc5e not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:58436
I20250109 08:57:54.061952 22574 tablet_replica.cc:331] T 126a163a89c448be89ccb9490e9f0168 P 787ddcc824c2490a808fc5a53452dc5e: stopping tablet replica
I20250109 08:57:54.062973 22573 raft_consensus.cc:2238] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:57:54.063412 22574 raft_consensus.cc:2238] T 126a163a89c448be89ccb9490e9f0168 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:57:54.065274 21396 catalog_manager.cc:5039] ChangeConfig:REMOVE_PEER RPC for tablet 239ca74a4c0a455a9db2652d7debb989 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250109 08:57:54.065496 22574 raft_consensus.cc:2267] T 126a163a89c448be89ccb9490e9f0168 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:57:54.080488 21411 catalog_manager.cc:5526] T 239ca74a4c0a455a9db2652d7debb989 P 88935ab835544e9889d05a6587d8cf21 reported cstate change: config changed from index 3 to 4, VOTER a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194) evicted. New cstate: current_term: 1 leader_uuid: "787ddcc824c2490a808fc5a53452dc5e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:54.082772 22574 ts_tablet_manager.cc:1905] T 126a163a89c448be89ccb9490e9f0168 P 787ddcc824c2490a808fc5a53452dc5e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250109 08:57:54.084519 22573 raft_consensus.cc:2267] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:57:54.088328 22573 ts_tablet_manager.cc:1905] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P a2b1a692435241ee988e0bd1661cd4c3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250109 08:57:54.100505 21570 tablet_service.cc:1514] Processing DeleteTablet for tablet 239ca74a4c0a455a9db2652d7debb989 with delete_type TABLET_DATA_TOMBSTONED (TS a2b1a692435241ee988e0bd1661cd4c3 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:34732
I20250109 08:57:54.105940 22573 ts_tablet_manager.cc:1918] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P a2b1a692435241ee988e0bd1661cd4c3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20250109 08:57:54.106470 22573 log.cc:1198] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P a2b1a692435241ee988e0bd1661cd4c3: Deleting WAL directory at /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736412987538469-20363-0/minicluster-data/ts-1-root/wals/b54d6381a2cc41dfb59b1f6b99c0c5d4
I20250109 08:57:54.107780 21397 catalog_manager.cc:4872] TS a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): tablet b54d6381a2cc41dfb59b1f6b99c0c5d4 (table test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]) successfully deleted
I20250109 08:57:54.107834 22573 tablet_replica.cc:331] T 239ca74a4c0a455a9db2652d7debb989 P a2b1a692435241ee988e0bd1661cd4c3: stopping tablet replica
I20250109 08:57:54.109601 22573 raft_consensus.cc:2238] T 239ca74a4c0a455a9db2652d7debb989 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:57:54.110301 22573 raft_consensus.cc:2267] T 239ca74a4c0a455a9db2652d7debb989 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:57:54.111040 22574 ts_tablet_manager.cc:1918] T 126a163a89c448be89ccb9490e9f0168 P 787ddcc824c2490a808fc5a53452dc5e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20250109 08:57:54.111476 22574 log.cc:1198] T 126a163a89c448be89ccb9490e9f0168 P 787ddcc824c2490a808fc5a53452dc5e: Deleting WAL directory at /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736412987538469-20363-0/minicluster-data/ts-2-root/wals/126a163a89c448be89ccb9490e9f0168
I20250109 08:57:54.113049 21396 catalog_manager.cc:4872] TS 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731): tablet 126a163a89c448be89ccb9490e9f0168 (table test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]) successfully deleted
I20250109 08:57:54.113189 22574 tablet_replica.cc:331] T 2935d69446cd458e92046d2b2f1b74ab P 787ddcc824c2490a808fc5a53452dc5e: stopping tablet replica
I20250109 08:57:54.113322 22573 ts_tablet_manager.cc:1905] T 239ca74a4c0a455a9db2652d7debb989 P a2b1a692435241ee988e0bd1661cd4c3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250109 08:57:54.114395 22574 raft_consensus.cc:2238] T 2935d69446cd458e92046d2b2f1b74ab P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:57:54.115088 22574 raft_consensus.cc:2267] T 2935d69446cd458e92046d2b2f1b74ab P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:57:54.119777 22574 ts_tablet_manager.cc:1905] T 2935d69446cd458e92046d2b2f1b74ab P 787ddcc824c2490a808fc5a53452dc5e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250109 08:57:54.132251 22574 ts_tablet_manager.cc:1918] T 2935d69446cd458e92046d2b2f1b74ab P 787ddcc824c2490a808fc5a53452dc5e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20250109 08:57:54.132583 22574 log.cc:1198] T 2935d69446cd458e92046d2b2f1b74ab P 787ddcc824c2490a808fc5a53452dc5e: Deleting WAL directory at /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736412987538469-20363-0/minicluster-data/ts-2-root/wals/2935d69446cd458e92046d2b2f1b74ab
I20250109 08:57:54.133528 22573 ts_tablet_manager.cc:1918] T 239ca74a4c0a455a9db2652d7debb989 P a2b1a692435241ee988e0bd1661cd4c3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20250109 08:57:54.133998 22573 log.cc:1198] T 239ca74a4c0a455a9db2652d7debb989 P a2b1a692435241ee988e0bd1661cd4c3: Deleting WAL directory at /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736412987538469-20363-0/minicluster-data/ts-1-root/wals/239ca74a4c0a455a9db2652d7debb989
I20250109 08:57:54.135228 21396 catalog_manager.cc:4872] TS 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731): tablet 2935d69446cd458e92046d2b2f1b74ab (table test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]) successfully deleted
I20250109 08:57:54.135303 21397 catalog_manager.cc:4872] TS a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): tablet 239ca74a4c0a455a9db2652d7debb989 (table test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]) successfully deleted
I20250109 08:57:55.146085 21660 consensus_queue.cc:237] T eb4516b9a16041c2adf34be520768f3f P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:57:55.151368 21583 raft_consensus.cc:1270] T eb4516b9a16041c2adf34be520768f3f P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250109 08:57:55.151739 21509 raft_consensus.cc:1270] T eb4516b9a16041c2adf34be520768f3f P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250109 08:57:55.152498 22480 consensus_queue.cc:1035] T eb4516b9a16041c2adf34be520768f3f P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20250109 08:57:55.153728 22576 consensus_queue.cc:1035] T eb4516b9a16041c2adf34be520768f3f P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20250109 08:57:55.157260 21627 consensus_peers.cc:487] T eb4516b9a16041c2adf34be520768f3f P 787ddcc824c2490a808fc5a53452dc5e -> Peer 88935ab835544e9889d05a6587d8cf21 (127.19.226.196:44363): Couldn't send request to peer 88935ab835544e9889d05a6587d8cf21. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: eb4516b9a16041c2adf34be520768f3f. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:57:55.160964 22480 raft_consensus.cc:2949] T eb4516b9a16041c2adf34be520768f3f P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:57:55.162359 21509 raft_consensus.cc:2949] T eb4516b9a16041c2adf34be520768f3f P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:57:55.171274 21583 raft_consensus.cc:2949] T eb4516b9a16041c2adf34be520768f3f P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:57:55.173434 21660 consensus_queue.cc:237] T df9715570a6844d9a2686307db16eff1 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:57:55.180994 21409 catalog_manager.cc:5526] T eb4516b9a16041c2adf34be520768f3f P 8bf6e26e98b14566899578a990b27d47 reported cstate change: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New cstate: current_term: 1 leader_uuid: "787ddcc824c2490a808fc5a53452dc5e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:57:55.193034 21583 raft_consensus.cc:1270] T df9715570a6844d9a2686307db16eff1 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250109 08:57:55.194296 22577 consensus_queue.cc:1035] T df9715570a6844d9a2686307db16eff1 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:57:55.194669 21509 raft_consensus.cc:1270] T df9715570a6844d9a2686307db16eff1 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20250109 08:57:55.195708 21627 consensus_peers.cc:487] T df9715570a6844d9a2686307db16eff1 P 787ddcc824c2490a808fc5a53452dc5e -> Peer 88935ab835544e9889d05a6587d8cf21 (127.19.226.196:44363): Couldn't send request to peer 88935ab835544e9889d05a6587d8cf21. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: df9715570a6844d9a2686307db16eff1. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:57:55.195973 22480 consensus_queue.cc:1035] T df9715570a6844d9a2686307db16eff1 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:57:55.202111 22577 raft_consensus.cc:2949] T df9715570a6844d9a2686307db16eff1 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:57:55.203650 21583 raft_consensus.cc:2949] T df9715570a6844d9a2686307db16eff1 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:57:55.211213 21509 raft_consensus.cc:2949] T df9715570a6844d9a2686307db16eff1 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:57:55.226188 21660 consensus_queue.cc:237] T 81d74e7d4472497aac5463a4b47e4fb9 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:57:55.228813 21409 catalog_manager.cc:5526] T df9715570a6844d9a2686307db16eff1 P 8bf6e26e98b14566899578a990b27d47 reported cstate change: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New cstate: current_term: 1 leader_uuid: "787ddcc824c2490a808fc5a53452dc5e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:57:55.236063 21583 raft_consensus.cc:1270] T 81d74e7d4472497aac5463a4b47e4fb9 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250109 08:57:55.237685 21509 raft_consensus.cc:1270] T 81d74e7d4472497aac5463a4b47e4fb9 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250109 08:57:55.237820 22480 consensus_queue.cc:1035] T 81d74e7d4472497aac5463a4b47e4fb9 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:57:55.240523 22577 consensus_queue.cc:1035] T 81d74e7d4472497aac5463a4b47e4fb9 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20250109 08:57:55.242662 21627 consensus_peers.cc:487] T 81d74e7d4472497aac5463a4b47e4fb9 P 787ddcc824c2490a808fc5a53452dc5e -> Peer 88935ab835544e9889d05a6587d8cf21 (127.19.226.196:44363): Couldn't send request to peer 88935ab835544e9889d05a6587d8cf21. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 81d74e7d4472497aac5463a4b47e4fb9. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:57:55.249361 22480 raft_consensus.cc:2949] T 81d74e7d4472497aac5463a4b47e4fb9 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:57:55.250967 21583 raft_consensus.cc:2949] T 81d74e7d4472497aac5463a4b47e4fb9 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:57:55.256343 21509 raft_consensus.cc:2949] T 81d74e7d4472497aac5463a4b47e4fb9 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:57:55.266309 21513 consensus_queue.cc:237] T d6f988a231a74f6fa112da9a527d4a0b P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:57:55.273708 21412 catalog_manager.cc:5526] T 81d74e7d4472497aac5463a4b47e4fb9 P a2b1a692435241ee988e0bd1661cd4c3 reported cstate change: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New cstate: current_term: 1 leader_uuid: "787ddcc824c2490a808fc5a53452dc5e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
W20250109 08:57:55.287657 21478 consensus_peers.cc:487] T d6f988a231a74f6fa112da9a527d4a0b P 8bf6e26e98b14566899578a990b27d47 -> Peer 88935ab835544e9889d05a6587d8cf21 (127.19.226.196:44363): Couldn't send request to peer 88935ab835544e9889d05a6587d8cf21. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d6f988a231a74f6fa112da9a527d4a0b. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:57:55.288558 21660 raft_consensus.cc:1270] T d6f988a231a74f6fa112da9a527d4a0b P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Refusing update from remote peer 8bf6e26e98b14566899578a990b27d47: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250109 08:57:55.289721 22575 consensus_queue.cc:1035] T d6f988a231a74f6fa112da9a527d4a0b P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:57:55.290282 21583 raft_consensus.cc:1270] T d6f988a231a74f6fa112da9a527d4a0b P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Refusing update from remote peer 8bf6e26e98b14566899578a990b27d47: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250109 08:57:55.291370 22569 consensus_queue.cc:1035] T d6f988a231a74f6fa112da9a527d4a0b P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:57:55.302444 22575 raft_consensus.cc:2949] T d6f988a231a74f6fa112da9a527d4a0b P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:57:55.304584 21660 raft_consensus.cc:2949] T d6f988a231a74f6fa112da9a527d4a0b P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:57:55.307293 21588 raft_consensus.cc:2949] T d6f988a231a74f6fa112da9a527d4a0b P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:57:55.318267 21409 catalog_manager.cc:5526] T d6f988a231a74f6fa112da9a527d4a0b P 8bf6e26e98b14566899578a990b27d47 reported cstate change: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New cstate: current_term: 1 leader_uuid: "8bf6e26e98b14566899578a990b27d47" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20250109 08:57:55.930292 22597 ts_tablet_manager.cc:927] T d6f988a231a74f6fa112da9a527d4a0b P 88935ab835544e9889d05a6587d8cf21: Initiating tablet copy from peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725)
I20250109 08:57:55.946465 22597 tablet_copy_client.cc:323] T d6f988a231a74f6fa112da9a527d4a0b P 88935ab835544e9889d05a6587d8cf21: tablet copy: Beginning tablet copy session from remote peer at address 127.19.226.193:42725
I20250109 08:57:55.956019 21524 tablet_copy_service.cc:140] P 8bf6e26e98b14566899578a990b27d47: Received BeginTabletCopySession request for tablet d6f988a231a74f6fa112da9a527d4a0b from peer 88935ab835544e9889d05a6587d8cf21 ({username='slave'} at 127.0.0.1:41812)
I20250109 08:57:55.956612 21524 tablet_copy_service.cc:161] P 8bf6e26e98b14566899578a990b27d47: Beginning new tablet copy session on tablet d6f988a231a74f6fa112da9a527d4a0b from peer 88935ab835544e9889d05a6587d8cf21 at {username='slave'} at 127.0.0.1:41812: session id = 88935ab835544e9889d05a6587d8cf21-d6f988a231a74f6fa112da9a527d4a0b
I20250109 08:57:55.975497 21524 tablet_copy_source_session.cc:215] T d6f988a231a74f6fa112da9a527d4a0b P 8bf6e26e98b14566899578a990b27d47: Tablet Copy: opened 0 blocks and 1 log segments
I20250109 08:57:55.982314 22597 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6f988a231a74f6fa112da9a527d4a0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:56.015662 22597 tablet_copy_client.cc:806] T d6f988a231a74f6fa112da9a527d4a0b P 88935ab835544e9889d05a6587d8cf21: tablet copy: Starting download of 0 data blocks...
I20250109 08:57:56.016396 22597 tablet_copy_client.cc:670] T d6f988a231a74f6fa112da9a527d4a0b P 88935ab835544e9889d05a6587d8cf21: tablet copy: Starting download of 1 WAL segments...
W20250109 08:57:56.025977 21552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.226.193:42725, user_credentials={real_user=slave}} blocked reactor thread for 103913us
I20250109 08:57:56.033463 22597 tablet_copy_client.cc:538] T d6f988a231a74f6fa112da9a527d4a0b P 88935ab835544e9889d05a6587d8cf21: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250109 08:57:56.051510 22597 tablet_bootstrap.cc:492] T d6f988a231a74f6fa112da9a527d4a0b P 88935ab835544e9889d05a6587d8cf21: Bootstrap starting.
I20250109 08:57:56.091125 22614 ts_tablet_manager.cc:927] T 81d74e7d4472497aac5463a4b47e4fb9 P 88935ab835544e9889d05a6587d8cf21: Initiating tablet copy from peer 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:56.097276 22615 ts_tablet_manager.cc:927] T eb4516b9a16041c2adf34be520768f3f P 88935ab835544e9889d05a6587d8cf21: Initiating tablet copy from peer 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:56.100997 22613 ts_tablet_manager.cc:927] T df9715570a6844d9a2686307db16eff1 P 88935ab835544e9889d05a6587d8cf21: Initiating tablet copy from peer 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:56.108418 22615 tablet_copy_client.cc:323] T eb4516b9a16041c2adf34be520768f3f P 88935ab835544e9889d05a6587d8cf21: tablet copy: Beginning tablet copy session from remote peer at address 127.19.226.195:35731
I20250109 08:57:56.109939 21673 tablet_copy_service.cc:140] P 787ddcc824c2490a808fc5a53452dc5e: Received BeginTabletCopySession request for tablet eb4516b9a16041c2adf34be520768f3f from peer 88935ab835544e9889d05a6587d8cf21 ({username='slave'} at 127.0.0.1:52376)
I20250109 08:57:56.110030 22613 tablet_copy_client.cc:323] T df9715570a6844d9a2686307db16eff1 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Beginning tablet copy session from remote peer at address 127.19.226.195:35731
I20250109 08:57:56.110460 21673 tablet_copy_service.cc:161] P 787ddcc824c2490a808fc5a53452dc5e: Beginning new tablet copy session on tablet eb4516b9a16041c2adf34be520768f3f from peer 88935ab835544e9889d05a6587d8cf21 at {username='slave'} at 127.0.0.1:52376: session id = 88935ab835544e9889d05a6587d8cf21-eb4516b9a16041c2adf34be520768f3f
I20250109 08:57:56.113597 21672 tablet_copy_service.cc:140] P 787ddcc824c2490a808fc5a53452dc5e: Received BeginTabletCopySession request for tablet df9715570a6844d9a2686307db16eff1 from peer 88935ab835544e9889d05a6587d8cf21 ({username='slave'} at 127.0.0.1:52376)
I20250109 08:57:56.114156 21672 tablet_copy_service.cc:161] P 787ddcc824c2490a808fc5a53452dc5e: Beginning new tablet copy session on tablet df9715570a6844d9a2686307db16eff1 from peer 88935ab835544e9889d05a6587d8cf21 at {username='slave'} at 127.0.0.1:52376: session id = 88935ab835544e9889d05a6587d8cf21-df9715570a6844d9a2686307db16eff1
I20250109 08:57:56.121688 21672 tablet_copy_source_session.cc:215] T df9715570a6844d9a2686307db16eff1 P 787ddcc824c2490a808fc5a53452dc5e: Tablet Copy: opened 0 blocks and 1 log segments
I20250109 08:57:56.124864 22613 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df9715570a6844d9a2686307db16eff1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:56.126592 21673 tablet_copy_source_session.cc:215] T eb4516b9a16041c2adf34be520768f3f P 787ddcc824c2490a808fc5a53452dc5e: Tablet Copy: opened 0 blocks and 1 log segments
I20250109 08:57:56.128746 22614 tablet_copy_client.cc:323] T 81d74e7d4472497aac5463a4b47e4fb9 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Beginning tablet copy session from remote peer at address 127.19.226.195:35731
I20250109 08:57:56.131208 22615 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb4516b9a16041c2adf34be520768f3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:56.131316 22597 tablet_bootstrap.cc:492] T d6f988a231a74f6fa112da9a527d4a0b P 88935ab835544e9889d05a6587d8cf21: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250109 08:57:56.132258 22597 tablet_bootstrap.cc:492] T d6f988a231a74f6fa112da9a527d4a0b P 88935ab835544e9889d05a6587d8cf21: Bootstrap complete.
I20250109 08:57:56.132997 22597 ts_tablet_manager.cc:1397] T d6f988a231a74f6fa112da9a527d4a0b P 88935ab835544e9889d05a6587d8cf21: Time spent bootstrapping tablet: real 0.082s user 0.020s sys 0.007s
I20250109 08:57:56.134943 21673 tablet_copy_service.cc:140] P 787ddcc824c2490a808fc5a53452dc5e: Received BeginTabletCopySession request for tablet 81d74e7d4472497aac5463a4b47e4fb9 from peer 88935ab835544e9889d05a6587d8cf21 ({username='slave'} at 127.0.0.1:52376)
I20250109 08:57:56.136420 22597 raft_consensus.cc:357] T d6f988a231a74f6fa112da9a527d4a0b P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:57:56.138629 21673 tablet_copy_service.cc:161] P 787ddcc824c2490a808fc5a53452dc5e: Beginning new tablet copy session on tablet 81d74e7d4472497aac5463a4b47e4fb9 from peer 88935ab835544e9889d05a6587d8cf21 at {username='slave'} at 127.0.0.1:52376: session id = 88935ab835544e9889d05a6587d8cf21-81d74e7d4472497aac5463a4b47e4fb9
I20250109 08:57:56.138828 22597 raft_consensus.cc:738] T d6f988a231a74f6fa112da9a527d4a0b P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 88935ab835544e9889d05a6587d8cf21, State: Initialized, Role: LEARNER
I20250109 08:57:56.139811 22613 tablet_copy_client.cc:806] T df9715570a6844d9a2686307db16eff1 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Starting download of 0 data blocks...
I20250109 08:57:56.139539 22597 consensus_queue.cc:260] T d6f988a231a74f6fa112da9a527d4a0b P 88935ab835544e9889d05a6587d8cf21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:57:56.140403 22613 tablet_copy_client.cc:670] T df9715570a6844d9a2686307db16eff1 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Starting download of 1 WAL segments...
I20250109 08:57:56.144203 22613 tablet_copy_client.cc:538] T df9715570a6844d9a2686307db16eff1 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250109 08:57:56.148743 22615 tablet_copy_client.cc:806] T eb4516b9a16041c2adf34be520768f3f P 88935ab835544e9889d05a6587d8cf21: tablet copy: Starting download of 0 data blocks...
I20250109 08:57:56.149593 22615 tablet_copy_client.cc:670] T eb4516b9a16041c2adf34be520768f3f P 88935ab835544e9889d05a6587d8cf21: tablet copy: Starting download of 1 WAL segments...
I20250109 08:57:56.149972 22597 ts_tablet_manager.cc:1428] T d6f988a231a74f6fa112da9a527d4a0b P 88935ab835544e9889d05a6587d8cf21: Time spent starting tablet: real 0.016s user 0.008s sys 0.004s
I20250109 08:57:56.152887 21524 tablet_copy_service.cc:342] P 8bf6e26e98b14566899578a990b27d47: Request end of tablet copy session 88935ab835544e9889d05a6587d8cf21-d6f988a231a74f6fa112da9a527d4a0b received from {username='slave'} at 127.0.0.1:41812
I20250109 08:57:56.153260 21524 tablet_copy_service.cc:434] P 8bf6e26e98b14566899578a990b27d47: ending tablet copy session 88935ab835544e9889d05a6587d8cf21-d6f988a231a74f6fa112da9a527d4a0b on tablet d6f988a231a74f6fa112da9a527d4a0b with peer 88935ab835544e9889d05a6587d8cf21
I20250109 08:57:56.155092 22615 tablet_copy_client.cc:538] T eb4516b9a16041c2adf34be520768f3f P 88935ab835544e9889d05a6587d8cf21: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250109 08:57:56.165215 22613 tablet_bootstrap.cc:492] T df9715570a6844d9a2686307db16eff1 P 88935ab835544e9889d05a6587d8cf21: Bootstrap starting.
I20250109 08:57:56.167913 22615 tablet_bootstrap.cc:492] T eb4516b9a16041c2adf34be520768f3f P 88935ab835544e9889d05a6587d8cf21: Bootstrap starting.
I20250109 08:57:56.174656 21673 tablet_copy_source_session.cc:215] T 81d74e7d4472497aac5463a4b47e4fb9 P 787ddcc824c2490a808fc5a53452dc5e: Tablet Copy: opened 0 blocks and 1 log segments
I20250109 08:57:56.184823 22614 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81d74e7d4472497aac5463a4b47e4fb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:56.188799 22613 tablet_bootstrap.cc:492] T df9715570a6844d9a2686307db16eff1 P 88935ab835544e9889d05a6587d8cf21: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250109 08:57:56.189488 22613 tablet_bootstrap.cc:492] T df9715570a6844d9a2686307db16eff1 P 88935ab835544e9889d05a6587d8cf21: Bootstrap complete.
I20250109 08:57:56.190008 22613 ts_tablet_manager.cc:1397] T df9715570a6844d9a2686307db16eff1 P 88935ab835544e9889d05a6587d8cf21: Time spent bootstrapping tablet: real 0.025s user 0.019s sys 0.001s
I20250109 08:57:56.193089 22613 raft_consensus.cc:357] T df9715570a6844d9a2686307db16eff1 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:57:56.193918 22613 raft_consensus.cc:738] T df9715570a6844d9a2686307db16eff1 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 88935ab835544e9889d05a6587d8cf21, State: Initialized, Role: LEARNER
I20250109 08:57:56.194506 22613 consensus_queue.cc:260] T df9715570a6844d9a2686307db16eff1 P 88935ab835544e9889d05a6587d8cf21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:57:56.197903 22614 tablet_copy_client.cc:806] T 81d74e7d4472497aac5463a4b47e4fb9 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Starting download of 0 data blocks...
I20250109 08:57:56.198427 22614 tablet_copy_client.cc:670] T 81d74e7d4472497aac5463a4b47e4fb9 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Starting download of 1 WAL segments...
I20250109 08:57:56.199939 22613 ts_tablet_manager.cc:1428] T df9715570a6844d9a2686307db16eff1 P 88935ab835544e9889d05a6587d8cf21: Time spent starting tablet: real 0.010s user 0.007s sys 0.000s
I20250109 08:57:56.202889 22614 tablet_copy_client.cc:538] T 81d74e7d4472497aac5463a4b47e4fb9 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250109 08:57:56.202986 21673 tablet_copy_service.cc:342] P 787ddcc824c2490a808fc5a53452dc5e: Request end of tablet copy session 88935ab835544e9889d05a6587d8cf21-df9715570a6844d9a2686307db16eff1 received from {username='slave'} at 127.0.0.1:52376
I20250109 08:57:56.203392 21673 tablet_copy_service.cc:434] P 787ddcc824c2490a808fc5a53452dc5e: ending tablet copy session 88935ab835544e9889d05a6587d8cf21-df9715570a6844d9a2686307db16eff1 on tablet df9715570a6844d9a2686307db16eff1 with peer 88935ab835544e9889d05a6587d8cf21
I20250109 08:57:56.207376 22615 tablet_bootstrap.cc:492] T eb4516b9a16041c2adf34be520768f3f P 88935ab835544e9889d05a6587d8cf21: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250109 08:57:56.208099 22615 tablet_bootstrap.cc:492] T eb4516b9a16041c2adf34be520768f3f P 88935ab835544e9889d05a6587d8cf21: Bootstrap complete.
I20250109 08:57:56.208739 22615 ts_tablet_manager.cc:1397] T eb4516b9a16041c2adf34be520768f3f P 88935ab835544e9889d05a6587d8cf21: Time spent bootstrapping tablet: real 0.041s user 0.019s sys 0.004s
I20250109 08:57:56.211334 22615 raft_consensus.cc:357] T eb4516b9a16041c2adf34be520768f3f P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:57:56.212548 22615 raft_consensus.cc:738] T eb4516b9a16041c2adf34be520768f3f P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 88935ab835544e9889d05a6587d8cf21, State: Initialized, Role: LEARNER
I20250109 08:57:56.213074 22615 consensus_queue.cc:260] T eb4516b9a16041c2adf34be520768f3f P 88935ab835544e9889d05a6587d8cf21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:57:56.213937 22614 tablet_bootstrap.cc:492] T 81d74e7d4472497aac5463a4b47e4fb9 P 88935ab835544e9889d05a6587d8cf21: Bootstrap starting.
I20250109 08:57:56.217900 22615 ts_tablet_manager.cc:1428] T eb4516b9a16041c2adf34be520768f3f P 88935ab835544e9889d05a6587d8cf21: Time spent starting tablet: real 0.009s user 0.002s sys 0.004s
I20250109 08:57:56.219343 21673 tablet_copy_service.cc:342] P 787ddcc824c2490a808fc5a53452dc5e: Request end of tablet copy session 88935ab835544e9889d05a6587d8cf21-eb4516b9a16041c2adf34be520768f3f received from {username='slave'} at 127.0.0.1:52376
I20250109 08:57:56.219729 21673 tablet_copy_service.cc:434] P 787ddcc824c2490a808fc5a53452dc5e: ending tablet copy session 88935ab835544e9889d05a6587d8cf21-eb4516b9a16041c2adf34be520768f3f on tablet eb4516b9a16041c2adf34be520768f3f with peer 88935ab835544e9889d05a6587d8cf21
I20250109 08:57:56.236126 22614 tablet_bootstrap.cc:492] T 81d74e7d4472497aac5463a4b47e4fb9 P 88935ab835544e9889d05a6587d8cf21: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250109 08:57:56.236801 22614 tablet_bootstrap.cc:492] T 81d74e7d4472497aac5463a4b47e4fb9 P 88935ab835544e9889d05a6587d8cf21: Bootstrap complete.
I20250109 08:57:56.237308 22614 ts_tablet_manager.cc:1397] T 81d74e7d4472497aac5463a4b47e4fb9 P 88935ab835544e9889d05a6587d8cf21: Time spent bootstrapping tablet: real 0.024s user 0.012s sys 0.008s
I20250109 08:57:56.239758 22614 raft_consensus.cc:357] T 81d74e7d4472497aac5463a4b47e4fb9 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:57:56.241163 22614 raft_consensus.cc:738] T 81d74e7d4472497aac5463a4b47e4fb9 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 88935ab835544e9889d05a6587d8cf21, State: Initialized, Role: LEARNER
I20250109 08:57:56.241719 22614 consensus_queue.cc:260] T 81d74e7d4472497aac5463a4b47e4fb9 P 88935ab835544e9889d05a6587d8cf21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:57:56.243487 22176 raft_consensus.cc:1212] T df9715570a6844d9a2686307db16eff1 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20250109 08:57:56.248400 22614 ts_tablet_manager.cc:1428] T 81d74e7d4472497aac5463a4b47e4fb9 P 88935ab835544e9889d05a6587d8cf21: Time spent starting tablet: real 0.011s user 0.001s sys 0.005s
I20250109 08:57:56.250010 21673 tablet_copy_service.cc:342] P 787ddcc824c2490a808fc5a53452dc5e: Request end of tablet copy session 88935ab835544e9889d05a6587d8cf21-81d74e7d4472497aac5463a4b47e4fb9 received from {username='slave'} at 127.0.0.1:52376
I20250109 08:57:56.250403 21673 tablet_copy_service.cc:434] P 787ddcc824c2490a808fc5a53452dc5e: ending tablet copy session 88935ab835544e9889d05a6587d8cf21-81d74e7d4472497aac5463a4b47e4fb9 on tablet 81d74e7d4472497aac5463a4b47e4fb9 with peer 88935ab835544e9889d05a6587d8cf21
I20250109 08:57:56.382126 22176 raft_consensus.cc:1212] T d6f988a231a74f6fa112da9a527d4a0b P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20250109 08:57:56.396908 22176 raft_consensus.cc:1212] T 81d74e7d4472497aac5463a4b47e4fb9 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20250109 08:57:56.479141 22174 raft_consensus.cc:1212] T eb4516b9a16041c2adf34be520768f3f P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20250109 08:57:56.725252 22598 raft_consensus.cc:1059] T df9715570a6844d9a2686307db16eff1 P 787ddcc824c2490a808fc5a53452dc5e: attempting to promote NON_VOTER 88935ab835544e9889d05a6587d8cf21 to VOTER
I20250109 08:57:56.727151 22598 consensus_queue.cc:237] T df9715570a6844d9a2686307db16eff1 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:57:56.732952 21583 raft_consensus.cc:1270] T df9715570a6844d9a2686307db16eff1 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:57:56.733072 22174 raft_consensus.cc:1270] T df9715570a6844d9a2686307db16eff1 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:57:56.733183 21513 raft_consensus.cc:1270] T df9715570a6844d9a2686307db16eff1 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:57:56.734481 22598 consensus_queue.cc:1035] T df9715570a6844d9a2686307db16eff1 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:57:56.735368 22607 consensus_queue.cc:1035] T df9715570a6844d9a2686307db16eff1 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:57:56.736099 22603 consensus_queue.cc:1035] T df9715570a6844d9a2686307db16eff1 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:57:56.746479 22607 raft_consensus.cc:2949] T df9715570a6844d9a2686307db16eff1 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:56.749305 21583 raft_consensus.cc:2949] T df9715570a6844d9a2686307db16eff1 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:56.751051 22174 raft_consensus.cc:2949] T df9715570a6844d9a2686307db16eff1 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:56.759680 21513 raft_consensus.cc:2949] T df9715570a6844d9a2686307db16eff1 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:56.768865 21409 catalog_manager.cc:5526] T df9715570a6844d9a2686307db16eff1 P a2b1a692435241ee988e0bd1661cd4c3 reported cstate change: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "787ddcc824c2490a808fc5a53452dc5e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:56.789462 22598 raft_consensus.cc:1059] T 81d74e7d4472497aac5463a4b47e4fb9 P 787ddcc824c2490a808fc5a53452dc5e: attempting to promote NON_VOTER 88935ab835544e9889d05a6587d8cf21 to VOTER
I20250109 08:57:56.791870 22598 consensus_queue.cc:237] T 81d74e7d4472497aac5463a4b47e4fb9 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:57:56.789727 21657 consensus_queue.cc:237] T df9715570a6844d9a2686307db16eff1 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:57:56.804320 22174 raft_consensus.cc:1270] T 81d74e7d4472497aac5463a4b47e4fb9 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:57:56.806928 22603 consensus_queue.cc:1035] T 81d74e7d4472497aac5463a4b47e4fb9 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:57:56.807379 22174 raft_consensus.cc:1270] T df9715570a6844d9a2686307db16eff1 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250109 08:57:56.808990 21513 raft_consensus.cc:1270] T df9715570a6844d9a2686307db16eff1 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250109 08:57:56.809589 21509 raft_consensus.cc:1270] T 81d74e7d4472497aac5463a4b47e4fb9 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:57:56.810659 21583 raft_consensus.cc:1270] T 81d74e7d4472497aac5463a4b47e4fb9 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:57:56.810555 22577 consensus_queue.cc:1035] T df9715570a6844d9a2686307db16eff1 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250109 08:57:56.811641 22607 consensus_queue.cc:1035] T df9715570a6844d9a2686307db16eff1 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250109 08:57:56.819114 22607 consensus_queue.cc:1035] T 81d74e7d4472497aac5463a4b47e4fb9 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:57:56.820340 22577 consensus_queue.cc:1035] T 81d74e7d4472497aac5463a4b47e4fb9 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20250109 08:57:56.820761 21509 raft_consensus.cc:2949] T df9715570a6844d9a2686307db16eff1 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:56.820613 22598 raft_consensus.cc:2949] T df9715570a6844d9a2686307db16eff1 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:56.824083 22176 raft_consensus.cc:2949] T df9715570a6844d9a2686307db16eff1 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:56.839851 21396 catalog_manager.cc:5039] ChangeConfig:REMOVE_PEER RPC for tablet df9715570a6844d9a2686307db16eff1 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250109 08:57:56.844909 21412 catalog_manager.cc:5526] T df9715570a6844d9a2686307db16eff1 P 8bf6e26e98b14566899578a990b27d47 reported cstate change: config changed from index 3 to 4, VOTER a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194) evicted. New cstate: current_term: 1 leader_uuid: "787ddcc824c2490a808fc5a53452dc5e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:56.866703 22603 raft_consensus.cc:2949] T 81d74e7d4472497aac5463a4b47e4fb9 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:56.868088 21513 raft_consensus.cc:2949] T 81d74e7d4472497aac5463a4b47e4fb9 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:56.877480 22607 raft_consensus.cc:1059] T eb4516b9a16041c2adf34be520768f3f P 787ddcc824c2490a808fc5a53452dc5e: attempting to promote NON_VOTER 88935ab835544e9889d05a6587d8cf21 to VOTER
I20250109 08:57:56.880997 21570 tablet_service.cc:1514] Processing DeleteTablet for tablet df9715570a6844d9a2686307db16eff1 with delete_type TABLET_DATA_TOMBSTONED (TS a2b1a692435241ee988e0bd1661cd4c3 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:34732
I20250109 08:57:56.879693 22607 consensus_queue.cc:237] T eb4516b9a16041c2adf34be520768f3f P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:57:56.884133 22174 raft_consensus.cc:2949] T 81d74e7d4472497aac5463a4b47e4fb9 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:56.899205 22637 tablet_replica.cc:331] T df9715570a6844d9a2686307db16eff1 P a2b1a692435241ee988e0bd1661cd4c3: stopping tablet replica
W20250109 08:57:56.900033 21684 debug-util.cc:398] Leaking SignalData structure 0x7b080015c100 after lost signal to thread 20366
W20250109 08:57:56.901985 21684 debug-util.cc:398] Leaking SignalData structure 0x7b080013b4e0 after lost signal to thread 21446
W20250109 08:57:56.903126 21684 debug-util.cc:398] Leaking SignalData structure 0x7b0800261460 after lost signal to thread 21538
W20250109 08:57:56.904148 21684 debug-util.cc:398] Leaking SignalData structure 0x7b0800324400 after lost signal to thread 21613
W20250109 08:57:56.905189 21684 debug-util.cc:398] Leaking SignalData structure 0x7b0800324360 after lost signal to thread 21687
I20250109 08:57:56.905539 22637 raft_consensus.cc:2238] T df9715570a6844d9a2686307db16eff1 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:57:56.906250 22637 raft_consensus.cc:2267] T df9715570a6844d9a2686307db16eff1 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Raft consensus is shut down!
W20250109 08:57:56.906168 21684 debug-util.cc:398] Leaking SignalData structure 0x7b0800217be0 after lost signal to thread 22202
I20250109 08:57:56.906781 21588 raft_consensus.cc:1270] T eb4516b9a16041c2adf34be520768f3f P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:57:56.911934 22176 raft_consensus.cc:1270] T eb4516b9a16041c2adf34be520768f3f P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:57:56.915201 22637 ts_tablet_manager.cc:1905] T df9715570a6844d9a2686307db16eff1 P a2b1a692435241ee988e0bd1661cd4c3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250109 08:57:56.917392 21513 raft_consensus.cc:1270] T eb4516b9a16041c2adf34be520768f3f P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:57:56.917189 21588 raft_consensus.cc:2949] T 81d74e7d4472497aac5463a4b47e4fb9 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:56.920480 22577 consensus_queue.cc:1035] T eb4516b9a16041c2adf34be520768f3f P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:57:56.926074 22607 consensus_queue.cc:1035] T eb4516b9a16041c2adf34be520768f3f P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:57:56.938472 21411 catalog_manager.cc:5526] T 81d74e7d4472497aac5463a4b47e4fb9 P 787ddcc824c2490a808fc5a53452dc5e reported cstate change: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "787ddcc824c2490a808fc5a53452dc5e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250109 08:57:56.947809 22577 consensus_queue.cc:1035] T eb4516b9a16041c2adf34be520768f3f P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20250109 08:57:56.962682 22637 ts_tablet_manager.cc:1918] T df9715570a6844d9a2686307db16eff1 P a2b1a692435241ee988e0bd1661cd4c3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20250109 08:57:56.963207 22637 log.cc:1198] T df9715570a6844d9a2686307db16eff1 P a2b1a692435241ee988e0bd1661cd4c3: Deleting WAL directory at /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736412987538469-20363-0/minicluster-data/ts-1-root/wals/df9715570a6844d9a2686307db16eff1
I20250109 08:57:56.965448 21397 catalog_manager.cc:4872] TS a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): tablet df9715570a6844d9a2686307db16eff1 (table test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]) successfully deleted
I20250109 08:57:56.969244 22575 raft_consensus.cc:1059] T d6f988a231a74f6fa112da9a527d4a0b P 8bf6e26e98b14566899578a990b27d47: attempting to promote NON_VOTER 88935ab835544e9889d05a6587d8cf21 to VOTER
I20250109 08:57:56.971715 22575 consensus_queue.cc:237] T d6f988a231a74f6fa112da9a527d4a0b P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:57:56.981190 21656 consensus_queue.cc:237] T 81d74e7d4472497aac5463a4b47e4fb9 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:57:57.002194 22176 raft_consensus.cc:1270] T 81d74e7d4472497aac5463a4b47e4fb9 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250109 08:57:57.002794 21588 raft_consensus.cc:1270] T 81d74e7d4472497aac5463a4b47e4fb9 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250109 08:57:57.009022 22633 raft_consensus.cc:2949] T eb4516b9a16041c2adf34be520768f3f P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:57.017338 22176 raft_consensus.cc:1270] T d6f988a231a74f6fa112da9a527d4a0b P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Refusing update from remote peer 8bf6e26e98b14566899578a990b27d47: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:57:57.018666 21583 raft_consensus.cc:1270] T d6f988a231a74f6fa112da9a527d4a0b P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Refusing update from remote peer 8bf6e26e98b14566899578a990b27d47: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:57:57.019407 22607 consensus_queue.cc:1035] T 81d74e7d4472497aac5463a4b47e4fb9 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250109 08:57:57.020033 22176 raft_consensus.cc:2949] T eb4516b9a16041c2adf34be520768f3f P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:57.024312 22606 consensus_queue.cc:1035] T d6f988a231a74f6fa112da9a527d4a0b P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:57:57.025255 22598 consensus_queue.cc:1035] T 81d74e7d4472497aac5463a4b47e4fb9 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250109 08:57:57.025306 22559 consensus_queue.cc:1035] T d6f988a231a74f6fa112da9a527d4a0b P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20250109 08:57:57.030077 21656 raft_consensus.cc:1270] T d6f988a231a74f6fa112da9a527d4a0b P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Refusing update from remote peer 8bf6e26e98b14566899578a990b27d47: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:57:57.035329 22575 consensus_queue.cc:1035] T d6f988a231a74f6fa112da9a527d4a0b P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20250109 08:57:57.036000 21584 raft_consensus.cc:2949] T eb4516b9a16041c2adf34be520768f3f P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:57.042582 22603 raft_consensus.cc:2949] T 81d74e7d4472497aac5463a4b47e4fb9 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8bf6e26e98b14566899578a990b27d47 (127.19.226.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:57.031150 21513 raft_consensus.cc:2949] T eb4516b9a16041c2adf34be520768f3f P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:57.049134 21411 catalog_manager.cc:5526] T eb4516b9a16041c2adf34be520768f3f P 88935ab835544e9889d05a6587d8cf21 reported cstate change: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "787ddcc824c2490a808fc5a53452dc5e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:57.064909 22559 raft_consensus.cc:2949] T d6f988a231a74f6fa112da9a527d4a0b P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:57.065085 22174 raft_consensus.cc:2949] T 81d74e7d4472497aac5463a4b47e4fb9 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8bf6e26e98b14566899578a990b27d47 (127.19.226.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:57.071157 21583 raft_consensus.cc:2949] T d6f988a231a74f6fa112da9a527d4a0b P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:57.087672 21411 catalog_manager.cc:5526] T 81d74e7d4472497aac5463a4b47e4fb9 P 88935ab835544e9889d05a6587d8cf21 reported cstate change: config changed from index 3 to 4, VOTER 8bf6e26e98b14566899578a990b27d47 (127.19.226.193) evicted. New cstate: current_term: 1 leader_uuid: "787ddcc824c2490a808fc5a53452dc5e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:57.095412 21396 catalog_manager.cc:5039] ChangeConfig:REMOVE_PEER RPC for tablet 81d74e7d4472497aac5463a4b47e4fb9 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250109 08:57:57.109655 21412 catalog_manager.cc:5526] T d6f988a231a74f6fa112da9a527d4a0b P 8bf6e26e98b14566899578a990b27d47 reported cstate change: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8bf6e26e98b14566899578a990b27d47" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20250109 08:57:57.129011 21503 tablet_service.cc:1514] Processing DeleteTablet for tablet 81d74e7d4472497aac5463a4b47e4fb9 with delete_type TABLET_DATA_TOMBSTONED (TS 8bf6e26e98b14566899578a990b27d47 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:39242
W20250109 08:57:57.135183 21627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.226.196:44363, user_credentials={real_user=slave}} blocked reactor thread for 59523.8us
I20250109 08:57:57.136994 21588 raft_consensus.cc:2949] T 81d74e7d4472497aac5463a4b47e4fb9 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8bf6e26e98b14566899578a990b27d47 (127.19.226.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:57.129940 22174 raft_consensus.cc:2949] T d6f988a231a74f6fa112da9a527d4a0b P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:57.168615 22651 tablet_replica.cc:331] T 81d74e7d4472497aac5463a4b47e4fb9 P 8bf6e26e98b14566899578a990b27d47: stopping tablet replica
I20250109 08:57:57.168686 21657 consensus_queue.cc:237] T eb4516b9a16041c2adf34be520768f3f P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:57:57.169689 22651 raft_consensus.cc:2238] T 81d74e7d4472497aac5463a4b47e4fb9 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:57:57.170379 22651 raft_consensus.cc:2267] T 81d74e7d4472497aac5463a4b47e4fb9 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:57:57.196624 21513 consensus_queue.cc:237] T d6f988a231a74f6fa112da9a527d4a0b P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:57:57.202801 22174 raft_consensus.cc:1270] T d6f988a231a74f6fa112da9a527d4a0b P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Refusing update from remote peer 8bf6e26e98b14566899578a990b27d47: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250109 08:57:57.204262 22606 consensus_queue.cc:1035] T d6f988a231a74f6fa112da9a527d4a0b P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250109 08:57:57.212636 21583 raft_consensus.cc:1270] T d6f988a231a74f6fa112da9a527d4a0b P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Refusing update from remote peer 8bf6e26e98b14566899578a990b27d47: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250109 08:57:57.214111 22606 consensus_queue.cc:1035] T d6f988a231a74f6fa112da9a527d4a0b P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250109 08:57:57.215968 22174 raft_consensus.cc:1270] T eb4516b9a16041c2adf34be520768f3f P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250109 08:57:57.217025 21584 raft_consensus.cc:1270] T eb4516b9a16041c2adf34be520768f3f P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250109 08:57:57.227216 22651 ts_tablet_manager.cc:1905] T 81d74e7d4472497aac5463a4b47e4fb9 P 8bf6e26e98b14566899578a990b27d47: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250109 08:57:57.230587 22559 raft_consensus.cc:2949] T d6f988a231a74f6fa112da9a527d4a0b P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:57.230808 21655 raft_consensus.cc:2949] T d6f988a231a74f6fa112da9a527d4a0b P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:57.237020 22653 consensus_queue.cc:1035] T eb4516b9a16041c2adf34be520768f3f P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250109 08:57:57.242476 22653 consensus_queue.cc:1035] T eb4516b9a16041c2adf34be520768f3f P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250109 08:57:57.247069 21396 catalog_manager.cc:5039] ChangeConfig:REMOVE_PEER RPC for tablet d6f988a231a74f6fa112da9a527d4a0b with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250109 08:57:57.251971 22174 raft_consensus.cc:2949] T d6f988a231a74f6fa112da9a527d4a0b P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:57.250780 21589 raft_consensus.cc:2949] T d6f988a231a74f6fa112da9a527d4a0b P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:57.267657 22653 raft_consensus.cc:2949] T eb4516b9a16041c2adf34be520768f3f P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8bf6e26e98b14566899578a990b27d47 (127.19.226.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:57.263267 22651 ts_tablet_manager.cc:1918] T 81d74e7d4472497aac5463a4b47e4fb9 P 8bf6e26e98b14566899578a990b27d47: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20250109 08:57:57.272387 22651 log.cc:1198] T 81d74e7d4472497aac5463a4b47e4fb9 P 8bf6e26e98b14566899578a990b27d47: Deleting WAL directory at /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736412987538469-20363-0/minicluster-data/ts-0-root/wals/81d74e7d4472497aac5463a4b47e4fb9
I20250109 08:57:57.274814 21396 catalog_manager.cc:4872] TS 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): tablet 81d74e7d4472497aac5463a4b47e4fb9 (table test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]) successfully deleted
I20250109 08:57:57.282361 21396 catalog_manager.cc:5039] ChangeConfig:REMOVE_PEER RPC for tablet eb4516b9a16041c2adf34be520768f3f with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250109 08:57:57.282852 21588 raft_consensus.cc:2949] T eb4516b9a16041c2adf34be520768f3f P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8bf6e26e98b14566899578a990b27d47 (127.19.226.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:57.291265 21411 catalog_manager.cc:5526] T d6f988a231a74f6fa112da9a527d4a0b P 8bf6e26e98b14566899578a990b27d47 reported cstate change: config changed from index 3 to 4, VOTER 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195) evicted. New cstate: current_term: 1 leader_uuid: "8bf6e26e98b14566899578a990b27d47" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250109 08:57:57.293074 22174 raft_consensus.cc:2949] T eb4516b9a16041c2adf34be520768f3f P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8bf6e26e98b14566899578a990b27d47 (127.19.226.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:57.308784 21653 tablet_service.cc:1514] Processing DeleteTablet for tablet d6f988a231a74f6fa112da9a527d4a0b with delete_type TABLET_DATA_TOMBSTONED (TS 787ddcc824c2490a808fc5a53452dc5e not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:58436
I20250109 08:57:57.316236 21409 catalog_manager.cc:5526] T eb4516b9a16041c2adf34be520768f3f P a2b1a692435241ee988e0bd1661cd4c3 reported cstate change: config changed from index 3 to 4, VOTER 8bf6e26e98b14566899578a990b27d47 (127.19.226.193) evicted. New cstate: current_term: 1 leader_uuid: "787ddcc824c2490a808fc5a53452dc5e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:57:57.342754 22655 tablet_replica.cc:331] T d6f988a231a74f6fa112da9a527d4a0b P 787ddcc824c2490a808fc5a53452dc5e: stopping tablet replica
I20250109 08:57:57.355185 22655 raft_consensus.cc:2238] T d6f988a231a74f6fa112da9a527d4a0b P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:57:57.355960 22655 raft_consensus.cc:2267] T d6f988a231a74f6fa112da9a527d4a0b P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:57:57.356950 21503 tablet_service.cc:1514] Processing DeleteTablet for tablet eb4516b9a16041c2adf34be520768f3f with delete_type TABLET_DATA_TOMBSTONED (TS 8bf6e26e98b14566899578a990b27d47 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:39242
I20250109 08:57:57.357795 22651 tablet_replica.cc:331] T eb4516b9a16041c2adf34be520768f3f P 8bf6e26e98b14566899578a990b27d47: stopping tablet replica
I20250109 08:57:57.359549 22651 raft_consensus.cc:2238] T eb4516b9a16041c2adf34be520768f3f P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:57:57.360160 22651 raft_consensus.cc:2267] T eb4516b9a16041c2adf34be520768f3f P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:57:57.360431 22655 ts_tablet_manager.cc:1905] T d6f988a231a74f6fa112da9a527d4a0b P 787ddcc824c2490a808fc5a53452dc5e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250109 08:57:57.363665 22651 ts_tablet_manager.cc:1905] T eb4516b9a16041c2adf34be520768f3f P 8bf6e26e98b14566899578a990b27d47: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250109 08:57:57.376981 22655 ts_tablet_manager.cc:1918] T d6f988a231a74f6fa112da9a527d4a0b P 787ddcc824c2490a808fc5a53452dc5e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20250109 08:57:57.377372 22655 log.cc:1198] T d6f988a231a74f6fa112da9a527d4a0b P 787ddcc824c2490a808fc5a53452dc5e: Deleting WAL directory at /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736412987538469-20363-0/minicluster-data/ts-2-root/wals/d6f988a231a74f6fa112da9a527d4a0b
I20250109 08:57:57.378844 21396 catalog_manager.cc:4872] TS 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731): tablet d6f988a231a74f6fa112da9a527d4a0b (table test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]) successfully deleted
I20250109 08:57:57.383095 22651 ts_tablet_manager.cc:1918] T eb4516b9a16041c2adf34be520768f3f P 8bf6e26e98b14566899578a990b27d47: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20250109 08:57:57.383539 22651 log.cc:1198] T eb4516b9a16041c2adf34be520768f3f P 8bf6e26e98b14566899578a990b27d47: Deleting WAL directory at /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736412987538469-20363-0/minicluster-data/ts-0-root/wals/eb4516b9a16041c2adf34be520768f3f
I20250109 08:57:57.384994 21396 catalog_manager.cc:4872] TS 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): tablet eb4516b9a16041c2adf34be520768f3f (table test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]) successfully deleted
I20250109 08:57:58.309893 20363 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250109 08:57:58.451231 21514 consensus_queue.cc:237] T 180b4d539a144a38905bf340ecb24ef4 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:57:58.492352 21655 raft_consensus.cc:1270] T 180b4d539a144a38905bf340ecb24ef4 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Refusing update from remote peer 8bf6e26e98b14566899578a990b27d47: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250109 08:57:58.496426 21588 raft_consensus.cc:1270] T 180b4d539a144a38905bf340ecb24ef4 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Refusing update from remote peer 8bf6e26e98b14566899578a990b27d47: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250109 08:57:58.501955 22659 consensus_queue.cc:1035] T 180b4d539a144a38905bf340ecb24ef4 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20250109 08:57:58.503083 22661 consensus_queue.cc:1035] T 180b4d539a144a38905bf340ecb24ef4 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:57:58.507357 21655 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "20ea30dba4314290ae45d9b4c13bebdd"
dest_uuid: "787ddcc824c2490a808fc5a53452dc5e"
mode: GRACEFUL
new_leader_uuid: "88935ab835544e9889d05a6587d8cf21"
from {username='slave'} at 127.0.0.1:55054
I20250109 08:57:58.508008 21655 raft_consensus.cc:604] T 20ea30dba4314290ae45d9b4c13bebdd P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Received request to transfer leadership to TS 88935ab835544e9889d05a6587d8cf21
W20250109 08:57:58.509235 21478 consensus_peers.cc:487] T 180b4d539a144a38905bf340ecb24ef4 P 8bf6e26e98b14566899578a990b27d47 -> Peer 88935ab835544e9889d05a6587d8cf21 (127.19.226.196:44363): Couldn't send request to peer 88935ab835544e9889d05a6587d8cf21. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 180b4d539a144a38905bf340ecb24ef4. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:57:58.537403 22659 raft_consensus.cc:2949] T 180b4d539a144a38905bf340ecb24ef4 P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:57:58.544212 21655 raft_consensus.cc:2949] T 180b4d539a144a38905bf340ecb24ef4 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:57:58.553596 21513 consensus_queue.cc:237] T 206785f29c824cddbe9bbc00d365ffb3 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:57:58.561357 21409 catalog_manager.cc:5526] T 180b4d539a144a38905bf340ecb24ef4 P 787ddcc824c2490a808fc5a53452dc5e reported cstate change: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New cstate: current_term: 1 leader_uuid: "8bf6e26e98b14566899578a990b27d47" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:57:58.565196 21655 raft_consensus.cc:1270] T 206785f29c824cddbe9bbc00d365ffb3 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Refusing update from remote peer 8bf6e26e98b14566899578a990b27d47: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250109 08:57:58.566679 21583 raft_consensus.cc:1270] T 206785f29c824cddbe9bbc00d365ffb3 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Refusing update from remote peer 8bf6e26e98b14566899578a990b27d47: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250109 08:57:58.569757 21588 raft_consensus.cc:2949] T 180b4d539a144a38905bf340ecb24ef4 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:57:58.571410 21513 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "2935d69446cd458e92046d2b2f1b74ab"
dest_uuid: "8bf6e26e98b14566899578a990b27d47"
mode: GRACEFUL
new_leader_uuid: "88935ab835544e9889d05a6587d8cf21"
from {username='slave'} at 127.0.0.1:39274
I20250109 08:57:58.571920 21513 raft_consensus.cc:604] T 2935d69446cd458e92046d2b2f1b74ab P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Received request to transfer leadership to TS 88935ab835544e9889d05a6587d8cf21
W20250109 08:57:58.572883 21478 consensus_peers.cc:487] T 206785f29c824cddbe9bbc00d365ffb3 P 8bf6e26e98b14566899578a990b27d47 -> Peer 88935ab835544e9889d05a6587d8cf21 (127.19.226.196:44363): Couldn't send request to peer 88935ab835544e9889d05a6587d8cf21. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 206785f29c824cddbe9bbc00d365ffb3. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:57:58.573673 22661 consensus_queue.cc:1035] T 206785f29c824cddbe9bbc00d365ffb3 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:57:58.575014 22660 consensus_queue.cc:1035] T 206785f29c824cddbe9bbc00d365ffb3 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:57:58.575322 22661 raft_consensus.cc:988] T 2935d69446cd458e92046d2b2f1b74ab P 8bf6e26e98b14566899578a990b27d47: : Instructing follower 88935ab835544e9889d05a6587d8cf21 to start an election
I20250109 08:57:58.575486 21655 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "8c0459d2eee64f0a83bd05ad5974f506"
dest_uuid: "787ddcc824c2490a808fc5a53452dc5e"
mode: GRACEFUL
new_leader_uuid: "a2b1a692435241ee988e0bd1661cd4c3"
from {username='slave'} at 127.0.0.1:55054
I20250109 08:57:58.576959 21655 raft_consensus.cc:604] T 8c0459d2eee64f0a83bd05ad5974f506 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Received request to transfer leadership to TS a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:57:58.577777 22661 raft_consensus.cc:1076] T 2935d69446cd458e92046d2b2f1b74ab P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Signalling peer 88935ab835544e9889d05a6587d8cf21 to start an election
I20250109 08:57:58.581344 22174 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2935d69446cd458e92046d2b2f1b74ab"
dest_uuid: "88935ab835544e9889d05a6587d8cf21"
from {username='slave'} at 127.0.0.1:60444
I20250109 08:57:58.598058 22174 raft_consensus.cc:491] T 2935d69446cd458e92046d2b2f1b74ab P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:57:58.598515 22174 raft_consensus.cc:3054] T 2935d69446cd458e92046d2b2f1b74ab P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:57:58.599711 21655 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "b00671c87aac405a81c6543f4defd722"
dest_uuid: "787ddcc824c2490a808fc5a53452dc5e"
mode: GRACEFUL
new_leader_uuid: "88935ab835544e9889d05a6587d8cf21"
from {username='slave'} at 127.0.0.1:55054
I20250109 08:57:58.600337 21655 raft_consensus.cc:604] T b00671c87aac405a81c6543f4defd722 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Received request to transfer leadership to TS 88935ab835544e9889d05a6587d8cf21
I20250109 08:57:58.608916 21655 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "d52f79ecf037457cbcc09d5fa43456cf"
dest_uuid: "787ddcc824c2490a808fc5a53452dc5e"
mode: GRACEFUL
new_leader_uuid: "88935ab835544e9889d05a6587d8cf21"
from {username='slave'} at 127.0.0.1:55054
I20250109 08:57:58.608892 22174 raft_consensus.cc:513] T 2935d69446cd458e92046d2b2f1b74ab P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:57:58.609755 21655 raft_consensus.cc:604] T d52f79ecf037457cbcc09d5fa43456cf P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Received request to transfer leadership to TS 88935ab835544e9889d05a6587d8cf21
I20250109 08:57:58.613598 21663 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "df9715570a6844d9a2686307db16eff1"
dest_uuid: "787ddcc824c2490a808fc5a53452dc5e"
mode: GRACEFUL
new_leader_uuid: "88935ab835544e9889d05a6587d8cf21"
from {username='slave'} at 127.0.0.1:55054
I20250109 08:57:58.614020 22649 raft_consensus.cc:988] T 8c0459d2eee64f0a83bd05ad5974f506 P 787ddcc824c2490a808fc5a53452dc5e: : Instructing follower a2b1a692435241ee988e0bd1661cd4c3 to start an election
I20250109 08:57:58.614199 21663 raft_consensus.cc:604] T df9715570a6844d9a2686307db16eff1 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Received request to transfer leadership to TS 88935ab835544e9889d05a6587d8cf21
I20250109 08:57:58.614488 22649 raft_consensus.cc:1076] T 8c0459d2eee64f0a83bd05ad5974f506 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Signalling peer a2b1a692435241ee988e0bd1661cd4c3 to start an election
I20250109 08:57:58.616026 21588 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "8c0459d2eee64f0a83bd05ad5974f506"
dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3"
from {username='slave'} at 127.0.0.1:34746
I20250109 08:57:58.616518 21588 raft_consensus.cc:491] T 8c0459d2eee64f0a83bd05ad5974f506 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:57:58.616820 21588 raft_consensus.cc:3054] T 8c0459d2eee64f0a83bd05ad5974f506 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:57:58.622694 21588 raft_consensus.cc:513] T 8c0459d2eee64f0a83bd05ad5974f506 P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:58.625682 21513 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c0459d2eee64f0a83bd05ad5974f506" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8bf6e26e98b14566899578a990b27d47"
I20250109 08:57:58.626299 21513 raft_consensus.cc:3054] T 8c0459d2eee64f0a83bd05ad5974f506 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:57:58.631589 21588 leader_election.cc:290] T 8c0459d2eee64f0a83bd05ad5974f506 P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 2 election: Requested vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:58.632481 21663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c0459d2eee64f0a83bd05ad5974f506" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "787ddcc824c2490a808fc5a53452dc5e"
I20250109 08:57:58.633067 21663 raft_consensus.cc:3049] T 8c0459d2eee64f0a83bd05ad5974f506 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Stepping down as leader of term 1
I20250109 08:57:58.633375 21663 raft_consensus.cc:738] T 8c0459d2eee64f0a83bd05ad5974f506 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Running, Role: LEADER
I20250109 08:57:58.633805 21513 raft_consensus.cc:2463] T 8c0459d2eee64f0a83bd05ad5974f506 P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2b1a692435241ee988e0bd1661cd4c3 in term 2.
I20250109 08:57:58.634557 21514 raft_consensus.cc:1235] T 8c0459d2eee64f0a83bd05ad5974f506 P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Rejecting Update request from peer 787ddcc824c2490a808fc5a53452dc5e for earlier term 1. Current term is 2. Ops: []
I20250109 08:57:58.634070 21663 consensus_queue.cc:260] T 8c0459d2eee64f0a83bd05ad5974f506 P 787ddcc824c2490a808fc5a53452dc5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:58.635797 21663 raft_consensus.cc:3054] T 8c0459d2eee64f0a83bd05ad5974f506 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:57:58.636324 22654 raft_consensus.cc:2949] T 206785f29c824cddbe9bbc00d365ffb3 P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:57:58.643340 22665 raft_consensus.cc:988] T b00671c87aac405a81c6543f4defd722 P 787ddcc824c2490a808fc5a53452dc5e: : Instructing follower 88935ab835544e9889d05a6587d8cf21 to start an election
I20250109 08:57:58.643806 22665 raft_consensus.cc:1076] T b00671c87aac405a81c6543f4defd722 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Signalling peer 88935ab835544e9889d05a6587d8cf21 to start an election
I20250109 08:57:58.644122 21657 raft_consensus.cc:2949] T 206785f29c824cddbe9bbc00d365ffb3 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:57:58.647166 22172 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "b00671c87aac405a81c6543f4defd722"
dest_uuid: "88935ab835544e9889d05a6587d8cf21"
from {username='slave'} at 127.0.0.1:60458
I20250109 08:57:58.647737 22172 raft_consensus.cc:491] T b00671c87aac405a81c6543f4defd722 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:57:58.648109 22172 raft_consensus.cc:3054] T b00671c87aac405a81c6543f4defd722 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:57:58.649688 21663 raft_consensus.cc:2463] T 8c0459d2eee64f0a83bd05ad5974f506 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2b1a692435241ee988e0bd1661cd4c3 in term 2.
I20250109 08:57:58.650779 21552 leader_election.cc:304] T 8c0459d2eee64f0a83bd05ad5974f506 P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, a2b1a692435241ee988e0bd1661cd4c3; no voters:
I20250109 08:57:58.651232 22174 leader_election.cc:290] T 2935d69446cd458e92046d2b2f1b74ab P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 election: Requested vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
I20250109 08:57:58.651758 22652 raft_consensus.cc:2798] T 8c0459d2eee64f0a83bd05ad5974f506 P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:57:58.652215 22652 raft_consensus.cc:695] T 8c0459d2eee64f0a83bd05ad5974f506 P a2b1a692435241ee988e0bd1661cd4c3 [term 2 LEADER]: Becoming Leader. State: Replica: a2b1a692435241ee988e0bd1661cd4c3, State: Running, Role: LEADER
I20250109 08:57:58.652351 21514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2935d69446cd458e92046d2b2f1b74ab" candidate_uuid: "88935ab835544e9889d05a6587d8cf21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "8bf6e26e98b14566899578a990b27d47"
I20250109 08:57:58.652923 21514 raft_consensus.cc:3049] T 2935d69446cd458e92046d2b2f1b74ab P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Stepping down as leader of term 1
I20250109 08:57:58.653220 21514 raft_consensus.cc:738] T 2935d69446cd458e92046d2b2f1b74ab P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8bf6e26e98b14566899578a990b27d47, State: Running, Role: LEADER
I20250109 08:57:58.653070 22652 consensus_queue.cc:237] T 8c0459d2eee64f0a83bd05ad5974f506 P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:57:58.653877 21514 consensus_queue.cc:260] T 2935d69446cd458e92046d2b2f1b74ab P 8bf6e26e98b14566899578a990b27d47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:57:58.656167 21588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2935d69446cd458e92046d2b2f1b74ab" candidate_uuid: "88935ab835544e9889d05a6587d8cf21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3"
I20250109 08:57:58.656709 21514 raft_consensus.cc:3054] T 2935d69446cd458e92046d2b2f1b74ab P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:57:58.656816 21588 raft_consensus.cc:3054] T 2935d69446cd458e92046d2b2f1b74ab P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:57:58.665753 22172 raft_consensus.cc:513] T b00671c87aac405a81c6543f4defd722 P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:57:58.683746 22172 leader_election.cc:290] T b00671c87aac405a81c6543f4defd722 P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 election: Requested vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:58.702939 20363 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 6
I20250109 08:57:58.730346 21583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b00671c87aac405a81c6543f4defd722" candidate_uuid: "88935ab835544e9889d05a6587d8cf21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3"
I20250109 08:57:58.732121 21583 raft_consensus.cc:3054] T b00671c87aac405a81c6543f4defd722 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:57:58.746315 21663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b00671c87aac405a81c6543f4defd722" candidate_uuid: "88935ab835544e9889d05a6587d8cf21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "787ddcc824c2490a808fc5a53452dc5e"
I20250109 08:57:58.747798 21663 raft_consensus.cc:3049] T b00671c87aac405a81c6543f4defd722 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Stepping down as leader of term 1
I20250109 08:57:58.748235 21663 raft_consensus.cc:738] T b00671c87aac405a81c6543f4defd722 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Running, Role: LEADER
I20250109 08:57:58.749704 21663 consensus_queue.cc:260] T b00671c87aac405a81c6543f4defd722 P 787ddcc824c2490a808fc5a53452dc5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
W20250109 08:57:58.759563 21478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.226.194:45527, user_credentials={real_user=slave}} blocked reactor thread for 47051.7us
I20250109 08:57:58.767511 21589 raft_consensus.cc:2949] T 206785f29c824cddbe9bbc00d365ffb3 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:57:58.770417 20363 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250109 08:57:58.771765 20363 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250109 08:57:58.774356 21663 raft_consensus.cc:3054] T b00671c87aac405a81c6543f4defd722 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:57:58.776775 21514 raft_consensus.cc:2463] T 2935d69446cd458e92046d2b2f1b74ab P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88935ab835544e9889d05a6587d8cf21 in term 2.
I20250109 08:57:58.747938 22653 raft_consensus.cc:988] T d52f79ecf037457cbcc09d5fa43456cf P 787ddcc824c2490a808fc5a53452dc5e: : Instructing follower 88935ab835544e9889d05a6587d8cf21 to start an election
I20250109 08:57:58.782912 22653 raft_consensus.cc:1076] T d52f79ecf037457cbcc09d5fa43456cf P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Signalling peer 88935ab835544e9889d05a6587d8cf21 to start an election
I20250109 08:57:58.788239 21657 consensus_queue.cc:237] T 64c667f887ea41b4a484ee565b21fa16 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:57:58.791383 21588 raft_consensus.cc:2463] T 2935d69446cd458e92046d2b2f1b74ab P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88935ab835544e9889d05a6587d8cf21 in term 2.
I20250109 08:57:58.793237 22142 leader_election.cc:304] T 2935d69446cd458e92046d2b2f1b74ab P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88935ab835544e9889d05a6587d8cf21, a2b1a692435241ee988e0bd1661cd4c3; no voters:
I20250109 08:57:58.795182 21583 raft_consensus.cc:2463] T b00671c87aac405a81c6543f4defd722 P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88935ab835544e9889d05a6587d8cf21 in term 2.
I20250109 08:57:58.806269 21409 catalog_manager.cc:5526] T 206785f29c824cddbe9bbc00d365ffb3 P 787ddcc824c2490a808fc5a53452dc5e reported cstate change: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New cstate: current_term: 1 leader_uuid: "8bf6e26e98b14566899578a990b27d47" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:57:58.810765 22172 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d52f79ecf037457cbcc09d5fa43456cf"
dest_uuid: "88935ab835544e9889d05a6587d8cf21"
from {username='slave'} at 127.0.0.1:60458
I20250109 08:57:58.812412 22172 raft_consensus.cc:491] T d52f79ecf037457cbcc09d5fa43456cf P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:57:58.812937 22172 raft_consensus.cc:3054] T d52f79ecf037457cbcc09d5fa43456cf P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:57:58.835906 21663 raft_consensus.cc:2463] T b00671c87aac405a81c6543f4defd722 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88935ab835544e9889d05a6587d8cf21 in term 2.
I20250109 08:57:58.836266 21409 catalog_manager.cc:5526] T 8c0459d2eee64f0a83bd05ad5974f506 P a2b1a692435241ee988e0bd1661cd4c3 reported cstate change: term changed from 1 to 2, leader changed from 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195) to a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194). New cstate: current_term: 2 leader_uuid: "a2b1a692435241ee988e0bd1661cd4c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: UNKNOWN } } }
I20250109 08:57:58.840487 22679 raft_consensus.cc:2798] T 2935d69446cd458e92046d2b2f1b74ab P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:57:58.844060 22172 raft_consensus.cc:513] T d52f79ecf037457cbcc09d5fa43456cf P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:57:58.853808 22142 leader_election.cc:304] T b00671c87aac405a81c6543f4defd722 P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, 88935ab835544e9889d05a6587d8cf21, a2b1a692435241ee988e0bd1661cd4c3; no voters:
I20250109 08:57:58.867636 21589 raft_consensus.cc:1270] T 64c667f887ea41b4a484ee565b21fa16 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250109 08:57:58.868048 22172 leader_election.cc:290] T d52f79ecf037457cbcc09d5fa43456cf P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 election: Requested vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:58.870321 21511 raft_consensus.cc:1270] T 64c667f887ea41b4a484ee565b21fa16 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250109 08:57:58.871049 22663 consensus_queue.cc:1035] T 64c667f887ea41b4a484ee565b21fa16 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20250109 08:57:58.872658 22653 consensus_queue.cc:1035] T 64c667f887ea41b4a484ee565b21fa16 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:57:58.880468 21657 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d52f79ecf037457cbcc09d5fa43456cf" candidate_uuid: "88935ab835544e9889d05a6587d8cf21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "787ddcc824c2490a808fc5a53452dc5e"
I20250109 08:57:58.881233 21657 raft_consensus.cc:3049] T d52f79ecf037457cbcc09d5fa43456cf P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Stepping down as leader of term 1
I20250109 08:57:58.881577 21657 raft_consensus.cc:738] T d52f79ecf037457cbcc09d5fa43456cf P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Running, Role: LEADER
I20250109 08:57:58.882531 21657 consensus_queue.cc:260] T d52f79ecf037457cbcc09d5fa43456cf P 787ddcc824c2490a808fc5a53452dc5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:57:58.884016 21657 raft_consensus.cc:3054] T d52f79ecf037457cbcc09d5fa43456cf P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:57:58.901460 22649 raft_consensus.cc:2949] T 64c667f887ea41b4a484ee565b21fa16 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:57:58.914016 21589 raft_consensus.cc:2949] T 64c667f887ea41b4a484ee565b21fa16 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:57:58.919119 21657 raft_consensus.cc:2463] T d52f79ecf037457cbcc09d5fa43456cf P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88935ab835544e9889d05a6587d8cf21 in term 2.
I20250109 08:57:58.923331 22141 leader_election.cc:304] T d52f79ecf037457cbcc09d5fa43456cf P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, 88935ab835544e9889d05a6587d8cf21; no voters:
W20250109 08:57:58.930323 22142 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.226.194:45527, user_credentials={real_user=slave}} blocked reactor thread for 108549us
I20250109 08:57:58.933437 21588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d52f79ecf037457cbcc09d5fa43456cf" candidate_uuid: "88935ab835544e9889d05a6587d8cf21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3"
I20250109 08:57:58.934109 21588 raft_consensus.cc:3054] T d52f79ecf037457cbcc09d5fa43456cf P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:57:58.938985 22682 raft_consensus.cc:2798] T b00671c87aac405a81c6543f4defd722 P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:57:58.939544 22682 raft_consensus.cc:695] T b00671c87aac405a81c6543f4defd722 P 88935ab835544e9889d05a6587d8cf21 [term 2 LEADER]: Becoming Leader. State: Replica: 88935ab835544e9889d05a6587d8cf21, State: Running, Role: LEADER
I20250109 08:57:58.940300 21513 consensus_queue.cc:237] T e601a034e10e418bac8683443d86a1c1 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
W20250109 08:57:58.941303 21627 consensus_peers.cc:487] T 64c667f887ea41b4a484ee565b21fa16 P 787ddcc824c2490a808fc5a53452dc5e -> Peer 88935ab835544e9889d05a6587d8cf21 (127.19.226.196:44363): Couldn't send request to peer 88935ab835544e9889d05a6587d8cf21. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 64c667f887ea41b4a484ee565b21fa16. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:57:58.944264 22665 raft_consensus.cc:988] T 20ea30dba4314290ae45d9b4c13bebdd P 787ddcc824c2490a808fc5a53452dc5e: : Instructing follower 88935ab835544e9889d05a6587d8cf21 to start an election
I20250109 08:57:58.944869 22665 raft_consensus.cc:1076] T 20ea30dba4314290ae45d9b4c13bebdd P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Signalling peer 88935ab835544e9889d05a6587d8cf21 to start an election
I20250109 08:57:58.948287 22172 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "20ea30dba4314290ae45d9b4c13bebdd"
dest_uuid: "88935ab835544e9889d05a6587d8cf21"
from {username='slave'} at 127.0.0.1:60458
I20250109 08:57:58.948930 22172 raft_consensus.cc:491] T 20ea30dba4314290ae45d9b4c13bebdd P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:57:58.949467 22172 raft_consensus.cc:3054] T 20ea30dba4314290ae45d9b4c13bebdd P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:57:58.953317 22679 raft_consensus.cc:695] T 2935d69446cd458e92046d2b2f1b74ab P 88935ab835544e9889d05a6587d8cf21 [term 2 LEADER]: Becoming Leader. State: Replica: 88935ab835544e9889d05a6587d8cf21, State: Running, Role: LEADER
I20250109 08:57:58.940475 22682 consensus_queue.cc:237] T b00671c87aac405a81c6543f4defd722 P 88935ab835544e9889d05a6587d8cf21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:57:58.954581 22679 consensus_queue.cc:237] T 2935d69446cd458e92046d2b2f1b74ab P 88935ab835544e9889d05a6587d8cf21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:57:58.961398 22681 raft_consensus.cc:2798] T d52f79ecf037457cbcc09d5fa43456cf P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:57:58.961851 22649 raft_consensus.cc:988] T df9715570a6844d9a2686307db16eff1 P 787ddcc824c2490a808fc5a53452dc5e: : Instructing follower 88935ab835544e9889d05a6587d8cf21 to start an election
I20250109 08:57:58.962435 22649 raft_consensus.cc:1076] T df9715570a6844d9a2686307db16eff1 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Signalling peer 88935ab835544e9889d05a6587d8cf21 to start an election
I20250109 08:57:58.963236 21588 raft_consensus.cc:2463] T d52f79ecf037457cbcc09d5fa43456cf P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88935ab835544e9889d05a6587d8cf21 in term 2.
I20250109 08:57:58.964164 22176 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "df9715570a6844d9a2686307db16eff1"
dest_uuid: "88935ab835544e9889d05a6587d8cf21"
from {username='slave'} at 127.0.0.1:60458
I20250109 08:57:58.986529 21589 raft_consensus.cc:1270] T e601a034e10e418bac8683443d86a1c1 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Refusing update from remote peer 8bf6e26e98b14566899578a990b27d47: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250109 08:57:58.986656 22176 raft_consensus.cc:491] T df9715570a6844d9a2686307db16eff1 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:57:58.987156 22176 raft_consensus.cc:3054] T df9715570a6844d9a2686307db16eff1 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:57:58.964993 22172 raft_consensus.cc:513] T 20ea30dba4314290ae45d9b4c13bebdd P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:57:58.989550 22172 leader_election.cc:290] T 20ea30dba4314290ae45d9b4c13bebdd P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 election: Requested vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:58.981992 21657 raft_consensus.cc:1270] T e601a034e10e418bac8683443d86a1c1 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Refusing update from remote peer 8bf6e26e98b14566899578a990b27d47: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250109 08:57:58.983191 21410 catalog_manager.cc:5526] T 64c667f887ea41b4a484ee565b21fa16 P 787ddcc824c2490a808fc5a53452dc5e reported cstate change: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New cstate: current_term: 1 leader_uuid: "787ddcc824c2490a808fc5a53452dc5e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20250109 08:57:59.000703 22678 consensus_queue.cc:1035] T e601a034e10e418bac8683443d86a1c1 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20250109 08:57:59.002525 22678 consensus_queue.cc:1035] T e601a034e10e418bac8683443d86a1c1 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20250109 08:57:59.010262 22141 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.226.195:35731, user_credentials={real_user=slave}} blocked reactor thread for 87140.3us
I20250109 08:57:59.012785 21663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20ea30dba4314290ae45d9b4c13bebdd" candidate_uuid: "88935ab835544e9889d05a6587d8cf21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "787ddcc824c2490a808fc5a53452dc5e"
I20250109 08:57:59.013381 21663 raft_consensus.cc:3049] T 20ea30dba4314290ae45d9b4c13bebdd P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Stepping down as leader of term 1
I20250109 08:57:59.013669 21663 raft_consensus.cc:738] T 20ea30dba4314290ae45d9b4c13bebdd P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Running, Role: LEADER
I20250109 08:57:59.014415 21663 consensus_queue.cc:260] T 20ea30dba4314290ae45d9b4c13bebdd P 787ddcc824c2490a808fc5a53452dc5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:57:59.015658 21663 raft_consensus.cc:3054] T 20ea30dba4314290ae45d9b4c13bebdd P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:57:59.018599 21514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20ea30dba4314290ae45d9b4c13bebdd" candidate_uuid: "88935ab835544e9889d05a6587d8cf21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "8bf6e26e98b14566899578a990b27d47"
I20250109 08:57:59.019188 21514 raft_consensus.cc:3054] T 20ea30dba4314290ae45d9b4c13bebdd P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:57:59.025175 21514 raft_consensus.cc:2463] T 20ea30dba4314290ae45d9b4c13bebdd P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88935ab835544e9889d05a6587d8cf21 in term 2.
I20250109 08:57:59.024583 21511 raft_consensus.cc:2949] T 64c667f887ea41b4a484ee565b21fa16 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:57:59.026264 21663 raft_consensus.cc:2463] T 20ea30dba4314290ae45d9b4c13bebdd P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88935ab835544e9889d05a6587d8cf21 in term 2.
I20250109 08:57:59.002375 22176 raft_consensus.cc:513] T df9715570a6844d9a2686307db16eff1 P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:57:59.034981 22176 leader_election.cc:290] T df9715570a6844d9a2686307db16eff1 P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 election: Requested vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:59.045472 21656 raft_consensus.cc:1270] T 8c0459d2eee64f0a83bd05ad5974f506 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Refusing update from remote peer a2b1a692435241ee988e0bd1661cd4c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:57:59.049176 22664 consensus_queue.cc:1035] T 8c0459d2eee64f0a83bd05ad5974f506 P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20250109 08:57:59.049638 22691 ts_tablet_manager.cc:927] T 180b4d539a144a38905bf340ecb24ef4 P 88935ab835544e9889d05a6587d8cf21: Initiating tablet copy from peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725)
I20250109 08:57:59.049309 22661 raft_consensus.cc:2949] T e601a034e10e418bac8683443d86a1c1 P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:57:59.056051 22693 ts_tablet_manager.cc:927] T 206785f29c824cddbe9bbc00d365ffb3 P 88935ab835544e9889d05a6587d8cf21: Initiating tablet copy from peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725)
I20250109 08:57:59.064244 22691 tablet_copy_client.cc:323] T 180b4d539a144a38905bf340ecb24ef4 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Beginning tablet copy session from remote peer at address 127.19.226.193:42725
I20250109 08:57:59.069972 21656 raft_consensus.cc:2949] T e601a034e10e418bac8683443d86a1c1 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:57:59.074771 21588 raft_consensus.cc:2949] T e601a034e10e418bac8683443d86a1c1 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:57:59.079934 22693 tablet_copy_client.cc:323] T 206785f29c824cddbe9bbc00d365ffb3 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Beginning tablet copy session from remote peer at address 127.19.226.193:42725
I20250109 08:57:59.088085 21657 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df9715570a6844d9a2686307db16eff1" candidate_uuid: "88935ab835544e9889d05a6587d8cf21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "787ddcc824c2490a808fc5a53452dc5e"
I20250109 08:57:59.088721 21657 raft_consensus.cc:3049] T df9715570a6844d9a2686307db16eff1 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Stepping down as leader of term 1
I20250109 08:57:59.090333 21657 raft_consensus.cc:738] T df9715570a6844d9a2686307db16eff1 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Running, Role: LEADER
I20250109 08:57:59.091127 21657 consensus_queue.cc:260] T df9715570a6844d9a2686307db16eff1 P 787ddcc824c2490a808fc5a53452dc5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:57:59.091897 21511 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df9715570a6844d9a2686307db16eff1" candidate_uuid: "88935ab835544e9889d05a6587d8cf21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "8bf6e26e98b14566899578a990b27d47"
I20250109 08:57:59.092507 21657 raft_consensus.cc:3054] T df9715570a6844d9a2686307db16eff1 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:57:59.093420 22141 leader_election.cc:304] T 20ea30dba4314290ae45d9b4c13bebdd P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88935ab835544e9889d05a6587d8cf21, 8bf6e26e98b14566899578a990b27d47; no voters:
I20250109 08:57:59.092537 21511 raft_consensus.cc:3054] T df9715570a6844d9a2686307db16eff1 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:57:59.107108 21410 catalog_manager.cc:5526] T e601a034e10e418bac8683443d86a1c1 P a2b1a692435241ee988e0bd1661cd4c3 reported cstate change: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New cstate: current_term: 1 leader_uuid: "8bf6e26e98b14566899578a990b27d47" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:57:59.110585 22685 raft_consensus.cc:2798] T 20ea30dba4314290ae45d9b4c13bebdd P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:57:59.113888 22685 raft_consensus.cc:695] T 20ea30dba4314290ae45d9b4c13bebdd P 88935ab835544e9889d05a6587d8cf21 [term 2 LEADER]: Becoming Leader. State: Replica: 88935ab835544e9889d05a6587d8cf21, State: Running, Role: LEADER
I20250109 08:57:59.117427 22681 raft_consensus.cc:695] T d52f79ecf037457cbcc09d5fa43456cf P 88935ab835544e9889d05a6587d8cf21 [term 2 LEADER]: Becoming Leader. State: Replica: 88935ab835544e9889d05a6587d8cf21, State: Running, Role: LEADER
I20250109 08:57:59.117849 21657 raft_consensus.cc:2463] T df9715570a6844d9a2686307db16eff1 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88935ab835544e9889d05a6587d8cf21 in term 2.
I20250109 08:57:59.118400 22681 consensus_queue.cc:237] T d52f79ecf037457cbcc09d5fa43456cf P 88935ab835544e9889d05a6587d8cf21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:57:59.119911 22141 leader_election.cc:304] T df9715570a6844d9a2686307db16eff1 P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, 88935ab835544e9889d05a6587d8cf21; no voters:
I20250109 08:57:59.120726 22679 raft_consensus.cc:2798] T df9715570a6844d9a2686307db16eff1 P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:57:59.122370 22679 raft_consensus.cc:695] T df9715570a6844d9a2686307db16eff1 P 88935ab835544e9889d05a6587d8cf21 [term 2 LEADER]: Becoming Leader. State: Replica: 88935ab835544e9889d05a6587d8cf21, State: Running, Role: LEADER
I20250109 08:57:59.123189 22679 consensus_queue.cc:237] T df9715570a6844d9a2686307db16eff1 P 88935ab835544e9889d05a6587d8cf21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:57:59.128924 21524 tablet_copy_service.cc:140] P 8bf6e26e98b14566899578a990b27d47: Received BeginTabletCopySession request for tablet 180b4d539a144a38905bf340ecb24ef4 from peer 88935ab835544e9889d05a6587d8cf21 ({username='slave'} at 127.0.0.1:41812)
I20250109 08:57:59.116070 22685 consensus_queue.cc:237] T 20ea30dba4314290ae45d9b4c13bebdd P 88935ab835544e9889d05a6587d8cf21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:57:59.129554 21524 tablet_copy_service.cc:161] P 8bf6e26e98b14566899578a990b27d47: Beginning new tablet copy session on tablet 180b4d539a144a38905bf340ecb24ef4 from peer 88935ab835544e9889d05a6587d8cf21 at {username='slave'} at 127.0.0.1:41812: session id = 88935ab835544e9889d05a6587d8cf21-180b4d539a144a38905bf340ecb24ef4
I20250109 08:57:59.132200 21511 raft_consensus.cc:2463] T df9715570a6844d9a2686307db16eff1 P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88935ab835544e9889d05a6587d8cf21 in term 2.
I20250109 08:57:59.132822 21523 tablet_copy_service.cc:140] P 8bf6e26e98b14566899578a990b27d47: Received BeginTabletCopySession request for tablet 206785f29c824cddbe9bbc00d365ffb3 from peer 88935ab835544e9889d05a6587d8cf21 ({username='slave'} at 127.0.0.1:41812)
I20250109 08:57:59.133390 21523 tablet_copy_service.cc:161] P 8bf6e26e98b14566899578a990b27d47: Beginning new tablet copy session on tablet 206785f29c824cddbe9bbc00d365ffb3 from peer 88935ab835544e9889d05a6587d8cf21 at {username='slave'} at 127.0.0.1:41812: session id = 88935ab835544e9889d05a6587d8cf21-206785f29c824cddbe9bbc00d365ffb3
W20250109 08:57:59.132771 21478 consensus_peers.cc:487] T e601a034e10e418bac8683443d86a1c1 P 8bf6e26e98b14566899578a990b27d47 -> Peer 88935ab835544e9889d05a6587d8cf21 (127.19.226.196:44363): Couldn't send request to peer 88935ab835544e9889d05a6587d8cf21. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e601a034e10e418bac8683443d86a1c1. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:57:59.146256 21524 tablet_copy_source_session.cc:215] T 180b4d539a144a38905bf340ecb24ef4 P 8bf6e26e98b14566899578a990b27d47: Tablet Copy: opened 0 blocks and 1 log segments
I20250109 08:57:59.148818 21523 tablet_copy_source_session.cc:215] T 206785f29c824cddbe9bbc00d365ffb3 P 8bf6e26e98b14566899578a990b27d47: Tablet Copy: opened 0 blocks and 1 log segments
I20250109 08:57:59.163787 21514 raft_consensus.cc:1270] T 8c0459d2eee64f0a83bd05ad5974f506 P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Refusing update from remote peer a2b1a692435241ee988e0bd1661cd4c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:57:59.165262 22664 consensus_queue.cc:1035] T 8c0459d2eee64f0a83bd05ad5974f506 P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20250109 08:57:59.181712 22691 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 180b4d539a144a38905bf340ecb24ef4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:59.182754 22693 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 206785f29c824cddbe9bbc00d365ffb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:59.225914 22691 tablet_copy_client.cc:806] T 180b4d539a144a38905bf340ecb24ef4 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Starting download of 0 data blocks...
I20250109 08:57:59.232816 22691 tablet_copy_client.cc:670] T 180b4d539a144a38905bf340ecb24ef4 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Starting download of 1 WAL segments...
I20250109 08:57:59.234606 21412 catalog_manager.cc:5526] T 2935d69446cd458e92046d2b2f1b74ab P 88935ab835544e9889d05a6587d8cf21 reported cstate change: term changed from 1 to 2, leader changed from 8bf6e26e98b14566899578a990b27d47 (127.19.226.193) to 88935ab835544e9889d05a6587d8cf21 (127.19.226.196). New cstate: current_term: 2 leader_uuid: "88935ab835544e9889d05a6587d8cf21" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250109 08:57:59.235793 22693 tablet_copy_client.cc:806] T 206785f29c824cddbe9bbc00d365ffb3 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Starting download of 0 data blocks...
I20250109 08:57:59.236361 22693 tablet_copy_client.cc:670] T 206785f29c824cddbe9bbc00d365ffb3 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Starting download of 1 WAL segments...
I20250109 08:57:59.253058 22691 tablet_copy_client.cc:538] T 180b4d539a144a38905bf340ecb24ef4 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250109 08:57:59.256171 21412 catalog_manager.cc:5526] T df9715570a6844d9a2686307db16eff1 P 88935ab835544e9889d05a6587d8cf21 reported cstate change: term changed from 1 to 2, leader changed from 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195) to 88935ab835544e9889d05a6587d8cf21 (127.19.226.196). New cstate: current_term: 2 leader_uuid: "88935ab835544e9889d05a6587d8cf21" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250109 08:57:59.257858 21412 catalog_manager.cc:5526] T b00671c87aac405a81c6543f4defd722 P 88935ab835544e9889d05a6587d8cf21 reported cstate change: term changed from 1 to 2, leader changed from 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195) to 88935ab835544e9889d05a6587d8cf21 (127.19.226.196). New cstate: current_term: 2 leader_uuid: "88935ab835544e9889d05a6587d8cf21" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250109 08:57:59.259105 21412 catalog_manager.cc:5526] T d52f79ecf037457cbcc09d5fa43456cf P 88935ab835544e9889d05a6587d8cf21 reported cstate change: term changed from 1 to 2, leader changed from 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195) to 88935ab835544e9889d05a6587d8cf21 (127.19.226.196). New cstate: current_term: 2 leader_uuid: "88935ab835544e9889d05a6587d8cf21" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250109 08:57:59.260568 21412 catalog_manager.cc:5526] T 20ea30dba4314290ae45d9b4c13bebdd P 88935ab835544e9889d05a6587d8cf21 reported cstate change: term changed from 1 to 2, leader changed from 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195) to 88935ab835544e9889d05a6587d8cf21 (127.19.226.196). New cstate: current_term: 2 leader_uuid: "88935ab835544e9889d05a6587d8cf21" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250109 08:57:59.266474 22693 tablet_copy_client.cc:538] T 206785f29c824cddbe9bbc00d365ffb3 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250109 08:57:59.270234 22691 tablet_bootstrap.cc:492] T 180b4d539a144a38905bf340ecb24ef4 P 88935ab835544e9889d05a6587d8cf21: Bootstrap starting.
I20250109 08:57:59.276736 22693 tablet_bootstrap.cc:492] T 206785f29c824cddbe9bbc00d365ffb3 P 88935ab835544e9889d05a6587d8cf21: Bootstrap starting.
I20250109 08:57:59.318167 22691 tablet_bootstrap.cc:492] T 180b4d539a144a38905bf340ecb24ef4 P 88935ab835544e9889d05a6587d8cf21: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250109 08:57:59.323079 22691 tablet_bootstrap.cc:492] T 180b4d539a144a38905bf340ecb24ef4 P 88935ab835544e9889d05a6587d8cf21: Bootstrap complete.
I20250109 08:57:59.323908 22691 ts_tablet_manager.cc:1397] T 180b4d539a144a38905bf340ecb24ef4 P 88935ab835544e9889d05a6587d8cf21: Time spent bootstrapping tablet: real 0.054s user 0.015s sys 0.015s
I20250109 08:57:59.327819 22691 raft_consensus.cc:357] T 180b4d539a144a38905bf340ecb24ef4 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:57:59.330180 22691 raft_consensus.cc:738] T 180b4d539a144a38905bf340ecb24ef4 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 88935ab835544e9889d05a6587d8cf21, State: Initialized, Role: LEARNER
I20250109 08:57:59.330860 22691 consensus_queue.cc:260] T 180b4d539a144a38905bf340ecb24ef4 P 88935ab835544e9889d05a6587d8cf21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:57:59.329859 22706 ts_tablet_manager.cc:927] T 64c667f887ea41b4a484ee565b21fa16 P 88935ab835544e9889d05a6587d8cf21: Initiating tablet copy from peer 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:57:59.333762 22693 tablet_bootstrap.cc:492] T 206785f29c824cddbe9bbc00d365ffb3 P 88935ab835544e9889d05a6587d8cf21: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250109 08:57:59.334662 22693 tablet_bootstrap.cc:492] T 206785f29c824cddbe9bbc00d365ffb3 P 88935ab835544e9889d05a6587d8cf21: Bootstrap complete.
I20250109 08:57:59.335350 22693 ts_tablet_manager.cc:1397] T 206785f29c824cddbe9bbc00d365ffb3 P 88935ab835544e9889d05a6587d8cf21: Time spent bootstrapping tablet: real 0.059s user 0.019s sys 0.008s
I20250109 08:57:59.342581 22691 ts_tablet_manager.cc:1428] T 180b4d539a144a38905bf340ecb24ef4 P 88935ab835544e9889d05a6587d8cf21: Time spent starting tablet: real 0.018s user 0.006s sys 0.000s
I20250109 08:57:59.340157 22693 raft_consensus.cc:357] T 206785f29c824cddbe9bbc00d365ffb3 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:57:59.343499 22693 raft_consensus.cc:738] T 206785f29c824cddbe9bbc00d365ffb3 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 88935ab835544e9889d05a6587d8cf21, State: Initialized, Role: LEARNER
I20250109 08:57:59.344473 22693 consensus_queue.cc:260] T 206785f29c824cddbe9bbc00d365ffb3 P 88935ab835544e9889d05a6587d8cf21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:57:59.351790 22706 tablet_copy_client.cc:323] T 64c667f887ea41b4a484ee565b21fa16 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Beginning tablet copy session from remote peer at address 127.19.226.195:35731
I20250109 08:57:59.354045 22693 ts_tablet_manager.cc:1428] T 206785f29c824cddbe9bbc00d365ffb3 P 88935ab835544e9889d05a6587d8cf21: Time spent starting tablet: real 0.017s user 0.007s sys 0.000s
I20250109 08:57:59.354013 21524 tablet_copy_service.cc:342] P 8bf6e26e98b14566899578a990b27d47: Request end of tablet copy session 88935ab835544e9889d05a6587d8cf21-180b4d539a144a38905bf340ecb24ef4 received from {username='slave'} at 127.0.0.1:41812
I20250109 08:57:59.358438 21524 tablet_copy_service.cc:434] P 8bf6e26e98b14566899578a990b27d47: ending tablet copy session 88935ab835544e9889d05a6587d8cf21-180b4d539a144a38905bf340ecb24ef4 on tablet 180b4d539a144a38905bf340ecb24ef4 with peer 88935ab835544e9889d05a6587d8cf21
I20250109 08:57:59.359436 21523 tablet_copy_service.cc:342] P 8bf6e26e98b14566899578a990b27d47: Request end of tablet copy session 88935ab835544e9889d05a6587d8cf21-206785f29c824cddbe9bbc00d365ffb3 received from {username='slave'} at 127.0.0.1:41812
I20250109 08:57:59.359822 21523 tablet_copy_service.cc:434] P 8bf6e26e98b14566899578a990b27d47: ending tablet copy session 88935ab835544e9889d05a6587d8cf21-206785f29c824cddbe9bbc00d365ffb3 on tablet 206785f29c824cddbe9bbc00d365ffb3 with peer 88935ab835544e9889d05a6587d8cf21
I20250109 08:57:59.362126 21657 raft_consensus.cc:1270] T b00671c87aac405a81c6543f4defd722 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Refusing update from remote peer 88935ab835544e9889d05a6587d8cf21: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250109 08:57:59.362195 21673 tablet_copy_service.cc:140] P 787ddcc824c2490a808fc5a53452dc5e: Received BeginTabletCopySession request for tablet 64c667f887ea41b4a484ee565b21fa16 from peer 88935ab835544e9889d05a6587d8cf21 ({username='slave'} at 127.0.0.1:52376)
I20250109 08:57:59.363238 21673 tablet_copy_service.cc:161] P 787ddcc824c2490a808fc5a53452dc5e: Beginning new tablet copy session on tablet 64c667f887ea41b4a484ee565b21fa16 from peer 88935ab835544e9889d05a6587d8cf21 at {username='slave'} at 127.0.0.1:52376: session id = 88935ab835544e9889d05a6587d8cf21-64c667f887ea41b4a484ee565b21fa16
I20250109 08:57:59.363881 22679 consensus_queue.cc:1035] T b00671c87aac405a81c6543f4defd722 P 88935ab835544e9889d05a6587d8cf21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250109 08:57:59.375222 21673 tablet_copy_source_session.cc:215] T 64c667f887ea41b4a484ee565b21fa16 P 787ddcc824c2490a808fc5a53452dc5e: Tablet Copy: opened 0 blocks and 1 log segments
I20250109 08:57:59.381575 22706 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 64c667f887ea41b4a484ee565b21fa16. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:59.403677 22706 tablet_copy_client.cc:806] T 64c667f887ea41b4a484ee565b21fa16 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Starting download of 0 data blocks...
I20250109 08:57:59.404386 22706 tablet_copy_client.cc:670] T 64c667f887ea41b4a484ee565b21fa16 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Starting download of 1 WAL segments...
I20250109 08:57:59.405355 21514 raft_consensus.cc:1270] T 2935d69446cd458e92046d2b2f1b74ab P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Refusing update from remote peer 88935ab835544e9889d05a6587d8cf21: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250109 08:57:59.408607 22681 consensus_queue.cc:1035] T 2935d69446cd458e92046d2b2f1b74ab P 88935ab835544e9889d05a6587d8cf21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250109 08:57:59.415002 21589 raft_consensus.cc:1270] T b00671c87aac405a81c6543f4defd722 P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Refusing update from remote peer 88935ab835544e9889d05a6587d8cf21: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250109 08:57:59.416589 22681 consensus_queue.cc:1035] T b00671c87aac405a81c6543f4defd722 P 88935ab835544e9889d05a6587d8cf21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250109 08:57:59.431213 21588 raft_consensus.cc:1270] T 2935d69446cd458e92046d2b2f1b74ab P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Refusing update from remote peer 88935ab835544e9889d05a6587d8cf21: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250109 08:57:59.432624 22681 consensus_queue.cc:1035] T 2935d69446cd458e92046d2b2f1b74ab P 88935ab835544e9889d05a6587d8cf21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250109 08:57:59.449534 22177 raft_consensus.cc:1212] T 206785f29c824cddbe9bbc00d365ffb3 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20250109 08:57:59.454274 22706 tablet_copy_client.cc:538] T 64c667f887ea41b4a484ee565b21fa16 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250109 08:57:59.470134 22706 tablet_bootstrap.cc:492] T 64c667f887ea41b4a484ee565b21fa16 P 88935ab835544e9889d05a6587d8cf21: Bootstrap starting.
I20250109 08:57:59.504066 22177 raft_consensus.cc:1212] T 180b4d539a144a38905bf340ecb24ef4 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20250109 08:57:59.510651 21663 raft_consensus.cc:1270] T d52f79ecf037457cbcc09d5fa43456cf P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Refusing update from remote peer 88935ab835544e9889d05a6587d8cf21: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250109 08:57:59.523630 22685 consensus_queue.cc:1035] T d52f79ecf037457cbcc09d5fa43456cf P 88935ab835544e9889d05a6587d8cf21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250109 08:57:59.526249 22706 tablet_bootstrap.cc:492] T 64c667f887ea41b4a484ee565b21fa16 P 88935ab835544e9889d05a6587d8cf21: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250109 08:57:59.527227 22706 tablet_bootstrap.cc:492] T 64c667f887ea41b4a484ee565b21fa16 P 88935ab835544e9889d05a6587d8cf21: Bootstrap complete.
I20250109 08:57:59.529693 21663 raft_consensus.cc:1270] T df9715570a6844d9a2686307db16eff1 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Refusing update from remote peer 88935ab835544e9889d05a6587d8cf21: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250109 08:57:59.530788 22693 ts_tablet_manager.cc:927] T e601a034e10e418bac8683443d86a1c1 P 88935ab835544e9889d05a6587d8cf21: Initiating tablet copy from peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725)
I20250109 08:57:59.537329 22685 consensus_queue.cc:1035] T df9715570a6844d9a2686307db16eff1 P 88935ab835544e9889d05a6587d8cf21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250109 08:57:59.544405 22706 ts_tablet_manager.cc:1397] T 64c667f887ea41b4a484ee565b21fa16 P 88935ab835544e9889d05a6587d8cf21: Time spent bootstrapping tablet: real 0.074s user 0.022s sys 0.006s
I20250109 08:57:59.550839 22706 raft_consensus.cc:357] T 64c667f887ea41b4a484ee565b21fa16 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:57:59.552842 22706 raft_consensus.cc:738] T 64c667f887ea41b4a484ee565b21fa16 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 88935ab835544e9889d05a6587d8cf21, State: Initialized, Role: LEARNER
I20250109 08:57:59.554301 22706 consensus_queue.cc:260] T 64c667f887ea41b4a484ee565b21fa16 P 88935ab835544e9889d05a6587d8cf21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:57:59.558292 22706 ts_tablet_manager.cc:1428] T 64c667f887ea41b4a484ee565b21fa16 P 88935ab835544e9889d05a6587d8cf21: Time spent starting tablet: real 0.012s user 0.004s sys 0.001s
I20250109 08:57:59.561846 21673 tablet_copy_service.cc:342] P 787ddcc824c2490a808fc5a53452dc5e: Request end of tablet copy session 88935ab835544e9889d05a6587d8cf21-64c667f887ea41b4a484ee565b21fa16 received from {username='slave'} at 127.0.0.1:52376
I20250109 08:57:59.562297 21673 tablet_copy_service.cc:434] P 787ddcc824c2490a808fc5a53452dc5e: ending tablet copy session 88935ab835544e9889d05a6587d8cf21-64c667f887ea41b4a484ee565b21fa16 on tablet 64c667f887ea41b4a484ee565b21fa16 with peer 88935ab835544e9889d05a6587d8cf21
I20250109 08:57:59.564318 22693 tablet_copy_client.cc:323] T e601a034e10e418bac8683443d86a1c1 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Beginning tablet copy session from remote peer at address 127.19.226.193:42725
I20250109 08:57:59.565840 21523 tablet_copy_service.cc:140] P 8bf6e26e98b14566899578a990b27d47: Received BeginTabletCopySession request for tablet e601a034e10e418bac8683443d86a1c1 from peer 88935ab835544e9889d05a6587d8cf21 ({username='slave'} at 127.0.0.1:41812)
I20250109 08:57:59.566375 21523 tablet_copy_service.cc:161] P 8bf6e26e98b14566899578a990b27d47: Beginning new tablet copy session on tablet e601a034e10e418bac8683443d86a1c1 from peer 88935ab835544e9889d05a6587d8cf21 at {username='slave'} at 127.0.0.1:41812: session id = 88935ab835544e9889d05a6587d8cf21-e601a034e10e418bac8683443d86a1c1
I20250109 08:57:59.574048 21511 raft_consensus.cc:1270] T df9715570a6844d9a2686307db16eff1 P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Refusing update from remote peer 88935ab835544e9889d05a6587d8cf21: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250109 08:57:59.575327 22685 consensus_queue.cc:1035] T df9715570a6844d9a2686307db16eff1 P 88935ab835544e9889d05a6587d8cf21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250109 08:57:59.577716 21523 tablet_copy_source_session.cc:215] T e601a034e10e418bac8683443d86a1c1 P 8bf6e26e98b14566899578a990b27d47: Tablet Copy: opened 0 blocks and 1 log segments
I20250109 08:57:59.590118 22693 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e601a034e10e418bac8683443d86a1c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:57:59.622291 22693 tablet_copy_client.cc:806] T e601a034e10e418bac8683443d86a1c1 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Starting download of 0 data blocks...
I20250109 08:57:59.625097 22693 tablet_copy_client.cc:670] T e601a034e10e418bac8683443d86a1c1 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Starting download of 1 WAL segments...
I20250109 08:57:59.630851 21514 raft_consensus.cc:1270] T 20ea30dba4314290ae45d9b4c13bebdd P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Refusing update from remote peer 88935ab835544e9889d05a6587d8cf21: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250109 08:57:59.640058 22679 consensus_queue.cc:1035] T 20ea30dba4314290ae45d9b4c13bebdd P 88935ab835544e9889d05a6587d8cf21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250109 08:57:59.643316 22693 tablet_copy_client.cc:538] T e601a034e10e418bac8683443d86a1c1 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250109 08:57:59.644901 21588 raft_consensus.cc:1270] T d52f79ecf037457cbcc09d5fa43456cf P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Refusing update from remote peer 88935ab835544e9889d05a6587d8cf21: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250109 08:57:59.653168 21656 raft_consensus.cc:1270] T 20ea30dba4314290ae45d9b4c13bebdd P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Refusing update from remote peer 88935ab835544e9889d05a6587d8cf21: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250109 08:57:59.653784 22689 consensus_queue.cc:1035] T d52f79ecf037457cbcc09d5fa43456cf P 88935ab835544e9889d05a6587d8cf21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20250109 08:57:59.665055 22689 consensus_queue.cc:1035] T 20ea30dba4314290ae45d9b4c13bebdd P 88935ab835544e9889d05a6587d8cf21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250109 08:57:59.682586 22693 tablet_bootstrap.cc:492] T e601a034e10e418bac8683443d86a1c1 P 88935ab835544e9889d05a6587d8cf21: Bootstrap starting.
I20250109 08:57:59.791304 22693 tablet_bootstrap.cc:492] T e601a034e10e418bac8683443d86a1c1 P 88935ab835544e9889d05a6587d8cf21: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250109 08:57:59.792119 22693 tablet_bootstrap.cc:492] T e601a034e10e418bac8683443d86a1c1 P 88935ab835544e9889d05a6587d8cf21: Bootstrap complete.
I20250109 08:57:59.792747 22693 ts_tablet_manager.cc:1397] T e601a034e10e418bac8683443d86a1c1 P 88935ab835544e9889d05a6587d8cf21: Time spent bootstrapping tablet: real 0.110s user 0.027s sys 0.001s
I20250109 08:57:59.795450 22693 raft_consensus.cc:357] T e601a034e10e418bac8683443d86a1c1 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:57:59.796274 22693 raft_consensus.cc:738] T e601a034e10e418bac8683443d86a1c1 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 88935ab835544e9889d05a6587d8cf21, State: Initialized, Role: LEARNER
I20250109 08:57:59.796937 22693 consensus_queue.cc:260] T e601a034e10e418bac8683443d86a1c1 P 88935ab835544e9889d05a6587d8cf21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:57:59.799470 22693 ts_tablet_manager.cc:1428] T e601a034e10e418bac8683443d86a1c1 P 88935ab835544e9889d05a6587d8cf21: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20250109 08:57:59.800856 21523 tablet_copy_service.cc:342] P 8bf6e26e98b14566899578a990b27d47: Request end of tablet copy session 88935ab835544e9889d05a6587d8cf21-e601a034e10e418bac8683443d86a1c1 received from {username='slave'} at 127.0.0.1:41812
I20250109 08:57:59.801254 21523 tablet_copy_service.cc:434] P 8bf6e26e98b14566899578a990b27d47: ending tablet copy session 88935ab835544e9889d05a6587d8cf21-e601a034e10e418bac8683443d86a1c1 on tablet e601a034e10e418bac8683443d86a1c1 with peer 88935ab835544e9889d05a6587d8cf21
I20250109 08:57:59.888686 22174 raft_consensus.cc:1212] T e601a034e10e418bac8683443d86a1c1 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20250109 08:57:59.914739 22174 raft_consensus.cc:1212] T 64c667f887ea41b4a484ee565b21fa16 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20250109 08:57:59.999847 20363 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250109 08:58:00.001026 22721 raft_consensus.cc:1059] T 206785f29c824cddbe9bbc00d365ffb3 P 8bf6e26e98b14566899578a990b27d47: attempting to promote NON_VOTER 88935ab835544e9889d05a6587d8cf21 to VOTER
I20250109 08:58:00.008718 22721 consensus_queue.cc:237] T 206785f29c824cddbe9bbc00d365ffb3 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:00.045774 22174 raft_consensus.cc:1270] T 206785f29c824cddbe9bbc00d365ffb3 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Refusing update from remote peer 8bf6e26e98b14566899578a990b27d47: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:58:00.047816 21583 raft_consensus.cc:1270] T 206785f29c824cddbe9bbc00d365ffb3 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Refusing update from remote peer 8bf6e26e98b14566899578a990b27d47: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:58:00.050089 22660 consensus_queue.cc:1035] T 206785f29c824cddbe9bbc00d365ffb3 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20250109 08:58:00.052331 22678 consensus_queue.cc:1035] T 206785f29c824cddbe9bbc00d365ffb3 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:58:00.062223 22721 raft_consensus.cc:1059] T 180b4d539a144a38905bf340ecb24ef4 P 8bf6e26e98b14566899578a990b27d47: attempting to promote NON_VOTER 88935ab835544e9889d05a6587d8cf21 to VOTER
I20250109 08:58:00.064554 22721 consensus_queue.cc:237] T 180b4d539a144a38905bf340ecb24ef4 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:00.072698 21656 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "5717a02ef53a4b22a96f431ca6c20d93"
dest_uuid: "787ddcc824c2490a808fc5a53452dc5e"
mode: GRACEFUL
new_leader_uuid: "88935ab835544e9889d05a6587d8cf21"
from {username='slave'} at 127.0.0.1:55054
I20250109 08:58:00.073364 21656 raft_consensus.cc:604] T 5717a02ef53a4b22a96f431ca6c20d93 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Received request to transfer leadership to TS 88935ab835544e9889d05a6587d8cf21
I20250109 08:58:00.048027 21657 raft_consensus.cc:1270] T 206785f29c824cddbe9bbc00d365ffb3 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Refusing update from remote peer 8bf6e26e98b14566899578a990b27d47: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:58:00.075875 21657 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "610441ef4b1d40c284eff3a714e4214e"
dest_uuid: "787ddcc824c2490a808fc5a53452dc5e"
mode: GRACEFUL
new_leader_uuid: "a2b1a692435241ee988e0bd1661cd4c3"
from {username='slave'} at 127.0.0.1:55054
I20250109 08:58:00.076447 21657 raft_consensus.cc:604] T 610441ef4b1d40c284eff3a714e4214e P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Received request to transfer leadership to TS a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:58:00.079605 22678 consensus_queue.cc:1035] T 206785f29c824cddbe9bbc00d365ffb3 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:58:00.087021 21663 raft_consensus.cc:1270] T 180b4d539a144a38905bf340ecb24ef4 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Refusing update from remote peer 8bf6e26e98b14566899578a990b27d47: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:58:00.088425 22661 consensus_queue.cc:1035] T 180b4d539a144a38905bf340ecb24ef4 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:58:00.088972 22174 raft_consensus.cc:1270] T 180b4d539a144a38905bf340ecb24ef4 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Refusing update from remote peer 8bf6e26e98b14566899578a990b27d47: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:58:00.090903 21583 raft_consensus.cc:1270] T 180b4d539a144a38905bf340ecb24ef4 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Refusing update from remote peer 8bf6e26e98b14566899578a990b27d47: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:58:00.092588 21514 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "80123146836349938344efc61333695f"
dest_uuid: "8bf6e26e98b14566899578a990b27d47"
mode: GRACEFUL
new_leader_uuid: "787ddcc824c2490a808fc5a53452dc5e"
from {username='slave'} at 127.0.0.1:39274
I20250109 08:58:00.093096 21514 raft_consensus.cc:604] T 80123146836349938344efc61333695f P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Received request to transfer leadership to TS 787ddcc824c2490a808fc5a53452dc5e
I20250109 08:58:00.093699 22678 raft_consensus.cc:2949] T 206785f29c824cddbe9bbc00d365ffb3 P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:00.100312 21663 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "81d74e7d4472497aac5463a4b47e4fb9"
dest_uuid: "787ddcc824c2490a808fc5a53452dc5e"
mode: GRACEFUL
new_leader_uuid: "88935ab835544e9889d05a6587d8cf21"
from {username='slave'} at 127.0.0.1:55054
I20250109 08:58:00.100919 21663 raft_consensus.cc:604] T 81d74e7d4472497aac5463a4b47e4fb9 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Received request to transfer leadership to TS 88935ab835544e9889d05a6587d8cf21
I20250109 08:58:00.099820 21583 raft_consensus.cc:2949] T 206785f29c824cddbe9bbc00d365ffb3 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:00.106760 22721 consensus_queue.cc:1035] T 180b4d539a144a38905bf340ecb24ef4 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20250109 08:58:00.115417 22721 consensus_queue.cc:1035] T 180b4d539a144a38905bf340ecb24ef4 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20250109 08:58:00.118060 21663 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "c18c1d46a6ca47598d7b161ed70a4e7a"
dest_uuid: "787ddcc824c2490a808fc5a53452dc5e"
mode: GRACEFUL
new_leader_uuid: "a2b1a692435241ee988e0bd1661cd4c3"
from {username='slave'} at 127.0.0.1:55054
I20250109 08:58:00.129834 21663 raft_consensus.cc:604] T c18c1d46a6ca47598d7b161ed70a4e7a P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Received request to transfer leadership to TS a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:58:00.132442 21663 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "d015a10f95eb4f19b6d7c53cb201fa80"
dest_uuid: "787ddcc824c2490a808fc5a53452dc5e"
mode: GRACEFUL
new_leader_uuid: "88935ab835544e9889d05a6587d8cf21"
from {username='slave'} at 127.0.0.1:55054
I20250109 08:58:00.132989 21663 raft_consensus.cc:604] T d015a10f95eb4f19b6d7c53cb201fa80 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Received request to transfer leadership to TS 88935ab835544e9889d05a6587d8cf21
I20250109 08:58:00.133996 20363 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 6
I20250109 08:58:00.135579 20363 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250109 08:58:00.136184 20363 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250109 08:58:00.147289 22649 raft_consensus.cc:988] T 610441ef4b1d40c284eff3a714e4214e P 787ddcc824c2490a808fc5a53452dc5e: : Instructing follower a2b1a692435241ee988e0bd1661cd4c3 to start an election
I20250109 08:58:00.147733 22649 raft_consensus.cc:1076] T 610441ef4b1d40c284eff3a714e4214e P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Signalling peer a2b1a692435241ee988e0bd1661cd4c3 to start an election
I20250109 08:58:00.149461 21584 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "610441ef4b1d40c284eff3a714e4214e"
dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3"
from {username='slave'} at 127.0.0.1:34746
I20250109 08:58:00.149978 21584 raft_consensus.cc:491] T 610441ef4b1d40c284eff3a714e4214e P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:58:00.150270 21584 raft_consensus.cc:3054] T 610441ef4b1d40c284eff3a714e4214e P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:00.153071 21663 raft_consensus.cc:2949] T 206785f29c824cddbe9bbc00d365ffb3 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:00.156615 21584 raft_consensus.cc:513] T 610441ef4b1d40c284eff3a714e4214e P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:58:00.158605 21584 leader_election.cc:290] T 610441ef4b1d40c284eff3a714e4214e P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 2 election: Requested vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:58:00.160526 21656 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "610441ef4b1d40c284eff3a714e4214e" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "787ddcc824c2490a808fc5a53452dc5e"
I20250109 08:58:00.161091 21656 raft_consensus.cc:3049] T 610441ef4b1d40c284eff3a714e4214e P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Stepping down as leader of term 1
I20250109 08:58:00.160995 21512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "610441ef4b1d40c284eff3a714e4214e" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8bf6e26e98b14566899578a990b27d47"
I20250109 08:58:00.161455 21656 raft_consensus.cc:738] T 610441ef4b1d40c284eff3a714e4214e P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Running, Role: LEADER
I20250109 08:58:00.162549 21512 raft_consensus.cc:3054] T 610441ef4b1d40c284eff3a714e4214e P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:00.163250 21656 consensus_queue.cc:260] T 610441ef4b1d40c284eff3a714e4214e P 787ddcc824c2490a808fc5a53452dc5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:58:00.164624 21656 raft_consensus.cc:3054] T 610441ef4b1d40c284eff3a714e4214e P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:00.170166 21512 raft_consensus.cc:2463] T 610441ef4b1d40c284eff3a714e4214e P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2b1a692435241ee988e0bd1661cd4c3 in term 2.
I20250109 08:58:00.172847 21552 leader_election.cc:304] T 610441ef4b1d40c284eff3a714e4214e P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3; no voters:
I20250109 08:58:00.174098 22722 raft_consensus.cc:2798] T 610441ef4b1d40c284eff3a714e4214e P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:58:00.174574 22722 raft_consensus.cc:695] T 610441ef4b1d40c284eff3a714e4214e P a2b1a692435241ee988e0bd1661cd4c3 [term 2 LEADER]: Becoming Leader. State: Replica: a2b1a692435241ee988e0bd1661cd4c3, State: Running, Role: LEADER
I20250109 08:58:00.174894 21656 raft_consensus.cc:2463] T 610441ef4b1d40c284eff3a714e4214e P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2b1a692435241ee988e0bd1661cd4c3 in term 2.
I20250109 08:58:00.175432 22722 consensus_queue.cc:237] T 610441ef4b1d40c284eff3a714e4214e P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:58:00.208645 21410 catalog_manager.cc:5526] T 206785f29c824cddbe9bbc00d365ffb3 P 787ddcc824c2490a808fc5a53452dc5e reported cstate change: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8bf6e26e98b14566899578a990b27d47" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:00.274387 22649 raft_consensus.cc:988] T 5717a02ef53a4b22a96f431ca6c20d93 P 787ddcc824c2490a808fc5a53452dc5e: : Instructing follower 88935ab835544e9889d05a6587d8cf21 to start an election
I20250109 08:58:00.274899 21409 catalog_manager.cc:5526] T 610441ef4b1d40c284eff3a714e4214e P a2b1a692435241ee988e0bd1661cd4c3 reported cstate change: term changed from 1 to 2, leader changed from 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195) to a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194). New cstate: current_term: 2 leader_uuid: "a2b1a692435241ee988e0bd1661cd4c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:00.306676 22665 raft_consensus.cc:1076] T 5717a02ef53a4b22a96f431ca6c20d93 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Signalling peer 88935ab835544e9889d05a6587d8cf21 to start an election
W20250109 08:58:00.360260 22732 log.cc:926] Time spent T 610441ef4b1d40c284eff3a714e4214e P a2b1a692435241ee988e0bd1661cd4c3: Append to log took a long time: real 0.083s user 0.001s sys 0.000s
W20250109 08:58:00.400640 21478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.226.194:45527, user_credentials={real_user=slave}} blocked reactor thread for 148280us
W20250109 08:58:00.429435 21477 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.226.195:35731, user_credentials={real_user=slave}} blocked reactor thread for 147442us
I20250109 08:58:00.433380 22721 raft_consensus.cc:988] T 80123146836349938344efc61333695f P 8bf6e26e98b14566899578a990b27d47: : Instructing follower 787ddcc824c2490a808fc5a53452dc5e to start an election
I20250109 08:58:00.434437 22739 raft_consensus.cc:1076] T 80123146836349938344efc61333695f P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Signalling peer 787ddcc824c2490a808fc5a53452dc5e to start an election
W20250109 08:58:00.436748 21627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.226.194:45527, user_credentials={real_user=slave}} blocked reactor thread for 126480us
I20250109 08:58:00.438246 22174 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "5717a02ef53a4b22a96f431ca6c20d93"
dest_uuid: "88935ab835544e9889d05a6587d8cf21"
from {username='slave'} at 127.0.0.1:60458
I20250109 08:58:00.438829 22174 raft_consensus.cc:491] T 5717a02ef53a4b22a96f431ca6c20d93 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:58:00.439172 22174 raft_consensus.cc:3054] T 5717a02ef53a4b22a96f431ca6c20d93 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:00.446349 22174 raft_consensus.cc:513] T 5717a02ef53a4b22a96f431ca6c20d93 P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:00.448908 22174 leader_election.cc:290] T 5717a02ef53a4b22a96f431ca6c20d93 P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 election: Requested vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:58:00.449827 21657 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5717a02ef53a4b22a96f431ca6c20d93" candidate_uuid: "88935ab835544e9889d05a6587d8cf21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "787ddcc824c2490a808fc5a53452dc5e"
I20250109 08:58:00.450363 21657 raft_consensus.cc:3049] T 5717a02ef53a4b22a96f431ca6c20d93 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Stepping down as leader of term 1
I20250109 08:58:00.450634 21657 raft_consensus.cc:738] T 5717a02ef53a4b22a96f431ca6c20d93 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Running, Role: LEADER
I20250109 08:58:00.451308 21657 consensus_queue.cc:260] T 5717a02ef53a4b22a96f431ca6c20d93 P 787ddcc824c2490a808fc5a53452dc5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:00.452335 21657 raft_consensus.cc:3054] T 5717a02ef53a4b22a96f431ca6c20d93 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:00.458952 21657 raft_consensus.cc:2463] T 5717a02ef53a4b22a96f431ca6c20d93 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88935ab835544e9889d05a6587d8cf21 in term 2.
I20250109 08:58:00.461577 22141 leader_election.cc:304] T 5717a02ef53a4b22a96f431ca6c20d93 P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, 88935ab835544e9889d05a6587d8cf21; no voters:
I20250109 08:58:00.464541 22689 raft_consensus.cc:2798] T 5717a02ef53a4b22a96f431ca6c20d93 P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:58:00.465015 22689 raft_consensus.cc:695] T 5717a02ef53a4b22a96f431ca6c20d93 P 88935ab835544e9889d05a6587d8cf21 [term 2 LEADER]: Becoming Leader. State: Replica: 88935ab835544e9889d05a6587d8cf21, State: Running, Role: LEADER
I20250109 08:58:00.465838 22689 consensus_queue.cc:237] T 5717a02ef53a4b22a96f431ca6c20d93 P 88935ab835544e9889d05a6587d8cf21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:00.471436 22177 raft_consensus.cc:2949] T 206785f29c824cddbe9bbc00d365ffb3 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:00.490413 21412 catalog_manager.cc:5526] T 5717a02ef53a4b22a96f431ca6c20d93 P 88935ab835544e9889d05a6587d8cf21 reported cstate change: term changed from 1 to 2, leader changed from 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195) to 88935ab835544e9889d05a6587d8cf21 (127.19.226.196). New cstate: current_term: 2 leader_uuid: "88935ab835544e9889d05a6587d8cf21" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250109 08:58:00.496104 22740 raft_consensus.cc:2949] T 180b4d539a144a38905bf340ecb24ef4 P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:00.498641 22721 raft_consensus.cc:1059] T e601a034e10e418bac8683443d86a1c1 P 8bf6e26e98b14566899578a990b27d47: attempting to promote NON_VOTER 88935ab835544e9889d05a6587d8cf21 to VOTER
I20250109 08:58:00.500690 22721 consensus_queue.cc:237] T e601a034e10e418bac8683443d86a1c1 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:00.505905 21511 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5717a02ef53a4b22a96f431ca6c20d93" candidate_uuid: "88935ab835544e9889d05a6587d8cf21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "8bf6e26e98b14566899578a990b27d47"
I20250109 08:58:00.506498 21511 raft_consensus.cc:3054] T 5717a02ef53a4b22a96f431ca6c20d93 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:00.509179 22665 raft_consensus.cc:1059] T 64c667f887ea41b4a484ee565b21fa16 P 787ddcc824c2490a808fc5a53452dc5e: attempting to promote NON_VOTER 88935ab835544e9889d05a6587d8cf21 to VOTER
I20250109 08:58:00.511479 22665 consensus_queue.cc:237] T 64c667f887ea41b4a484ee565b21fa16 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:00.516552 21663 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "80123146836349938344efc61333695f"
dest_uuid: "787ddcc824c2490a808fc5a53452dc5e"
from {username='slave'} at 127.0.0.1:58440
I20250109 08:58:00.517081 21663 raft_consensus.cc:491] T 80123146836349938344efc61333695f P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:58:00.517412 21663 raft_consensus.cc:3054] T 80123146836349938344efc61333695f P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:00.534914 22174 raft_consensus.cc:2949] T 180b4d539a144a38905bf340ecb24ef4 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:00.538064 21514 raft_consensus.cc:1270] T 64c667f887ea41b4a484ee565b21fa16 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:58:00.540452 22649 consensus_queue.cc:1035] T 64c667f887ea41b4a484ee565b21fa16 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:58:00.542228 22177 raft_consensus.cc:1270] T e601a034e10e418bac8683443d86a1c1 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Refusing update from remote peer 8bf6e26e98b14566899578a990b27d47: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:58:00.542229 21589 raft_consensus.cc:2949] T 180b4d539a144a38905bf340ecb24ef4 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:00.545384 21663 raft_consensus.cc:513] T 80123146836349938344efc61333695f P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:58:00.546895 21583 raft_consensus.cc:1270] T e601a034e10e418bac8683443d86a1c1 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Refusing update from remote peer 8bf6e26e98b14566899578a990b27d47: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:58:00.547945 21511 raft_consensus.cc:2463] T 5717a02ef53a4b22a96f431ca6c20d93 P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88935ab835544e9889d05a6587d8cf21 in term 2.
I20250109 08:58:00.548352 21513 consensus_queue.cc:237] T 206785f29c824cddbe9bbc00d365ffb3 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:00.563915 22740 consensus_queue.cc:1035] T e601a034e10e418bac8683443d86a1c1 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:58:00.569057 22177 raft_consensus.cc:1270] T 206785f29c824cddbe9bbc00d365ffb3 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Refusing update from remote peer 8bf6e26e98b14566899578a990b27d47: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250109 08:58:00.570128 21663 leader_election.cc:290] T 80123146836349938344efc61333695f P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 2 election: Requested vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
I20250109 08:58:00.571363 21512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80123146836349938344efc61333695f" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8bf6e26e98b14566899578a990b27d47"
I20250109 08:58:00.576205 21512 raft_consensus.cc:3049] T 80123146836349938344efc61333695f P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Stepping down as leader of term 1
I20250109 08:58:00.576584 22740 consensus_queue.cc:1035] T e601a034e10e418bac8683443d86a1c1 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:58:00.576593 21512 raft_consensus.cc:738] T 80123146836349938344efc61333695f P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8bf6e26e98b14566899578a990b27d47, State: Running, Role: LEADER
I20250109 08:58:00.579489 21512 consensus_queue.cc:260] T 80123146836349938344efc61333695f P 8bf6e26e98b14566899578a990b27d47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:58:00.574501 21409 catalog_manager.cc:5526] T 180b4d539a144a38905bf340ecb24ef4 P 8bf6e26e98b14566899578a990b27d47 reported cstate change: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8bf6e26e98b14566899578a990b27d47" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250109 08:58:00.584977 21512 raft_consensus.cc:3054] T 80123146836349938344efc61333695f P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:00.590291 21663 raft_consensus.cc:1270] T 610441ef4b1d40c284eff3a714e4214e P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Refusing update from remote peer a2b1a692435241ee988e0bd1661cd4c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:58:00.591991 22743 consensus_queue.cc:1035] T 610441ef4b1d40c284eff3a714e4214e P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:58:00.592893 22737 consensus_queue.cc:1035] T 206785f29c824cddbe9bbc00d365ffb3 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250109 08:58:00.608561 22740 raft_consensus.cc:2949] T 206785f29c824cddbe9bbc00d365ffb3 P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:00.612886 21512 raft_consensus.cc:2463] T 80123146836349938344efc61333695f P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 2.
I20250109 08:58:00.619448 21626 leader_election.cc:304] T 80123146836349938344efc61333695f P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, 8bf6e26e98b14566899578a990b27d47; no voters:
W20250109 08:58:00.621711 21627 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.226.194:45527, user_credentials={real_user=slave}} blocked reactor thread for 103281us
I20250109 08:58:00.622385 22653 raft_consensus.cc:2798] T 80123146836349938344efc61333695f P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:58:00.625339 22737 raft_consensus.cc:2949] T e601a034e10e418bac8683443d86a1c1 P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:00.628116 22653 raft_consensus.cc:695] T 80123146836349938344efc61333695f P 787ddcc824c2490a808fc5a53452dc5e [term 2 LEADER]: Becoming Leader. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Running, Role: LEADER
I20250109 08:58:00.628948 22653 consensus_queue.cc:237] T 80123146836349938344efc61333695f P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:58:00.641319 22174 raft_consensus.cc:2949] T 206785f29c824cddbe9bbc00d365ffb3 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:00.643095 22751 raft_consensus.cc:988] T d015a10f95eb4f19b6d7c53cb201fa80 P 787ddcc824c2490a808fc5a53452dc5e: : Instructing follower 88935ab835544e9889d05a6587d8cf21 to start an election
I20250109 08:58:00.643651 22665 raft_consensus.cc:1076] T d015a10f95eb4f19b6d7c53cb201fa80 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Signalling peer 88935ab835544e9889d05a6587d8cf21 to start an election
I20250109 08:58:00.648953 21583 raft_consensus.cc:1270] T 64c667f887ea41b4a484ee565b21fa16 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:58:00.650643 21589 raft_consensus.cc:2949] T e601a034e10e418bac8683443d86a1c1 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:00.660410 22176 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d015a10f95eb4f19b6d7c53cb201fa80"
dest_uuid: "88935ab835544e9889d05a6587d8cf21"
from {username='slave'} at 127.0.0.1:60458
I20250109 08:58:00.660933 22176 raft_consensus.cc:491] T d015a10f95eb4f19b6d7c53cb201fa80 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:58:00.661242 22176 raft_consensus.cc:3054] T d015a10f95eb4f19b6d7c53cb201fa80 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:00.653581 21396 catalog_manager.cc:5039] ChangeConfig:REMOVE_PEER RPC for tablet 206785f29c824cddbe9bbc00d365ffb3 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250109 08:58:00.653280 21512 consensus_queue.cc:237] T 180b4d539a144a38905bf340ecb24ef4 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:00.657028 22172 raft_consensus.cc:1270] T 64c667f887ea41b4a484ee565b21fa16 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:58:00.666947 21657 raft_consensus.cc:2949] T 180b4d539a144a38905bf340ecb24ef4 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:00.657804 21583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80123146836349938344efc61333695f" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3"
I20250109 08:58:00.668762 21583 raft_consensus.cc:3054] T 80123146836349938344efc61333695f P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:00.672250 22176 raft_consensus.cc:513] T d015a10f95eb4f19b6d7c53cb201fa80 P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:00.676597 22177 raft_consensus.cc:1270] T 180b4d539a144a38905bf340ecb24ef4 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Refusing update from remote peer 8bf6e26e98b14566899578a990b27d47: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250109 08:58:00.676812 21588 raft_consensus.cc:1270] T 180b4d539a144a38905bf340ecb24ef4 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Refusing update from remote peer 8bf6e26e98b14566899578a990b27d47: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250109 08:58:00.677539 22172 raft_consensus.cc:2949] T e601a034e10e418bac8683443d86a1c1 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:00.679302 21657 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d015a10f95eb4f19b6d7c53cb201fa80" candidate_uuid: "88935ab835544e9889d05a6587d8cf21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "787ddcc824c2490a808fc5a53452dc5e"
I20250109 08:58:00.681604 21657 raft_consensus.cc:3049] T d015a10f95eb4f19b6d7c53cb201fa80 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Stepping down as leader of term 1
I20250109 08:58:00.681942 21657 raft_consensus.cc:738] T d015a10f95eb4f19b6d7c53cb201fa80 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Running, Role: LEADER
I20250109 08:58:00.682477 21588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d015a10f95eb4f19b6d7c53cb201fa80" candidate_uuid: "88935ab835544e9889d05a6587d8cf21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3"
I20250109 08:58:00.683485 21588 raft_consensus.cc:3054] T d015a10f95eb4f19b6d7c53cb201fa80 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:00.688328 21583 raft_consensus.cc:2463] T 80123146836349938344efc61333695f P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 2.
I20250109 08:58:00.688238 22739 consensus_queue.cc:1035] T 180b4d539a144a38905bf340ecb24ef4 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20250109 08:58:00.689736 22737 consensus_queue.cc:1035] T 180b4d539a144a38905bf340ecb24ef4 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250109 08:58:00.679237 22176 leader_election.cc:290] T d015a10f95eb4f19b6d7c53cb201fa80 P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 election: Requested vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:58:00.696466 21656 raft_consensus.cc:1270] T e601a034e10e418bac8683443d86a1c1 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Refusing update from remote peer 8bf6e26e98b14566899578a990b27d47: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:58:00.697197 21410 catalog_manager.cc:5526] T e601a034e10e418bac8683443d86a1c1 P a2b1a692435241ee988e0bd1661cd4c3 reported cstate change: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8bf6e26e98b14566899578a990b27d47" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:00.699913 21412 catalog_manager.cc:5526] T 206785f29c824cddbe9bbc00d365ffb3 P 8bf6e26e98b14566899578a990b27d47 reported cstate change: config changed from index 3 to 4, VOTER a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194) evicted. New cstate: current_term: 1 leader_uuid: "8bf6e26e98b14566899578a990b27d47" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250109 08:58:00.701997 21657 consensus_queue.cc:260] T d015a10f95eb4f19b6d7c53cb201fa80 P 787ddcc824c2490a808fc5a53452dc5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:00.702665 21657 raft_consensus.cc:3054] T d015a10f95eb4f19b6d7c53cb201fa80 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:00.708940 21588 raft_consensus.cc:2463] T d015a10f95eb4f19b6d7c53cb201fa80 P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88935ab835544e9889d05a6587d8cf21 in term 2.
I20250109 08:58:00.710678 22142 leader_election.cc:304] T d015a10f95eb4f19b6d7c53cb201fa80 P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88935ab835544e9889d05a6587d8cf21, a2b1a692435241ee988e0bd1661cd4c3; no voters:
I20250109 08:58:00.712838 22689 raft_consensus.cc:2798] T d015a10f95eb4f19b6d7c53cb201fa80 P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:58:00.712996 21657 raft_consensus.cc:2463] T d015a10f95eb4f19b6d7c53cb201fa80 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88935ab835544e9889d05a6587d8cf21 in term 2.
I20250109 08:58:00.713374 22689 raft_consensus.cc:695] T d015a10f95eb4f19b6d7c53cb201fa80 P 88935ab835544e9889d05a6587d8cf21 [term 2 LEADER]: Becoming Leader. State: Replica: 88935ab835544e9889d05a6587d8cf21, State: Running, Role: LEADER
I20250109 08:58:00.715364 22689 consensus_queue.cc:237] T d015a10f95eb4f19b6d7c53cb201fa80 P 88935ab835544e9889d05a6587d8cf21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:00.714869 22738 raft_consensus.cc:2949] T 180b4d539a144a38905bf340ecb24ef4 P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:00.724224 21589 raft_consensus.cc:2949] T 180b4d539a144a38905bf340ecb24ef4 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:00.729929 21658 raft_consensus.cc:1270] T 206785f29c824cddbe9bbc00d365ffb3 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Refusing update from remote peer 8bf6e26e98b14566899578a990b27d47: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250109 08:58:00.733713 22172 raft_consensus.cc:2949] T 180b4d539a144a38905bf340ecb24ef4 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:00.747553 21396 catalog_manager.cc:5039] ChangeConfig:REMOVE_PEER RPC for tablet 180b4d539a144a38905bf340ecb24ef4 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250109 08:58:00.753327 22721 consensus_queue.cc:1035] T e601a034e10e418bac8683443d86a1c1 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:58:00.755043 21570 tablet_service.cc:1514] Processing DeleteTablet for tablet 206785f29c824cddbe9bbc00d365ffb3 with delete_type TABLET_DATA_TOMBSTONED (TS a2b1a692435241ee988e0bd1661cd4c3 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:34732
I20250109 08:58:00.755864 21410 catalog_manager.cc:5526] T 180b4d539a144a38905bf340ecb24ef4 P a2b1a692435241ee988e0bd1661cd4c3 reported cstate change: config changed from index 3 to 4, VOTER 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195) evicted. New cstate: current_term: 1 leader_uuid: "8bf6e26e98b14566899578a990b27d47" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:00.769027 22660 consensus_queue.cc:1035] T 206785f29c824cddbe9bbc00d365ffb3 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250109 08:58:00.781251 22750 consensus_queue.cc:1035] T 64c667f887ea41b4a484ee565b21fa16 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:58:00.781986 21653 tablet_service.cc:1514] Processing DeleteTablet for tablet 180b4d539a144a38905bf340ecb24ef4 with delete_type TABLET_DATA_TOMBSTONED (TS 787ddcc824c2490a808fc5a53452dc5e not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:58436
I20250109 08:58:00.785279 21514 raft_consensus.cc:1270] T 610441ef4b1d40c284eff3a714e4214e P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Refusing update from remote peer a2b1a692435241ee988e0bd1661cd4c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:58:00.788941 22653 raft_consensus.cc:988] T 81d74e7d4472497aac5463a4b47e4fb9 P 787ddcc824c2490a808fc5a53452dc5e: : Instructing follower 88935ab835544e9889d05a6587d8cf21 to start an election
I20250109 08:58:00.789398 22653 raft_consensus.cc:1076] T 81d74e7d4472497aac5463a4b47e4fb9 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Signalling peer 88935ab835544e9889d05a6587d8cf21 to start an election
I20250109 08:58:00.791931 22743 consensus_queue.cc:1035] T 610441ef4b1d40c284eff3a714e4214e P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:58:00.794301 22758 tablet_replica.cc:331] T 206785f29c824cddbe9bbc00d365ffb3 P a2b1a692435241ee988e0bd1661cd4c3: stopping tablet replica
I20250109 08:58:00.796111 22758 raft_consensus.cc:2238] T 206785f29c824cddbe9bbc00d365ffb3 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:00.796851 22758 raft_consensus.cc:2267] T 206785f29c824cddbe9bbc00d365ffb3 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:00.803798 22758 ts_tablet_manager.cc:1905] T 206785f29c824cddbe9bbc00d365ffb3 P a2b1a692435241ee988e0bd1661cd4c3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250109 08:58:00.804986 21514 consensus_queue.cc:237] T e601a034e10e418bac8683443d86a1c1 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:00.811318 22750 consensus_queue.cc:1035] T 64c667f887ea41b4a484ee565b21fa16 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:58:00.813947 22172 raft_consensus.cc:1270] T e601a034e10e418bac8683443d86a1c1 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Refusing update from remote peer 8bf6e26e98b14566899578a990b27d47: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250109 08:58:00.816483 21589 raft_consensus.cc:1270] T e601a034e10e418bac8683443d86a1c1 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Refusing update from remote peer 8bf6e26e98b14566899578a990b27d47: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250109 08:58:00.817929 22660 consensus_queue.cc:1035] T e601a034e10e418bac8683443d86a1c1 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250109 08:58:00.820571 22721 consensus_queue.cc:1035] T e601a034e10e418bac8683443d86a1c1 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250109 08:58:00.829428 22721 raft_consensus.cc:2949] T e601a034e10e418bac8683443d86a1c1 P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:00.832940 21409 catalog_manager.cc:5526] T 80123146836349938344efc61333695f P 787ddcc824c2490a808fc5a53452dc5e reported cstate change: term changed from 1 to 2, leader changed from 8bf6e26e98b14566899578a990b27d47 (127.19.226.193) to 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195). New cstate: current_term: 2 leader_uuid: "787ddcc824c2490a808fc5a53452dc5e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: HEALTHY } } }
I20250109 08:58:00.837378 22177 raft_consensus.cc:2949] T e601a034e10e418bac8683443d86a1c1 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:00.838672 22761 tablet_replica.cc:331] T 180b4d539a144a38905bf340ecb24ef4 P 787ddcc824c2490a808fc5a53452dc5e: stopping tablet replica
I20250109 08:58:00.843924 22758 ts_tablet_manager.cc:1918] T 206785f29c824cddbe9bbc00d365ffb3 P a2b1a692435241ee988e0bd1661cd4c3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20250109 08:58:00.844995 22758 log.cc:1198] T 206785f29c824cddbe9bbc00d365ffb3 P a2b1a692435241ee988e0bd1661cd4c3: Deleting WAL directory at /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736412987538469-20363-0/minicluster-data/ts-1-root/wals/206785f29c824cddbe9bbc00d365ffb3
I20250109 08:58:00.845100 22761 raft_consensus.cc:2238] T 180b4d539a144a38905bf340ecb24ef4 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:00.847328 21397 catalog_manager.cc:4872] TS a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): tablet 206785f29c824cddbe9bbc00d365ffb3 (table test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]) successfully deleted
I20250109 08:58:00.847522 22761 raft_consensus.cc:2267] T 180b4d539a144a38905bf340ecb24ef4 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:00.850754 22761 ts_tablet_manager.cc:1905] T 180b4d539a144a38905bf340ecb24ef4 P 787ddcc824c2490a808fc5a53452dc5e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250109 08:58:00.856606 21396 catalog_manager.cc:5039] ChangeConfig:REMOVE_PEER RPC for tablet e601a034e10e418bac8683443d86a1c1 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250109 08:58:00.858708 21410 catalog_manager.cc:5526] T e601a034e10e418bac8683443d86a1c1 P 8bf6e26e98b14566899578a990b27d47 reported cstate change: config changed from index 3 to 4, VOTER 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195) evicted. New cstate: current_term: 1 leader_uuid: "8bf6e26e98b14566899578a990b27d47" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250109 08:58:00.846493 21589 raft_consensus.cc:2949] T e601a034e10e418bac8683443d86a1c1 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:00.871567 22751 raft_consensus.cc:988] T c18c1d46a6ca47598d7b161ed70a4e7a P 787ddcc824c2490a808fc5a53452dc5e: : Instructing follower a2b1a692435241ee988e0bd1661cd4c3 to start an election
I20250109 08:58:00.872038 22665 raft_consensus.cc:1076] T c18c1d46a6ca47598d7b161ed70a4e7a P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Signalling peer a2b1a692435241ee988e0bd1661cd4c3 to start an election
I20250109 08:58:00.876153 21653 tablet_service.cc:1514] Processing DeleteTablet for tablet e601a034e10e418bac8683443d86a1c1 with delete_type TABLET_DATA_TOMBSTONED (TS 787ddcc824c2490a808fc5a53452dc5e not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:58436
I20250109 08:58:00.876842 22761 ts_tablet_manager.cc:1918] T 180b4d539a144a38905bf340ecb24ef4 P 787ddcc824c2490a808fc5a53452dc5e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20250109 08:58:00.877446 22761 log.cc:1198] T 180b4d539a144a38905bf340ecb24ef4 P 787ddcc824c2490a808fc5a53452dc5e: Deleting WAL directory at /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736412987538469-20363-0/minicluster-data/ts-2-root/wals/180b4d539a144a38905bf340ecb24ef4
I20250109 08:58:00.879338 21396 catalog_manager.cc:4872] TS 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731): tablet 180b4d539a144a38905bf340ecb24ef4 (table test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]) successfully deleted
I20250109 08:58:00.881250 22761 tablet_replica.cc:331] T e601a034e10e418bac8683443d86a1c1 P 787ddcc824c2490a808fc5a53452dc5e: stopping tablet replica
I20250109 08:58:00.882483 22761 raft_consensus.cc:2238] T e601a034e10e418bac8683443d86a1c1 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:00.885216 22761 raft_consensus.cc:2267] T e601a034e10e418bac8683443d86a1c1 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:00.885677 22172 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "81d74e7d4472497aac5463a4b47e4fb9"
dest_uuid: "88935ab835544e9889d05a6587d8cf21"
from {username='slave'} at 127.0.0.1:60458
I20250109 08:58:00.890610 22172 raft_consensus.cc:491] T 81d74e7d4472497aac5463a4b47e4fb9 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:58:00.891000 22172 raft_consensus.cc:3054] T 81d74e7d4472497aac5463a4b47e4fb9 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:00.897039 22761 ts_tablet_manager.cc:1905] T e601a034e10e418bac8683443d86a1c1 P 787ddcc824c2490a808fc5a53452dc5e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250109 08:58:00.898126 22172 raft_consensus.cc:513] T 81d74e7d4472497aac5463a4b47e4fb9 P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:00.903954 22172 leader_election.cc:290] T 81d74e7d4472497aac5463a4b47e4fb9 P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 election: Requested vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:58:00.904137 21658 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81d74e7d4472497aac5463a4b47e4fb9" candidate_uuid: "88935ab835544e9889d05a6587d8cf21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "787ddcc824c2490a808fc5a53452dc5e"
I20250109 08:58:00.904752 21658 raft_consensus.cc:3049] T 81d74e7d4472497aac5463a4b47e4fb9 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Stepping down as leader of term 1
I20250109 08:58:00.905030 21658 raft_consensus.cc:738] T 81d74e7d4472497aac5463a4b47e4fb9 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Running, Role: LEADER
I20250109 08:58:00.905681 21658 consensus_queue.cc:260] T 81d74e7d4472497aac5463a4b47e4fb9 P 787ddcc824c2490a808fc5a53452dc5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:00.906724 21658 raft_consensus.cc:3054] T 81d74e7d4472497aac5463a4b47e4fb9 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:00.914455 21658 raft_consensus.cc:2463] T 81d74e7d4472497aac5463a4b47e4fb9 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88935ab835544e9889d05a6587d8cf21 in term 2.
I20250109 08:58:00.916957 22141 leader_election.cc:304] T 81d74e7d4472497aac5463a4b47e4fb9 P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, 88935ab835544e9889d05a6587d8cf21; no voters:
I20250109 08:58:00.917704 22689 raft_consensus.cc:2798] T 81d74e7d4472497aac5463a4b47e4fb9 P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:58:00.918151 22689 raft_consensus.cc:695] T 81d74e7d4472497aac5463a4b47e4fb9 P 88935ab835544e9889d05a6587d8cf21 [term 2 LEADER]: Becoming Leader. State: Replica: 88935ab835544e9889d05a6587d8cf21, State: Running, Role: LEADER
I20250109 08:58:00.919014 21589 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "c18c1d46a6ca47598d7b161ed70a4e7a"
dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3"
from {username='slave'} at 127.0.0.1:34746
I20250109 08:58:00.919011 22689 consensus_queue.cc:237] T 81d74e7d4472497aac5463a4b47e4fb9 P 88935ab835544e9889d05a6587d8cf21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:00.877558 21412 catalog_manager.cc:5526] T d015a10f95eb4f19b6d7c53cb201fa80 P 88935ab835544e9889d05a6587d8cf21 reported cstate change: term changed from 1 to 2, leader changed from 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195) to 88935ab835544e9889d05a6587d8cf21 (127.19.226.196). New cstate: current_term: 2 leader_uuid: "88935ab835544e9889d05a6587d8cf21" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250109 08:58:00.923995 21588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81d74e7d4472497aac5463a4b47e4fb9" candidate_uuid: "88935ab835544e9889d05a6587d8cf21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3"
I20250109 08:58:00.924608 21588 raft_consensus.cc:3054] T 81d74e7d4472497aac5463a4b47e4fb9 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:00.928680 21660 raft_consensus.cc:2949] T 206785f29c824cddbe9bbc00d365ffb3 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:00.937024 22665 raft_consensus.cc:2949] T 64c667f887ea41b4a484ee565b21fa16 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:00.940725 21589 raft_consensus.cc:491] T c18c1d46a6ca47598d7b161ed70a4e7a P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:58:00.941104 21589 raft_consensus.cc:3054] T c18c1d46a6ca47598d7b161ed70a4e7a P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:00.947126 21589 raft_consensus.cc:513] T c18c1d46a6ca47598d7b161ed70a4e7a P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:58:00.960135 22172 raft_consensus.cc:2949] T 64c667f887ea41b4a484ee565b21fa16 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:00.965827 21412 catalog_manager.cc:5526] T 81d74e7d4472497aac5463a4b47e4fb9 P 88935ab835544e9889d05a6587d8cf21 reported cstate change: term changed from 1 to 2, leader changed from 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195) to 88935ab835544e9889d05a6587d8cf21 (127.19.226.196). New cstate: current_term: 2 leader_uuid: "88935ab835544e9889d05a6587d8cf21" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250109 08:58:00.968706 22761 ts_tablet_manager.cc:1918] T e601a034e10e418bac8683443d86a1c1 P 787ddcc824c2490a808fc5a53452dc5e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20250109 08:58:00.969190 22761 log.cc:1198] T e601a034e10e418bac8683443d86a1c1 P 787ddcc824c2490a808fc5a53452dc5e: Deleting WAL directory at /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736412987538469-20363-0/minicluster-data/ts-2-root/wals/e601a034e10e418bac8683443d86a1c1
I20250109 08:58:00.970897 21396 catalog_manager.cc:4872] TS 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731): tablet e601a034e10e418bac8683443d86a1c1 (table test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]) successfully deleted
I20250109 08:58:00.972826 21588 raft_consensus.cc:2463] T 81d74e7d4472497aac5463a4b47e4fb9 P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88935ab835544e9889d05a6587d8cf21 in term 2.
I20250109 08:58:00.981206 21512 raft_consensus.cc:1270] T 5717a02ef53a4b22a96f431ca6c20d93 P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Refusing update from remote peer 88935ab835544e9889d05a6587d8cf21: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250109 08:58:00.982609 22689 consensus_queue.cc:1035] T 5717a02ef53a4b22a96f431ca6c20d93 P 88935ab835544e9889d05a6587d8cf21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250109 08:58:00.987721 21514 raft_consensus.cc:2949] T 64c667f887ea41b4a484ee565b21fa16 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:00.992954 21657 raft_consensus.cc:1270] T 5717a02ef53a4b22a96f431ca6c20d93 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Refusing update from remote peer 88935ab835544e9889d05a6587d8cf21: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250109 08:58:00.998172 22689 consensus_queue.cc:1035] T 5717a02ef53a4b22a96f431ca6c20d93 P 88935ab835544e9889d05a6587d8cf21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250109 08:58:01.003073 21589 leader_election.cc:290] T c18c1d46a6ca47598d7b161ed70a4e7a P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 2 election: Requested vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:58:01.024524 21588 raft_consensus.cc:2949] T 64c667f887ea41b4a484ee565b21fa16 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:01.041757 21658 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c18c1d46a6ca47598d7b161ed70a4e7a" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "787ddcc824c2490a808fc5a53452dc5e"
I20250109 08:58:01.042364 21658 raft_consensus.cc:3049] T c18c1d46a6ca47598d7b161ed70a4e7a P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Stepping down as leader of term 1
I20250109 08:58:01.042659 21658 raft_consensus.cc:738] T c18c1d46a6ca47598d7b161ed70a4e7a P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Running, Role: LEADER
I20250109 08:58:01.043329 21658 consensus_queue.cc:260] T c18c1d46a6ca47598d7b161ed70a4e7a P 787ddcc824c2490a808fc5a53452dc5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:58:01.044677 21513 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c18c1d46a6ca47598d7b161ed70a4e7a" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8bf6e26e98b14566899578a990b27d47"
I20250109 08:58:01.045238 21513 raft_consensus.cc:3054] T c18c1d46a6ca47598d7b161ed70a4e7a P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:01.045027 21410 catalog_manager.cc:5526] T 64c667f887ea41b4a484ee565b21fa16 P 88935ab835544e9889d05a6587d8cf21 reported cstate change: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "787ddcc824c2490a808fc5a53452dc5e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:01.048532 21658 raft_consensus.cc:3054] T c18c1d46a6ca47598d7b161ed70a4e7a P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:01.057720 21513 raft_consensus.cc:2463] T c18c1d46a6ca47598d7b161ed70a4e7a P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2b1a692435241ee988e0bd1661cd4c3 in term 2.
I20250109 08:58:01.060671 21658 raft_consensus.cc:2463] T c18c1d46a6ca47598d7b161ed70a4e7a P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2b1a692435241ee988e0bd1661cd4c3 in term 2.
I20250109 08:58:01.098606 21552 leader_election.cc:304] T c18c1d46a6ca47598d7b161ed70a4e7a P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, a2b1a692435241ee988e0bd1661cd4c3; no voters:
I20250109 08:58:01.082743 21657 consensus_queue.cc:237] T 64c667f887ea41b4a484ee565b21fa16 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:01.103672 22652 raft_consensus.cc:2798] T c18c1d46a6ca47598d7b161ed70a4e7a P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:58:01.108109 22652 raft_consensus.cc:695] T c18c1d46a6ca47598d7b161ed70a4e7a P a2b1a692435241ee988e0bd1661cd4c3 [term 2 LEADER]: Becoming Leader. State: Replica: a2b1a692435241ee988e0bd1661cd4c3, State: Running, Role: LEADER
I20250109 08:58:01.109046 22172 raft_consensus.cc:1270] T 64c667f887ea41b4a484ee565b21fa16 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250109 08:58:01.109766 22652 consensus_queue.cc:237] T c18c1d46a6ca47598d7b161ed70a4e7a P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:58:01.111099 22653 consensus_queue.cc:1035] T 64c667f887ea41b4a484ee565b21fa16 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250109 08:58:01.120501 22653 raft_consensus.cc:2949] T 64c667f887ea41b4a484ee565b21fa16 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8bf6e26e98b14566899578a990b27d47 (127.19.226.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:01.126060 22172 raft_consensus.cc:2949] T 64c667f887ea41b4a484ee565b21fa16 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8bf6e26e98b14566899578a990b27d47 (127.19.226.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:01.128019 21583 raft_consensus.cc:1270] T 80123146836349938344efc61333695f P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:58:01.130928 22750 consensus_queue.cc:1035] T 80123146836349938344efc61333695f P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:58:01.133564 21589 raft_consensus.cc:1270] T 64c667f887ea41b4a484ee565b21fa16 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250109 08:58:01.139619 22665 consensus_queue.cc:1035] T 64c667f887ea41b4a484ee565b21fa16 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20250109 08:58:01.142697 21409 catalog_manager.cc:5526] T c18c1d46a6ca47598d7b161ed70a4e7a P a2b1a692435241ee988e0bd1661cd4c3 reported cstate change: term changed from 1 to 2, leader changed from 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195) to a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194). New cstate: current_term: 2 leader_uuid: "a2b1a692435241ee988e0bd1661cd4c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:01.145785 21589 raft_consensus.cc:2949] T 64c667f887ea41b4a484ee565b21fa16 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8bf6e26e98b14566899578a990b27d47 (127.19.226.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:01.147715 21396 catalog_manager.cc:5039] ChangeConfig:REMOVE_PEER RPC for tablet 64c667f887ea41b4a484ee565b21fa16 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250109 08:58:01.156244 21514 raft_consensus.cc:1270] T 80123146836349938344efc61333695f P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:58:01.158435 22665 consensus_queue.cc:1035] T 80123146836349938344efc61333695f P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:58:01.180567 21658 raft_consensus.cc:1270] T d015a10f95eb4f19b6d7c53cb201fa80 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Refusing update from remote peer 88935ab835544e9889d05a6587d8cf21: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250109 08:58:01.182085 22770 consensus_queue.cc:1035] T d015a10f95eb4f19b6d7c53cb201fa80 P 88935ab835544e9889d05a6587d8cf21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250109 08:58:01.170109 21409 catalog_manager.cc:5526] T 64c667f887ea41b4a484ee565b21fa16 P 88935ab835544e9889d05a6587d8cf21 reported cstate change: config changed from index 3 to 4, VOTER 8bf6e26e98b14566899578a990b27d47 (127.19.226.193) evicted. New cstate: current_term: 1 leader_uuid: "787ddcc824c2490a808fc5a53452dc5e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:01.210042 21503 tablet_service.cc:1514] Processing DeleteTablet for tablet 64c667f887ea41b4a484ee565b21fa16 with delete_type TABLET_DATA_TOMBSTONED (TS 8bf6e26e98b14566899578a990b27d47 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:39242
W20250109 08:58:01.224216 21552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.226.193:42725, user_credentials={real_user=slave}} blocked reactor thread for 80115.1us
I20250109 08:58:01.231694 22776 tablet_replica.cc:331] T 64c667f887ea41b4a484ee565b21fa16 P 8bf6e26e98b14566899578a990b27d47: stopping tablet replica
I20250109 08:58:01.232913 22776 raft_consensus.cc:2238] T 64c667f887ea41b4a484ee565b21fa16 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:01.233613 22776 raft_consensus.cc:2267] T 64c667f887ea41b4a484ee565b21fa16 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:01.241451 22776 ts_tablet_manager.cc:1905] T 64c667f887ea41b4a484ee565b21fa16 P 8bf6e26e98b14566899578a990b27d47: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250109 08:58:01.245332 20363 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250109 08:58:01.264665 21583 raft_consensus.cc:1270] T d015a10f95eb4f19b6d7c53cb201fa80 P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Refusing update from remote peer 88935ab835544e9889d05a6587d8cf21: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250109 08:58:01.278110 22770 consensus_queue.cc:1035] T d015a10f95eb4f19b6d7c53cb201fa80 P 88935ab835544e9889d05a6587d8cf21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250109 08:58:01.291198 22776 ts_tablet_manager.cc:1918] T 64c667f887ea41b4a484ee565b21fa16 P 8bf6e26e98b14566899578a990b27d47: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20250109 08:58:01.291663 22776 log.cc:1198] T 64c667f887ea41b4a484ee565b21fa16 P 8bf6e26e98b14566899578a990b27d47: Deleting WAL directory at /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736412987538469-20363-0/minicluster-data/ts-0-root/wals/64c667f887ea41b4a484ee565b21fa16
I20250109 08:58:01.293748 21396 catalog_manager.cc:4872] TS 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): tablet 64c667f887ea41b4a484ee565b21fa16 (table test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]) successfully deleted
I20250109 08:58:01.317809 21514 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "0cc4fcda876242c396b804edebebb01f"
dest_uuid: "8bf6e26e98b14566899578a990b27d47"
mode: GRACEFUL
new_leader_uuid: "a2b1a692435241ee988e0bd1661cd4c3"
from {username='slave'} at 127.0.0.1:39274
I20250109 08:58:01.318392 21514 raft_consensus.cc:604] T 0cc4fcda876242c396b804edebebb01f P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Received request to transfer leadership to TS a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:58:01.321067 21658 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "239ca74a4c0a455a9db2652d7debb989"
dest_uuid: "787ddcc824c2490a808fc5a53452dc5e"
mode: GRACEFUL
new_leader_uuid: "88935ab835544e9889d05a6587d8cf21"
from {username='slave'} at 127.0.0.1:55054
I20250109 08:58:01.321614 21658 raft_consensus.cc:604] T 239ca74a4c0a455a9db2652d7debb989 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Received request to transfer leadership to TS 88935ab835544e9889d05a6587d8cf21
I20250109 08:58:01.323921 21514 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "56bbc306275e4a409e028a7ca1dd1602"
dest_uuid: "8bf6e26e98b14566899578a990b27d47"
mode: GRACEFUL
new_leader_uuid: "787ddcc824c2490a808fc5a53452dc5e"
from {username='slave'} at 127.0.0.1:39274
I20250109 08:58:01.324347 21514 raft_consensus.cc:604] T 56bbc306275e4a409e028a7ca1dd1602 P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Received request to transfer leadership to TS 787ddcc824c2490a808fc5a53452dc5e
I20250109 08:58:01.326225 21514 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "7a7211b8d4374ad1afd82e8cf85bad81"
dest_uuid: "8bf6e26e98b14566899578a990b27d47"
mode: GRACEFUL
new_leader_uuid: "787ddcc824c2490a808fc5a53452dc5e"
from {username='slave'} at 127.0.0.1:39274
I20250109 08:58:01.326610 21514 raft_consensus.cc:604] T 7a7211b8d4374ad1afd82e8cf85bad81 P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Received request to transfer leadership to TS 787ddcc824c2490a808fc5a53452dc5e
I20250109 08:58:01.349725 21589 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "c761cb691bd241ae825917a3fbc3ad24"
dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3"
mode: GRACEFUL
new_leader_uuid: "8bf6e26e98b14566899578a990b27d47"
from {username='slave'} at 127.0.0.1:46832
I20250109 08:58:01.350313 21589 raft_consensus.cc:604] T c761cb691bd241ae825917a3fbc3ad24 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 LEADER]: Received request to transfer leadership to TS 8bf6e26e98b14566899578a990b27d47
I20250109 08:58:01.352913 20363 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 5
I20250109 08:58:01.354468 20363 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250109 08:58:01.355091 20363 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250109 08:58:01.373256 22738 raft_consensus.cc:988] T 7a7211b8d4374ad1afd82e8cf85bad81 P 8bf6e26e98b14566899578a990b27d47: : Instructing follower 787ddcc824c2490a808fc5a53452dc5e to start an election
I20250109 08:58:01.373958 22737 raft_consensus.cc:1076] T 7a7211b8d4374ad1afd82e8cf85bad81 P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Signalling peer 787ddcc824c2490a808fc5a53452dc5e to start an election
I20250109 08:58:01.376080 21658 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "7a7211b8d4374ad1afd82e8cf85bad81"
dest_uuid: "787ddcc824c2490a808fc5a53452dc5e"
from {username='slave'} at 127.0.0.1:58440
I20250109 08:58:01.376726 21658 raft_consensus.cc:491] T 7a7211b8d4374ad1afd82e8cf85bad81 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:58:01.377104 21658 raft_consensus.cc:3054] T 7a7211b8d4374ad1afd82e8cf85bad81 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:01.384043 21658 raft_consensus.cc:513] T 7a7211b8d4374ad1afd82e8cf85bad81 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:58:01.386554 21514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a7211b8d4374ad1afd82e8cf85bad81" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8bf6e26e98b14566899578a990b27d47"
I20250109 08:58:01.386664 21658 leader_election.cc:290] T 7a7211b8d4374ad1afd82e8cf85bad81 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 2 election: Requested vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
I20250109 08:58:01.387099 21514 raft_consensus.cc:3049] T 7a7211b8d4374ad1afd82e8cf85bad81 P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Stepping down as leader of term 1
I20250109 08:58:01.387429 21514 raft_consensus.cc:738] T 7a7211b8d4374ad1afd82e8cf85bad81 P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8bf6e26e98b14566899578a990b27d47, State: Running, Role: LEADER
I20250109 08:58:01.388082 21514 consensus_queue.cc:260] T 7a7211b8d4374ad1afd82e8cf85bad81 P 8bf6e26e98b14566899578a990b27d47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:58:01.389101 21514 raft_consensus.cc:3054] T 7a7211b8d4374ad1afd82e8cf85bad81 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:01.393425 21589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a7211b8d4374ad1afd82e8cf85bad81" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3"
I20250109 08:58:01.394026 21589 raft_consensus.cc:3054] T 7a7211b8d4374ad1afd82e8cf85bad81 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:01.408087 21514 raft_consensus.cc:2463] T 7a7211b8d4374ad1afd82e8cf85bad81 P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 2.
I20250109 08:58:01.408700 21589 raft_consensus.cc:2463] T 7a7211b8d4374ad1afd82e8cf85bad81 P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 2.
I20250109 08:58:01.409330 21626 leader_election.cc:304] T 7a7211b8d4374ad1afd82e8cf85bad81 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, 8bf6e26e98b14566899578a990b27d47; no voters:
I20250109 08:58:01.409509 21658 raft_consensus.cc:1270] T 81d74e7d4472497aac5463a4b47e4fb9 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Refusing update from remote peer 88935ab835544e9889d05a6587d8cf21: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250109 08:58:01.410107 22665 raft_consensus.cc:2798] T 7a7211b8d4374ad1afd82e8cf85bad81 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:58:01.410605 22665 raft_consensus.cc:695] T 7a7211b8d4374ad1afd82e8cf85bad81 P 787ddcc824c2490a808fc5a53452dc5e [term 2 LEADER]: Becoming Leader. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Running, Role: LEADER
I20250109 08:58:01.410775 22679 consensus_queue.cc:1035] T 81d74e7d4472497aac5463a4b47e4fb9 P 88935ab835544e9889d05a6587d8cf21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250109 08:58:01.411420 22665 consensus_queue.cc:237] T 7a7211b8d4374ad1afd82e8cf85bad81 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:58:01.424770 21410 catalog_manager.cc:5526] T 7a7211b8d4374ad1afd82e8cf85bad81 P 787ddcc824c2490a808fc5a53452dc5e reported cstate change: term changed from 1 to 2, leader changed from 8bf6e26e98b14566899578a990b27d47 (127.19.226.193) to 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195). New cstate: current_term: 2 leader_uuid: "787ddcc824c2490a808fc5a53452dc5e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: HEALTHY } } }
I20250109 08:58:01.428593 21589 raft_consensus.cc:1270] T 81d74e7d4472497aac5463a4b47e4fb9 P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Refusing update from remote peer 88935ab835544e9889d05a6587d8cf21: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250109 08:58:01.432133 22679 consensus_queue.cc:1035] T 81d74e7d4472497aac5463a4b47e4fb9 P 88935ab835544e9889d05a6587d8cf21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250109 08:58:01.452428 22743 raft_consensus.cc:988] T c761cb691bd241ae825917a3fbc3ad24 P a2b1a692435241ee988e0bd1661cd4c3: : Instructing follower 8bf6e26e98b14566899578a990b27d47 to start an election
I20250109 08:58:01.452922 22735 raft_consensus.cc:1076] T c761cb691bd241ae825917a3fbc3ad24 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 LEADER]: Signalling peer 8bf6e26e98b14566899578a990b27d47 to start an election
I20250109 08:58:01.454419 21514 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "c761cb691bd241ae825917a3fbc3ad24"
dest_uuid: "8bf6e26e98b14566899578a990b27d47"
from {username='slave'} at 127.0.0.1:39250
I20250109 08:58:01.454900 21514 raft_consensus.cc:491] T c761cb691bd241ae825917a3fbc3ad24 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:58:01.455163 21514 raft_consensus.cc:3054] T c761cb691bd241ae825917a3fbc3ad24 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:01.460409 21514 raft_consensus.cc:513] T c761cb691bd241ae825917a3fbc3ad24 P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:58:01.464536 21658 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c761cb691bd241ae825917a3fbc3ad24" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "787ddcc824c2490a808fc5a53452dc5e"
I20250109 08:58:01.465082 21658 raft_consensus.cc:3054] T c761cb691bd241ae825917a3fbc3ad24 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:01.467410 21588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c761cb691bd241ae825917a3fbc3ad24" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3"
I20250109 08:58:01.467995 21588 raft_consensus.cc:3049] T c761cb691bd241ae825917a3fbc3ad24 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 LEADER]: Stepping down as leader of term 1
I20250109 08:58:01.468267 21588 raft_consensus.cc:738] T c761cb691bd241ae825917a3fbc3ad24 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a2b1a692435241ee988e0bd1661cd4c3, State: Running, Role: LEADER
I20250109 08:58:01.469200 21588 consensus_queue.cc:260] T c761cb691bd241ae825917a3fbc3ad24 P a2b1a692435241ee988e0bd1661cd4c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:58:01.470343 21588 raft_consensus.cc:3054] T c761cb691bd241ae825917a3fbc3ad24 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:01.471200 21658 raft_consensus.cc:2463] T c761cb691bd241ae825917a3fbc3ad24 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8bf6e26e98b14566899578a990b27d47 in term 2.
I20250109 08:58:01.473774 21514 leader_election.cc:290] T c761cb691bd241ae825917a3fbc3ad24 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 2 election: Requested vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:58:01.474119 21477 leader_election.cc:304] T c761cb691bd241ae825917a3fbc3ad24 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, 8bf6e26e98b14566899578a990b27d47; no voters:
I20250109 08:58:01.475026 22737 raft_consensus.cc:2798] T c761cb691bd241ae825917a3fbc3ad24 P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:58:01.475394 22737 raft_consensus.cc:695] T c761cb691bd241ae825917a3fbc3ad24 P 8bf6e26e98b14566899578a990b27d47 [term 2 LEADER]: Becoming Leader. State: Replica: 8bf6e26e98b14566899578a990b27d47, State: Running, Role: LEADER
I20250109 08:58:01.476168 22737 consensus_queue.cc:237] T c761cb691bd241ae825917a3fbc3ad24 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:58:01.483517 21588 raft_consensus.cc:2463] T c761cb691bd241ae825917a3fbc3ad24 P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8bf6e26e98b14566899578a990b27d47 in term 2.
I20250109 08:58:01.503954 21410 catalog_manager.cc:5526] T c761cb691bd241ae825917a3fbc3ad24 P 8bf6e26e98b14566899578a990b27d47 reported cstate change: term changed from 1 to 2, leader changed from a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194) to 8bf6e26e98b14566899578a990b27d47 (127.19.226.193). New cstate: current_term: 2 leader_uuid: "8bf6e26e98b14566899578a990b27d47" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:01.612962 22740 raft_consensus.cc:988] T 56bbc306275e4a409e028a7ca1dd1602 P 8bf6e26e98b14566899578a990b27d47: : Instructing follower 787ddcc824c2490a808fc5a53452dc5e to start an election
I20250109 08:58:01.613438 22737 raft_consensus.cc:1076] T 56bbc306275e4a409e028a7ca1dd1602 P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Signalling peer 787ddcc824c2490a808fc5a53452dc5e to start an election
I20250109 08:58:01.614894 21657 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "56bbc306275e4a409e028a7ca1dd1602"
dest_uuid: "787ddcc824c2490a808fc5a53452dc5e"
from {username='slave'} at 127.0.0.1:58440
I20250109 08:58:01.615424 21657 raft_consensus.cc:491] T 56bbc306275e4a409e028a7ca1dd1602 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:58:01.615751 21657 raft_consensus.cc:3054] T 56bbc306275e4a409e028a7ca1dd1602 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:01.622151 21657 raft_consensus.cc:513] T 56bbc306275e4a409e028a7ca1dd1602 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:58:01.623836 21657 leader_election.cc:290] T 56bbc306275e4a409e028a7ca1dd1602 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 2 election: Requested vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
I20250109 08:58:01.624336 21512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56bbc306275e4a409e028a7ca1dd1602" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8bf6e26e98b14566899578a990b27d47"
I20250109 08:58:01.624804 21588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56bbc306275e4a409e028a7ca1dd1602" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3"
I20250109 08:58:01.625059 21512 raft_consensus.cc:3049] T 56bbc306275e4a409e028a7ca1dd1602 P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Stepping down as leader of term 1
I20250109 08:58:01.625452 21512 raft_consensus.cc:738] T 56bbc306275e4a409e028a7ca1dd1602 P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8bf6e26e98b14566899578a990b27d47, State: Running, Role: LEADER
I20250109 08:58:01.625487 21588 raft_consensus.cc:3054] T 56bbc306275e4a409e028a7ca1dd1602 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:01.626204 21512 consensus_queue.cc:260] T 56bbc306275e4a409e028a7ca1dd1602 P 8bf6e26e98b14566899578a990b27d47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:58:01.627348 21512 raft_consensus.cc:3054] T 56bbc306275e4a409e028a7ca1dd1602 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:01.633265 21588 raft_consensus.cc:2463] T 56bbc306275e4a409e028a7ca1dd1602 P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 2.
I20250109 08:58:01.634392 21627 leader_election.cc:304] T 56bbc306275e4a409e028a7ca1dd1602 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, a2b1a692435241ee988e0bd1661cd4c3; no voters:
I20250109 08:58:01.635344 22653 raft_consensus.cc:2798] T 56bbc306275e4a409e028a7ca1dd1602 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:58:01.635699 21514 raft_consensus.cc:1270] T c18c1d46a6ca47598d7b161ed70a4e7a P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Refusing update from remote peer a2b1a692435241ee988e0bd1661cd4c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:58:01.635771 22653 raft_consensus.cc:695] T 56bbc306275e4a409e028a7ca1dd1602 P 787ddcc824c2490a808fc5a53452dc5e [term 2 LEADER]: Becoming Leader. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Running, Role: LEADER
I20250109 08:58:01.635869 21512 raft_consensus.cc:2463] T 56bbc306275e4a409e028a7ca1dd1602 P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 2.
I20250109 08:58:01.636806 22653 consensus_queue.cc:237] T 56bbc306275e4a409e028a7ca1dd1602 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:58:01.639335 22773 consensus_queue.cc:1035] T c18c1d46a6ca47598d7b161ed70a4e7a P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:58:01.649364 21410 catalog_manager.cc:5526] T 56bbc306275e4a409e028a7ca1dd1602 P 787ddcc824c2490a808fc5a53452dc5e reported cstate change: term changed from 1 to 2, leader changed from 8bf6e26e98b14566899578a990b27d47 (127.19.226.193) to 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195). New cstate: current_term: 2 leader_uuid: "787ddcc824c2490a808fc5a53452dc5e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: HEALTHY } } }
I20250109 08:58:01.659701 22653 raft_consensus.cc:988] T 239ca74a4c0a455a9db2652d7debb989 P 787ddcc824c2490a808fc5a53452dc5e: : Instructing follower 88935ab835544e9889d05a6587d8cf21 to start an election
I20250109 08:58:01.660109 22653 raft_consensus.cc:1076] T 239ca74a4c0a455a9db2652d7debb989 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Signalling peer 88935ab835544e9889d05a6587d8cf21 to start an election
I20250109 08:58:01.662124 22172 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "239ca74a4c0a455a9db2652d7debb989"
dest_uuid: "88935ab835544e9889d05a6587d8cf21"
from {username='slave'} at 127.0.0.1:60458
I20250109 08:58:01.662674 22172 raft_consensus.cc:491] T 239ca74a4c0a455a9db2652d7debb989 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:58:01.663021 22172 raft_consensus.cc:3054] T 239ca74a4c0a455a9db2652d7debb989 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:01.669749 22740 raft_consensus.cc:988] T 0cc4fcda876242c396b804edebebb01f P 8bf6e26e98b14566899578a990b27d47: : Instructing follower a2b1a692435241ee988e0bd1661cd4c3 to start an election
I20250109 08:58:01.670430 22738 raft_consensus.cc:1076] T 0cc4fcda876242c396b804edebebb01f P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Signalling peer a2b1a692435241ee988e0bd1661cd4c3 to start an election
I20250109 08:58:01.672223 21589 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "0cc4fcda876242c396b804edebebb01f"
dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3"
from {username='slave'} at 127.0.0.1:34754
I20250109 08:58:01.672786 21589 raft_consensus.cc:491] T 0cc4fcda876242c396b804edebebb01f P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:58:01.673105 21589 raft_consensus.cc:3054] T 0cc4fcda876242c396b804edebebb01f P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:01.676203 22172 raft_consensus.cc:513] T 239ca74a4c0a455a9db2652d7debb989 P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:01.678197 22172 leader_election.cc:290] T 239ca74a4c0a455a9db2652d7debb989 P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 election: Requested vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:58:01.681279 21513 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "239ca74a4c0a455a9db2652d7debb989" candidate_uuid: "88935ab835544e9889d05a6587d8cf21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "8bf6e26e98b14566899578a990b27d47"
I20250109 08:58:01.681881 21513 raft_consensus.cc:3054] T 239ca74a4c0a455a9db2652d7debb989 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:01.682777 21589 raft_consensus.cc:513] T 0cc4fcda876242c396b804edebebb01f P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:01.684867 21658 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "239ca74a4c0a455a9db2652d7debb989" candidate_uuid: "88935ab835544e9889d05a6587d8cf21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "787ddcc824c2490a808fc5a53452dc5e"
I20250109 08:58:01.685406 21658 raft_consensus.cc:3049] T 239ca74a4c0a455a9db2652d7debb989 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Stepping down as leader of term 1
I20250109 08:58:01.685689 21658 raft_consensus.cc:738] T 239ca74a4c0a455a9db2652d7debb989 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Running, Role: LEADER
I20250109 08:58:01.686331 21658 consensus_queue.cc:260] T 239ca74a4c0a455a9db2652d7debb989 P 787ddcc824c2490a808fc5a53452dc5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:01.687709 21658 raft_consensus.cc:3054] T 239ca74a4c0a455a9db2652d7debb989 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:01.691058 22177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cc4fcda876242c396b804edebebb01f" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "88935ab835544e9889d05a6587d8cf21"
I20250109 08:58:01.691634 22177 raft_consensus.cc:3054] T 0cc4fcda876242c396b804edebebb01f P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:01.692452 21513 raft_consensus.cc:2463] T 239ca74a4c0a455a9db2652d7debb989 P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88935ab835544e9889d05a6587d8cf21 in term 2.
I20250109 08:58:01.692852 21657 raft_consensus.cc:1270] T c18c1d46a6ca47598d7b161ed70a4e7a P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Refusing update from remote peer a2b1a692435241ee988e0bd1661cd4c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:58:01.695178 22735 consensus_queue.cc:1035] T c18c1d46a6ca47598d7b161ed70a4e7a P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:58:01.695778 21514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cc4fcda876242c396b804edebebb01f" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "8bf6e26e98b14566899578a990b27d47"
I20250109 08:58:01.696303 21514 raft_consensus.cc:3049] T 0cc4fcda876242c396b804edebebb01f P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Stepping down as leader of term 1
I20250109 08:58:01.696604 21514 raft_consensus.cc:738] T 0cc4fcda876242c396b804edebebb01f P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8bf6e26e98b14566899578a990b27d47, State: Running, Role: LEADER
I20250109 08:58:01.697301 21514 consensus_queue.cc:260] T 0cc4fcda876242c396b804edebebb01f P 8bf6e26e98b14566899578a990b27d47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:01.698371 21514 raft_consensus.cc:3054] T 0cc4fcda876242c396b804edebebb01f P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:01.700881 21589 leader_election.cc:290] T 0cc4fcda876242c396b804edebebb01f P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 2 election: Requested vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), 88935ab835544e9889d05a6587d8cf21 (127.19.226.196:44363)
I20250109 08:58:01.700815 22141 leader_election.cc:304] T 239ca74a4c0a455a9db2652d7debb989 P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88935ab835544e9889d05a6587d8cf21, 8bf6e26e98b14566899578a990b27d47; no voters:
I20250109 08:58:01.701977 22689 raft_consensus.cc:2798] T 239ca74a4c0a455a9db2652d7debb989 P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:58:01.702463 22689 raft_consensus.cc:695] T 239ca74a4c0a455a9db2652d7debb989 P 88935ab835544e9889d05a6587d8cf21 [term 2 LEADER]: Becoming Leader. State: Replica: 88935ab835544e9889d05a6587d8cf21, State: Running, Role: LEADER
I20250109 08:58:01.703286 22689 consensus_queue.cc:237] T 239ca74a4c0a455a9db2652d7debb989 P 88935ab835544e9889d05a6587d8cf21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:01.711793 21514 raft_consensus.cc:2463] T 0cc4fcda876242c396b804edebebb01f P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2b1a692435241ee988e0bd1661cd4c3 in term 2.
I20250109 08:58:01.712916 21552 leader_election.cc:304] T 0cc4fcda876242c396b804edebebb01f P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3; no voters:
I20250109 08:58:01.713821 22735 raft_consensus.cc:2798] T 0cc4fcda876242c396b804edebebb01f P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:58:01.714298 22735 raft_consensus.cc:695] T 0cc4fcda876242c396b804edebebb01f P a2b1a692435241ee988e0bd1661cd4c3 [term 2 LEADER]: Becoming Leader. State: Replica: a2b1a692435241ee988e0bd1661cd4c3, State: Running, Role: LEADER
I20250109 08:58:01.714103 22177 raft_consensus.cc:2463] T 0cc4fcda876242c396b804edebebb01f P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2b1a692435241ee988e0bd1661cd4c3 in term 2.
I20250109 08:58:01.715148 21658 raft_consensus.cc:2463] T 239ca74a4c0a455a9db2652d7debb989 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88935ab835544e9889d05a6587d8cf21 in term 2.
I20250109 08:58:01.715231 22735 consensus_queue.cc:237] T 0cc4fcda876242c396b804edebebb01f P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:01.735167 21409 catalog_manager.cc:5526] T 239ca74a4c0a455a9db2652d7debb989 P 88935ab835544e9889d05a6587d8cf21 reported cstate change: term changed from 1 to 2, leader changed from 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195) to 88935ab835544e9889d05a6587d8cf21 (127.19.226.196). New cstate: current_term: 2 leader_uuid: "88935ab835544e9889d05a6587d8cf21" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250109 08:58:01.738823 21410 catalog_manager.cc:5526] T 0cc4fcda876242c396b804edebebb01f P a2b1a692435241ee988e0bd1661cd4c3 reported cstate change: term changed from 1 to 2, leader changed from 8bf6e26e98b14566899578a990b27d47 (127.19.226.193) to a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194). New cstate: current_term: 2 leader_uuid: "a2b1a692435241ee988e0bd1661cd4c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:01.903913 21658 raft_consensus.cc:1270] T c761cb691bd241ae825917a3fbc3ad24 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Refusing update from remote peer 8bf6e26e98b14566899578a990b27d47: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:58:01.904981 22740 consensus_queue.cc:1035] T c761cb691bd241ae825917a3fbc3ad24 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:58:01.929844 21588 raft_consensus.cc:1270] T c761cb691bd241ae825917a3fbc3ad24 P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Refusing update from remote peer 8bf6e26e98b14566899578a990b27d47: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:58:01.931499 22738 consensus_queue.cc:1035] T c761cb691bd241ae825917a3fbc3ad24 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:58:01.999433 21589 raft_consensus.cc:1270] T 7a7211b8d4374ad1afd82e8cf85bad81 P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:58:02.000615 22665 consensus_queue.cc:1035] T 7a7211b8d4374ad1afd82e8cf85bad81 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:58:02.008725 21514 raft_consensus.cc:1270] T 7a7211b8d4374ad1afd82e8cf85bad81 P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:58:02.010002 22665 consensus_queue.cc:1035] T 7a7211b8d4374ad1afd82e8cf85bad81 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:58:02.129207 21513 raft_consensus.cc:1270] T 239ca74a4c0a455a9db2652d7debb989 P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Refusing update from remote peer 88935ab835544e9889d05a6587d8cf21: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250109 08:58:02.130465 22787 consensus_queue.cc:1035] T 239ca74a4c0a455a9db2652d7debb989 P 88935ab835544e9889d05a6587d8cf21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250109 08:58:02.146266 21513 raft_consensus.cc:1270] T 56bbc306275e4a409e028a7ca1dd1602 P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:58:02.147290 21658 raft_consensus.cc:1270] T 239ca74a4c0a455a9db2652d7debb989 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Refusing update from remote peer 88935ab835544e9889d05a6587d8cf21: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250109 08:58:02.148734 22679 consensus_queue.cc:1035] T 239ca74a4c0a455a9db2652d7debb989 P 88935ab835544e9889d05a6587d8cf21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250109 08:58:02.151732 22653 consensus_queue.cc:1035] T 56bbc306275e4a409e028a7ca1dd1602 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:58:02.169338 22177 raft_consensus.cc:1270] T 0cc4fcda876242c396b804edebebb01f P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Refusing update from remote peer a2b1a692435241ee988e0bd1661cd4c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250109 08:58:02.171114 22735 consensus_queue.cc:1035] T 0cc4fcda876242c396b804edebebb01f P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250109 08:58:02.190469 21514 raft_consensus.cc:1270] T 0cc4fcda876242c396b804edebebb01f P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Refusing update from remote peer a2b1a692435241ee988e0bd1661cd4c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250109 08:58:02.205591 22735 consensus_queue.cc:1035] T 0cc4fcda876242c396b804edebebb01f P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250109 08:58:02.212916 21589 raft_consensus.cc:1270] T 56bbc306275e4a409e028a7ca1dd1602 P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:58:02.218766 22653 consensus_queue.cc:1035] T 56bbc306275e4a409e028a7ca1dd1602 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:58:02.304303 21658 consensus_queue.cc:237] T 1b582ab8d8f044d9b9038a28e958feeb P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:58:02.309788 21583 raft_consensus.cc:1270] T 1b582ab8d8f044d9b9038a28e958feeb P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250109 08:58:02.310223 21514 raft_consensus.cc:1270] T 1b582ab8d8f044d9b9038a28e958feeb P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20250109 08:58:02.310482 21627 consensus_peers.cc:487] T 1b582ab8d8f044d9b9038a28e958feeb P 787ddcc824c2490a808fc5a53452dc5e -> Peer 88935ab835544e9889d05a6587d8cf21 (127.19.226.196:44363): Couldn't send request to peer 88935ab835544e9889d05a6587d8cf21. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1b582ab8d8f044d9b9038a28e958feeb. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:02.311331 22750 consensus_queue.cc:1035] T 1b582ab8d8f044d9b9038a28e958feeb P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:58:02.313179 22653 consensus_queue.cc:1035] T 1b582ab8d8f044d9b9038a28e958feeb P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:58:02.334842 22665 raft_consensus.cc:2949] T 1b582ab8d8f044d9b9038a28e958feeb P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:58:02.336270 21514 raft_consensus.cc:2949] T 1b582ab8d8f044d9b9038a28e958feeb P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:58:02.338979 21583 raft_consensus.cc:2949] T 1b582ab8d8f044d9b9038a28e958feeb P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:58:02.351658 21589 consensus_queue.cc:237] T f5f85ad7c7c94acfaf9d550d55fce48c P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:58:02.359421 21409 catalog_manager.cc:5526] T 1b582ab8d8f044d9b9038a28e958feeb P 787ddcc824c2490a808fc5a53452dc5e reported cstate change: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New cstate: current_term: 1 leader_uuid: "787ddcc824c2490a808fc5a53452dc5e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:02.374859 21658 raft_consensus.cc:1270] T f5f85ad7c7c94acfaf9d550d55fce48c P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Refusing update from remote peer a2b1a692435241ee988e0bd1661cd4c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20250109 08:58:02.376318 21553 consensus_peers.cc:487] T f5f85ad7c7c94acfaf9d550d55fce48c P a2b1a692435241ee988e0bd1661cd4c3 -> Peer 88935ab835544e9889d05a6587d8cf21 (127.19.226.196:44363): Couldn't send request to peer 88935ab835544e9889d05a6587d8cf21. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f5f85ad7c7c94acfaf9d550d55fce48c. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:02.376611 21514 raft_consensus.cc:1270] T f5f85ad7c7c94acfaf9d550d55fce48c P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Refusing update from remote peer a2b1a692435241ee988e0bd1661cd4c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250109 08:58:02.376498 22735 consensus_queue.cc:1035] T f5f85ad7c7c94acfaf9d550d55fce48c P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20250109 08:58:02.378614 22735 consensus_queue.cc:1035] T f5f85ad7c7c94acfaf9d550d55fce48c P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:58:02.391403 22735 raft_consensus.cc:2949] T f5f85ad7c7c94acfaf9d550d55fce48c P a2b1a692435241ee988e0bd1661cd4c3 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:58:02.396471 21658 raft_consensus.cc:2949] T f5f85ad7c7c94acfaf9d550d55fce48c P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:58:02.399709 21512 raft_consensus.cc:2949] T f5f85ad7c7c94acfaf9d550d55fce48c P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:58:02.428164 20363 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250109 08:58:02.428741 21509 consensus_queue.cc:237] T 71ac0243d72845659291184835652dde P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:58:02.432533 21409 catalog_manager.cc:5526] T f5f85ad7c7c94acfaf9d550d55fce48c P a2b1a692435241ee988e0bd1661cd4c3 reported cstate change: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New cstate: current_term: 1 leader_uuid: "a2b1a692435241ee988e0bd1661cd4c3" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20250109 08:58:02.458607 21478 consensus_peers.cc:487] T 71ac0243d72845659291184835652dde P 8bf6e26e98b14566899578a990b27d47 -> Peer 88935ab835544e9889d05a6587d8cf21 (127.19.226.196:44363): Couldn't send request to peer 88935ab835544e9889d05a6587d8cf21. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 71ac0243d72845659291184835652dde. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:02.459965 21589 raft_consensus.cc:1270] T 71ac0243d72845659291184835652dde P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Refusing update from remote peer 8bf6e26e98b14566899578a990b27d47: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250109 08:58:02.460264 21658 raft_consensus.cc:1270] T 71ac0243d72845659291184835652dde P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Refusing update from remote peer 8bf6e26e98b14566899578a990b27d47: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250109 08:58:02.461663 22738 consensus_queue.cc:1035] T 71ac0243d72845659291184835652dde P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:58:02.463375 22738 consensus_queue.cc:1035] T 71ac0243d72845659291184835652dde P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:58:02.477224 22737 raft_consensus.cc:2949] T 71ac0243d72845659291184835652dde P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:58:02.498947 21657 raft_consensus.cc:2949] T 71ac0243d72845659291184835652dde P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:58:02.506605 21410 catalog_manager.cc:5526] T 71ac0243d72845659291184835652dde P 8bf6e26e98b14566899578a990b27d47 reported cstate change: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New cstate: current_term: 1 leader_uuid: "8bf6e26e98b14566899578a990b27d47" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:02.533069 21656 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "306c4892bbff4dbfbe5a0645294aa627"
dest_uuid: "787ddcc824c2490a808fc5a53452dc5e"
mode: GRACEFUL
new_leader_uuid: "88935ab835544e9889d05a6587d8cf21"
from {username='slave'} at 127.0.0.1:55054
I20250109 08:58:02.533650 21656 raft_consensus.cc:604] T 306c4892bbff4dbfbe5a0645294aa627 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Received request to transfer leadership to TS 88935ab835544e9889d05a6587d8cf21
I20250109 08:58:02.506958 21660 consensus_queue.cc:237] T d8493108cba04514bd2236192cef1079 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:58:02.548110 21588 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "610441ef4b1d40c284eff3a714e4214e"
dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3"
mode: GRACEFUL
new_leader_uuid: "8bf6e26e98b14566899578a990b27d47"
from {username='slave'} at 127.0.0.1:46832
I20250109 08:58:02.548736 21588 raft_consensus.cc:604] T 610441ef4b1d40c284eff3a714e4214e P a2b1a692435241ee988e0bd1661cd4c3 [term 2 LEADER]: Received request to transfer leadership to TS 8bf6e26e98b14566899578a990b27d47
I20250109 08:58:02.544998 21589 raft_consensus.cc:2949] T 71ac0243d72845659291184835652dde P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:58:02.569670 21583 raft_consensus.cc:1270] T d8493108cba04514bd2236192cef1079 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250109 08:58:02.571105 22665 consensus_queue.cc:1035] T d8493108cba04514bd2236192cef1079 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:58:02.571633 21509 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "a3aa271f2f0144eea9261e0e07ab833b"
dest_uuid: "8bf6e26e98b14566899578a990b27d47"
mode: GRACEFUL
new_leader_uuid: "787ddcc824c2490a808fc5a53452dc5e"
from {username='slave'} at 127.0.0.1:39274
I20250109 08:58:02.572190 21509 raft_consensus.cc:604] T a3aa271f2f0144eea9261e0e07ab833b P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Received request to transfer leadership to TS 787ddcc824c2490a808fc5a53452dc5e
I20250109 08:58:02.579082 21656 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "eb4516b9a16041c2adf34be520768f3f"
dest_uuid: "787ddcc824c2490a808fc5a53452dc5e"
mode: GRACEFUL
new_leader_uuid: "a2b1a692435241ee988e0bd1661cd4c3"
from {username='slave'} at 127.0.0.1:55054
I20250109 08:58:02.579653 21656 raft_consensus.cc:604] T eb4516b9a16041c2adf34be520768f3f P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Received request to transfer leadership to TS a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:58:02.580634 20363 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 4
I20250109 08:58:02.582289 20363 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250109 08:58:02.582940 20363 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250109 08:58:02.605330 22665 raft_consensus.cc:2949] T d8493108cba04514bd2236192cef1079 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:58:02.610075 21583 raft_consensus.cc:2949] T d8493108cba04514bd2236192cef1079 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
W20250109 08:58:02.626246 21552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.226.195:35731, user_credentials={real_user=slave}} blocked reactor thread for 80599us
W20250109 08:58:02.626783 21627 consensus_peers.cc:487] T d8493108cba04514bd2236192cef1079 P 787ddcc824c2490a808fc5a53452dc5e -> Peer 88935ab835544e9889d05a6587d8cf21 (127.19.226.196:44363): Couldn't send request to peer 88935ab835544e9889d05a6587d8cf21. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d8493108cba04514bd2236192cef1079. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:02.633916 21514 raft_consensus.cc:1270] T d8493108cba04514bd2236192cef1079 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250109 08:58:02.635061 22653 consensus_queue.cc:1035] T d8493108cba04514bd2236192cef1079 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:58:02.643370 21514 raft_consensus.cc:2949] T d8493108cba04514bd2236192cef1079 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:58:02.648777 21412 catalog_manager.cc:5526] T d8493108cba04514bd2236192cef1079 P a2b1a692435241ee988e0bd1661cd4c3 reported cstate change: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New cstate: current_term: 1 leader_uuid: "787ddcc824c2490a808fc5a53452dc5e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:58:02.725834 22743 raft_consensus.cc:988] T 610441ef4b1d40c284eff3a714e4214e P a2b1a692435241ee988e0bd1661cd4c3: : Instructing follower 8bf6e26e98b14566899578a990b27d47 to start an election
I20250109 08:58:02.727885 22773 raft_consensus.cc:1076] T 610441ef4b1d40c284eff3a714e4214e P a2b1a692435241ee988e0bd1661cd4c3 [term 2 LEADER]: Signalling peer 8bf6e26e98b14566899578a990b27d47 to start an election
I20250109 08:58:02.729399 21514 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "610441ef4b1d40c284eff3a714e4214e"
dest_uuid: "8bf6e26e98b14566899578a990b27d47"
from {username='slave'} at 127.0.0.1:39250
I20250109 08:58:02.729923 21514 raft_consensus.cc:491] T 610441ef4b1d40c284eff3a714e4214e P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:58:02.730259 21514 raft_consensus.cc:3054] T 610441ef4b1d40c284eff3a714e4214e P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Advancing to term 3
I20250109 08:58:02.737826 22660 raft_consensus.cc:988] T a3aa271f2f0144eea9261e0e07ab833b P 8bf6e26e98b14566899578a990b27d47: : Instructing follower 787ddcc824c2490a808fc5a53452dc5e to start an election
I20250109 08:58:02.738319 22660 raft_consensus.cc:1076] T a3aa271f2f0144eea9261e0e07ab833b P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Signalling peer 787ddcc824c2490a808fc5a53452dc5e to start an election
I20250109 08:58:02.740948 21514 raft_consensus.cc:513] T 610441ef4b1d40c284eff3a714e4214e P 8bf6e26e98b14566899578a990b27d47 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:58:02.741853 21657 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "a3aa271f2f0144eea9261e0e07ab833b"
dest_uuid: "787ddcc824c2490a808fc5a53452dc5e"
from {username='slave'} at 127.0.0.1:58440
I20250109 08:58:02.742344 21657 raft_consensus.cc:491] T a3aa271f2f0144eea9261e0e07ab833b P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:58:02.742663 21657 raft_consensus.cc:3054] T a3aa271f2f0144eea9261e0e07ab833b P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:02.743260 21583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "610441ef4b1d40c284eff3a714e4214e" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3"
I20250109 08:58:02.743817 21583 raft_consensus.cc:3049] T 610441ef4b1d40c284eff3a714e4214e P a2b1a692435241ee988e0bd1661cd4c3 [term 2 LEADER]: Stepping down as leader of term 2
I20250109 08:58:02.744086 21583 raft_consensus.cc:738] T 610441ef4b1d40c284eff3a714e4214e P a2b1a692435241ee988e0bd1661cd4c3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a2b1a692435241ee988e0bd1661cd4c3, State: Running, Role: LEADER
I20250109 08:58:02.744797 21583 consensus_queue.cc:260] T 610441ef4b1d40c284eff3a714e4214e P a2b1a692435241ee988e0bd1661cd4c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:58:02.753499 21583 raft_consensus.cc:3054] T 610441ef4b1d40c284eff3a714e4214e P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Advancing to term 3
I20250109 08:58:02.759517 21514 leader_election.cc:290] T 610441ef4b1d40c284eff3a714e4214e P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 3 election: Requested vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:58:02.759985 21660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "610441ef4b1d40c284eff3a714e4214e" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "787ddcc824c2490a808fc5a53452dc5e"
I20250109 08:58:02.760569 21660 raft_consensus.cc:3054] T 610441ef4b1d40c284eff3a714e4214e P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Advancing to term 3
I20250109 08:58:02.762203 21583 raft_consensus.cc:2463] T 610441ef4b1d40c284eff3a714e4214e P a2b1a692435241ee988e0bd1661cd4c3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8bf6e26e98b14566899578a990b27d47 in term 3.
I20250109 08:58:02.763351 21478 leader_election.cc:304] T 610441ef4b1d40c284eff3a714e4214e P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3; no voters:
I20250109 08:58:02.764034 22814 raft_consensus.cc:2798] T 610441ef4b1d40c284eff3a714e4214e P 8bf6e26e98b14566899578a990b27d47 [term 3 FOLLOWER]: Leader election won for term 3
I20250109 08:58:02.764626 22814 raft_consensus.cc:695] T 610441ef4b1d40c284eff3a714e4214e P 8bf6e26e98b14566899578a990b27d47 [term 3 LEADER]: Becoming Leader. State: Replica: 8bf6e26e98b14566899578a990b27d47, State: Running, Role: LEADER
I20250109 08:58:02.765578 22814 consensus_queue.cc:237] T 610441ef4b1d40c284eff3a714e4214e P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:58:02.767746 21657 raft_consensus.cc:513] T a3aa271f2f0144eea9261e0e07ab833b P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:58:02.769464 21657 leader_election.cc:290] T a3aa271f2f0144eea9261e0e07ab833b P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 2 election: Requested vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
I20250109 08:58:02.770015 21514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3aa271f2f0144eea9261e0e07ab833b" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8bf6e26e98b14566899578a990b27d47"
I20250109 08:58:02.770567 21514 raft_consensus.cc:3049] T a3aa271f2f0144eea9261e0e07ab833b P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Stepping down as leader of term 1
I20250109 08:58:02.770840 21514 raft_consensus.cc:738] T a3aa271f2f0144eea9261e0e07ab833b P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8bf6e26e98b14566899578a990b27d47, State: Running, Role: LEADER
I20250109 08:58:02.771515 21514 consensus_queue.cc:260] T a3aa271f2f0144eea9261e0e07ab833b P 8bf6e26e98b14566899578a990b27d47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:58:02.772583 21514 raft_consensus.cc:3054] T a3aa271f2f0144eea9261e0e07ab833b P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:02.767113 21660 raft_consensus.cc:2463] T 610441ef4b1d40c284eff3a714e4214e P 787ddcc824c2490a808fc5a53452dc5e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8bf6e26e98b14566899578a990b27d47 in term 3.
I20250109 08:58:02.774959 21583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3aa271f2f0144eea9261e0e07ab833b" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3"
I20250109 08:58:02.776167 21583 raft_consensus.cc:3054] T a3aa271f2f0144eea9261e0e07ab833b P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:02.788455 21514 raft_consensus.cc:2463] T a3aa271f2f0144eea9261e0e07ab833b P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 2.
I20250109 08:58:02.789492 21626 leader_election.cc:304] T a3aa271f2f0144eea9261e0e07ab833b P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, 8bf6e26e98b14566899578a990b27d47; no voters:
I20250109 08:58:02.790199 22653 raft_consensus.cc:2798] T a3aa271f2f0144eea9261e0e07ab833b P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:58:02.790613 22653 raft_consensus.cc:695] T a3aa271f2f0144eea9261e0e07ab833b P 787ddcc824c2490a808fc5a53452dc5e [term 2 LEADER]: Becoming Leader. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Running, Role: LEADER
I20250109 08:58:02.791347 22653 consensus_queue.cc:237] T a3aa271f2f0144eea9261e0e07ab833b P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:58:02.788689 21412 catalog_manager.cc:5526] T 610441ef4b1d40c284eff3a714e4214e P 8bf6e26e98b14566899578a990b27d47 reported cstate change: term changed from 2 to 3, leader changed from a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194) to 8bf6e26e98b14566899578a990b27d47 (127.19.226.193). New cstate: current_term: 3 leader_uuid: "8bf6e26e98b14566899578a990b27d47" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:02.807401 21583 raft_consensus.cc:2463] T a3aa271f2f0144eea9261e0e07ab833b P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 2.
I20250109 08:58:02.817694 22822 ts_tablet_manager.cc:927] T 1b582ab8d8f044d9b9038a28e958feeb P 88935ab835544e9889d05a6587d8cf21: Initiating tablet copy from peer 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:58:02.823565 21412 catalog_manager.cc:5526] T a3aa271f2f0144eea9261e0e07ab833b P 787ddcc824c2490a808fc5a53452dc5e reported cstate change: term changed from 1 to 2, leader changed from 8bf6e26e98b14566899578a990b27d47 (127.19.226.193) to 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195). New cstate: current_term: 2 leader_uuid: "787ddcc824c2490a808fc5a53452dc5e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: HEALTHY } } }
I20250109 08:58:02.837678 22822 tablet_copy_client.cc:323] T 1b582ab8d8f044d9b9038a28e958feeb P 88935ab835544e9889d05a6587d8cf21: tablet copy: Beginning tablet copy session from remote peer at address 127.19.226.195:35731
I20250109 08:58:02.840693 21673 tablet_copy_service.cc:140] P 787ddcc824c2490a808fc5a53452dc5e: Received BeginTabletCopySession request for tablet 1b582ab8d8f044d9b9038a28e958feeb from peer 88935ab835544e9889d05a6587d8cf21 ({username='slave'} at 127.0.0.1:52376)
I20250109 08:58:02.841226 21673 tablet_copy_service.cc:161] P 787ddcc824c2490a808fc5a53452dc5e: Beginning new tablet copy session on tablet 1b582ab8d8f044d9b9038a28e958feeb from peer 88935ab835544e9889d05a6587d8cf21 at {username='slave'} at 127.0.0.1:52376: session id = 88935ab835544e9889d05a6587d8cf21-1b582ab8d8f044d9b9038a28e958feeb
I20250109 08:58:02.849256 22824 ts_tablet_manager.cc:927] T f5f85ad7c7c94acfaf9d550d55fce48c P 88935ab835544e9889d05a6587d8cf21: Initiating tablet copy from peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
I20250109 08:58:02.851912 21673 tablet_copy_source_session.cc:215] T 1b582ab8d8f044d9b9038a28e958feeb P 787ddcc824c2490a808fc5a53452dc5e: Tablet Copy: opened 0 blocks and 1 log segments
I20250109 08:58:02.855866 22822 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b582ab8d8f044d9b9038a28e958feeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:02.857362 22824 tablet_copy_client.cc:323] T f5f85ad7c7c94acfaf9d550d55fce48c P 88935ab835544e9889d05a6587d8cf21: tablet copy: Beginning tablet copy session from remote peer at address 127.19.226.194:45527
I20250109 08:58:02.859025 21599 tablet_copy_service.cc:140] P a2b1a692435241ee988e0bd1661cd4c3: Received BeginTabletCopySession request for tablet f5f85ad7c7c94acfaf9d550d55fce48c from peer 88935ab835544e9889d05a6587d8cf21 ({username='slave'} at 127.0.0.1:33652)
I20250109 08:58:02.859524 21599 tablet_copy_service.cc:161] P a2b1a692435241ee988e0bd1661cd4c3: Beginning new tablet copy session on tablet f5f85ad7c7c94acfaf9d550d55fce48c from peer 88935ab835544e9889d05a6587d8cf21 at {username='slave'} at 127.0.0.1:33652: session id = 88935ab835544e9889d05a6587d8cf21-f5f85ad7c7c94acfaf9d550d55fce48c
I20250109 08:58:02.866346 21599 tablet_copy_source_session.cc:215] T f5f85ad7c7c94acfaf9d550d55fce48c P a2b1a692435241ee988e0bd1661cd4c3: Tablet Copy: opened 0 blocks and 1 log segments
I20250109 08:58:02.869338 22824 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5f85ad7c7c94acfaf9d550d55fce48c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:02.885197 22653 raft_consensus.cc:988] T 306c4892bbff4dbfbe5a0645294aa627 P 787ddcc824c2490a808fc5a53452dc5e: : Instructing follower 88935ab835544e9889d05a6587d8cf21 to start an election
I20250109 08:58:02.885710 22750 raft_consensus.cc:1076] T 306c4892bbff4dbfbe5a0645294aa627 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Signalling peer 88935ab835544e9889d05a6587d8cf21 to start an election
I20250109 08:58:02.887248 22172 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "306c4892bbff4dbfbe5a0645294aa627"
dest_uuid: "88935ab835544e9889d05a6587d8cf21"
from {username='slave'} at 127.0.0.1:60458
I20250109 08:58:02.887784 22172 raft_consensus.cc:491] T 306c4892bbff4dbfbe5a0645294aa627 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:58:02.888101 22172 raft_consensus.cc:3054] T 306c4892bbff4dbfbe5a0645294aa627 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:02.898445 22172 raft_consensus.cc:513] T 306c4892bbff4dbfbe5a0645294aa627 P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:02.902542 22750 raft_consensus.cc:988] T eb4516b9a16041c2adf34be520768f3f P 787ddcc824c2490a808fc5a53452dc5e: : Instructing follower a2b1a692435241ee988e0bd1661cd4c3 to start an election
I20250109 08:58:02.903044 22750 raft_consensus.cc:1076] T eb4516b9a16041c2adf34be520768f3f P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Signalling peer a2b1a692435241ee988e0bd1661cd4c3 to start an election
I20250109 08:58:02.904101 22172 leader_election.cc:290] T 306c4892bbff4dbfbe5a0645294aa627 P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 election: Requested vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:58:02.905010 21657 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "306c4892bbff4dbfbe5a0645294aa627" candidate_uuid: "88935ab835544e9889d05a6587d8cf21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "787ddcc824c2490a808fc5a53452dc5e"
I20250109 08:58:02.905566 21657 raft_consensus.cc:3049] T 306c4892bbff4dbfbe5a0645294aa627 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Stepping down as leader of term 1
I20250109 08:58:02.905833 21657 raft_consensus.cc:738] T 306c4892bbff4dbfbe5a0645294aa627 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Running, Role: LEADER
I20250109 08:58:02.906479 21657 consensus_queue.cc:260] T 306c4892bbff4dbfbe5a0645294aa627 P 787ddcc824c2490a808fc5a53452dc5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:02.907526 21657 raft_consensus.cc:3054] T 306c4892bbff4dbfbe5a0645294aa627 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:02.910181 21583 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "eb4516b9a16041c2adf34be520768f3f"
dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3"
from {username='slave'} at 127.0.0.1:34746
I20250109 08:58:02.910719 21583 raft_consensus.cc:491] T eb4516b9a16041c2adf34be520768f3f P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:58:02.910864 22824 tablet_copy_client.cc:806] T f5f85ad7c7c94acfaf9d550d55fce48c P 88935ab835544e9889d05a6587d8cf21: tablet copy: Starting download of 0 data blocks...
I20250109 08:58:02.911352 22824 tablet_copy_client.cc:670] T f5f85ad7c7c94acfaf9d550d55fce48c P 88935ab835544e9889d05a6587d8cf21: tablet copy: Starting download of 1 WAL segments...
I20250109 08:58:02.904176 21588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "306c4892bbff4dbfbe5a0645294aa627" candidate_uuid: "88935ab835544e9889d05a6587d8cf21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3"
I20250109 08:58:02.912120 21588 raft_consensus.cc:3054] T 306c4892bbff4dbfbe5a0645294aa627 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:02.914906 22824 tablet_copy_client.cc:538] T f5f85ad7c7c94acfaf9d550d55fce48c P 88935ab835544e9889d05a6587d8cf21: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250109 08:58:02.916482 21583 raft_consensus.cc:3054] T eb4516b9a16041c2adf34be520768f3f P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:02.922024 21583 raft_consensus.cc:513] T eb4516b9a16041c2adf34be520768f3f P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:02.924263 21656 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb4516b9a16041c2adf34be520768f3f" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "787ddcc824c2490a808fc5a53452dc5e"
I20250109 08:58:02.924897 21656 raft_consensus.cc:3049] T eb4516b9a16041c2adf34be520768f3f P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Stepping down as leader of term 1
I20250109 08:58:02.925184 21656 raft_consensus.cc:738] T eb4516b9a16041c2adf34be520768f3f P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Running, Role: LEADER
I20250109 08:58:02.925909 21657 raft_consensus.cc:2463] T 306c4892bbff4dbfbe5a0645294aa627 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88935ab835544e9889d05a6587d8cf21 in term 2.
I20250109 08:58:02.925837 21656 consensus_queue.cc:260] T eb4516b9a16041c2adf34be520768f3f P 787ddcc824c2490a808fc5a53452dc5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:02.927991 21656 raft_consensus.cc:3054] T eb4516b9a16041c2adf34be520768f3f P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:02.929208 22822 tablet_copy_client.cc:806] T 1b582ab8d8f044d9b9038a28e958feeb P 88935ab835544e9889d05a6587d8cf21: tablet copy: Starting download of 0 data blocks...
I20250109 08:58:02.923074 22824 tablet_bootstrap.cc:492] T f5f85ad7c7c94acfaf9d550d55fce48c P 88935ab835544e9889d05a6587d8cf21: Bootstrap starting.
I20250109 08:58:02.929811 22822 tablet_copy_client.cc:670] T 1b582ab8d8f044d9b9038a28e958feeb P 88935ab835544e9889d05a6587d8cf21: tablet copy: Starting download of 1 WAL segments...
I20250109 08:58:02.930332 22141 leader_election.cc:304] T 306c4892bbff4dbfbe5a0645294aa627 P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, 88935ab835544e9889d05a6587d8cf21; no voters:
I20250109 08:58:02.931602 22689 raft_consensus.cc:2798] T 306c4892bbff4dbfbe5a0645294aa627 P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:58:02.932050 22689 raft_consensus.cc:695] T 306c4892bbff4dbfbe5a0645294aa627 P 88935ab835544e9889d05a6587d8cf21 [term 2 LEADER]: Becoming Leader. State: Replica: 88935ab835544e9889d05a6587d8cf21, State: Running, Role: LEADER
I20250109 08:58:02.932868 22689 consensus_queue.cc:237] T 306c4892bbff4dbfbe5a0645294aa627 P 88935ab835544e9889d05a6587d8cf21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:02.926931 21588 raft_consensus.cc:2463] T 306c4892bbff4dbfbe5a0645294aa627 P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88935ab835544e9889d05a6587d8cf21 in term 2.
I20250109 08:58:02.939580 21583 leader_election.cc:290] T eb4516b9a16041c2adf34be520768f3f P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 2 election: Requested vote from peers 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731), 88935ab835544e9889d05a6587d8cf21 (127.19.226.196:44363)
I20250109 08:58:02.940721 21656 raft_consensus.cc:2463] T eb4516b9a16041c2adf34be520768f3f P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2b1a692435241ee988e0bd1661cd4c3 in term 2.
I20250109 08:58:02.943850 21552 leader_election.cc:304] T eb4516b9a16041c2adf34be520768f3f P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, a2b1a692435241ee988e0bd1661cd4c3; no voters:
I20250109 08:58:02.945322 22773 raft_consensus.cc:2798] T eb4516b9a16041c2adf34be520768f3f P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:58:02.945765 22172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb4516b9a16041c2adf34be520768f3f" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "88935ab835544e9889d05a6587d8cf21"
I20250109 08:58:02.946530 22172 raft_consensus.cc:3054] T eb4516b9a16041c2adf34be520768f3f P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:02.945799 22773 raft_consensus.cc:695] T eb4516b9a16041c2adf34be520768f3f P a2b1a692435241ee988e0bd1661cd4c3 [term 2 LEADER]: Becoming Leader. State: Replica: a2b1a692435241ee988e0bd1661cd4c3, State: Running, Role: LEADER
I20250109 08:58:02.948707 22773 consensus_queue.cc:237] T eb4516b9a16041c2adf34be520768f3f P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:02.959383 21412 catalog_manager.cc:5526] T 306c4892bbff4dbfbe5a0645294aa627 P 88935ab835544e9889d05a6587d8cf21 reported cstate change: term changed from 1 to 2, leader changed from 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195) to 88935ab835544e9889d05a6587d8cf21 (127.19.226.196). New cstate: current_term: 2 leader_uuid: "88935ab835544e9889d05a6587d8cf21" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250109 08:58:02.968434 22172 raft_consensus.cc:2463] T eb4516b9a16041c2adf34be520768f3f P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2b1a692435241ee988e0bd1661cd4c3 in term 2.
I20250109 08:58:02.941114 22822 tablet_copy_client.cc:538] T 1b582ab8d8f044d9b9038a28e958feeb P 88935ab835544e9889d05a6587d8cf21: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250109 08:58:02.989835 21412 catalog_manager.cc:5526] T eb4516b9a16041c2adf34be520768f3f P a2b1a692435241ee988e0bd1661cd4c3 reported cstate change: term changed from 1 to 2, leader changed from 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195) to a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194). New cstate: current_term: 2 leader_uuid: "a2b1a692435241ee988e0bd1661cd4c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:03.011435 22827 ts_tablet_manager.cc:927] T 71ac0243d72845659291184835652dde P 88935ab835544e9889d05a6587d8cf21: Initiating tablet copy from peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725)
I20250109 08:58:03.012394 22824 tablet_bootstrap.cc:492] T f5f85ad7c7c94acfaf9d550d55fce48c P 88935ab835544e9889d05a6587d8cf21: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250109 08:58:03.013299 22824 tablet_bootstrap.cc:492] T f5f85ad7c7c94acfaf9d550d55fce48c P 88935ab835544e9889d05a6587d8cf21: Bootstrap complete.
I20250109 08:58:03.014032 22824 ts_tablet_manager.cc:1397] T f5f85ad7c7c94acfaf9d550d55fce48c P 88935ab835544e9889d05a6587d8cf21: Time spent bootstrapping tablet: real 0.091s user 0.030s sys 0.010s
I20250109 08:58:03.016867 22824 raft_consensus.cc:357] T f5f85ad7c7c94acfaf9d550d55fce48c P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:58:03.017849 22824 raft_consensus.cc:738] T f5f85ad7c7c94acfaf9d550d55fce48c P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 88935ab835544e9889d05a6587d8cf21, State: Initialized, Role: LEARNER
I20250109 08:58:03.018528 22824 consensus_queue.cc:260] T f5f85ad7c7c94acfaf9d550d55fce48c P 88935ab835544e9889d05a6587d8cf21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:58:03.022037 22822 tablet_bootstrap.cc:492] T 1b582ab8d8f044d9b9038a28e958feeb P 88935ab835544e9889d05a6587d8cf21: Bootstrap starting.
I20250109 08:58:03.023264 22824 ts_tablet_manager.cc:1428] T f5f85ad7c7c94acfaf9d550d55fce48c P 88935ab835544e9889d05a6587d8cf21: Time spent starting tablet: real 0.009s user 0.003s sys 0.004s
I20250109 08:58:03.026177 21599 tablet_copy_service.cc:342] P a2b1a692435241ee988e0bd1661cd4c3: Request end of tablet copy session 88935ab835544e9889d05a6587d8cf21-f5f85ad7c7c94acfaf9d550d55fce48c received from {username='slave'} at 127.0.0.1:33652
I20250109 08:58:03.026309 22827 tablet_copy_client.cc:323] T 71ac0243d72845659291184835652dde P 88935ab835544e9889d05a6587d8cf21: tablet copy: Beginning tablet copy session from remote peer at address 127.19.226.193:42725
I20250109 08:58:03.026721 21599 tablet_copy_service.cc:434] P a2b1a692435241ee988e0bd1661cd4c3: ending tablet copy session 88935ab835544e9889d05a6587d8cf21-f5f85ad7c7c94acfaf9d550d55fce48c on tablet f5f85ad7c7c94acfaf9d550d55fce48c with peer 88935ab835544e9889d05a6587d8cf21
I20250109 08:58:03.027531 22831 ts_tablet_manager.cc:927] T d8493108cba04514bd2236192cef1079 P 88935ab835544e9889d05a6587d8cf21: Initiating tablet copy from peer 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:58:03.029151 21523 tablet_copy_service.cc:140] P 8bf6e26e98b14566899578a990b27d47: Received BeginTabletCopySession request for tablet 71ac0243d72845659291184835652dde from peer 88935ab835544e9889d05a6587d8cf21 ({username='slave'} at 127.0.0.1:41812)
I20250109 08:58:03.030148 21523 tablet_copy_service.cc:161] P 8bf6e26e98b14566899578a990b27d47: Beginning new tablet copy session on tablet 71ac0243d72845659291184835652dde from peer 88935ab835544e9889d05a6587d8cf21 at {username='slave'} at 127.0.0.1:41812: session id = 88935ab835544e9889d05a6587d8cf21-71ac0243d72845659291184835652dde
I20250109 08:58:03.042275 21523 tablet_copy_source_session.cc:215] T 71ac0243d72845659291184835652dde P 8bf6e26e98b14566899578a990b27d47: Tablet Copy: opened 0 blocks and 1 log segments
I20250109 08:58:03.047060 22827 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 71ac0243d72845659291184835652dde. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:03.051352 22831 tablet_copy_client.cc:323] T d8493108cba04514bd2236192cef1079 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Beginning tablet copy session from remote peer at address 127.19.226.195:35731
I20250109 08:58:03.053273 21673 tablet_copy_service.cc:140] P 787ddcc824c2490a808fc5a53452dc5e: Received BeginTabletCopySession request for tablet d8493108cba04514bd2236192cef1079 from peer 88935ab835544e9889d05a6587d8cf21 ({username='slave'} at 127.0.0.1:52376)
I20250109 08:58:03.053776 21673 tablet_copy_service.cc:161] P 787ddcc824c2490a808fc5a53452dc5e: Beginning new tablet copy session on tablet d8493108cba04514bd2236192cef1079 from peer 88935ab835544e9889d05a6587d8cf21 at {username='slave'} at 127.0.0.1:52376: session id = 88935ab835544e9889d05a6587d8cf21-d8493108cba04514bd2236192cef1079
I20250109 08:58:03.061686 22822 tablet_bootstrap.cc:492] T 1b582ab8d8f044d9b9038a28e958feeb P 88935ab835544e9889d05a6587d8cf21: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250109 08:58:03.062541 22822 tablet_bootstrap.cc:492] T 1b582ab8d8f044d9b9038a28e958feeb P 88935ab835544e9889d05a6587d8cf21: Bootstrap complete.
I20250109 08:58:03.063174 22822 ts_tablet_manager.cc:1397] T 1b582ab8d8f044d9b9038a28e958feeb P 88935ab835544e9889d05a6587d8cf21: Time spent bootstrapping tablet: real 0.041s user 0.020s sys 0.001s
I20250109 08:58:03.066076 21673 tablet_copy_source_session.cc:215] T d8493108cba04514bd2236192cef1079 P 787ddcc824c2490a808fc5a53452dc5e: Tablet Copy: opened 0 blocks and 1 log segments
I20250109 08:58:03.065797 22822 raft_consensus.cc:357] T 1b582ab8d8f044d9b9038a28e958feeb P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:58:03.066622 22822 raft_consensus.cc:738] T 1b582ab8d8f044d9b9038a28e958feeb P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 88935ab835544e9889d05a6587d8cf21, State: Initialized, Role: LEARNER
I20250109 08:58:03.067198 22822 consensus_queue.cc:260] T 1b582ab8d8f044d9b9038a28e958feeb P 88935ab835544e9889d05a6587d8cf21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:58:03.068845 22831 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8493108cba04514bd2236192cef1079. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:03.073028 22822 ts_tablet_manager.cc:1428] T 1b582ab8d8f044d9b9038a28e958feeb P 88935ab835544e9889d05a6587d8cf21: Time spent starting tablet: real 0.010s user 0.006s sys 0.000s
I20250109 08:58:03.074486 21673 tablet_copy_service.cc:342] P 787ddcc824c2490a808fc5a53452dc5e: Request end of tablet copy session 88935ab835544e9889d05a6587d8cf21-1b582ab8d8f044d9b9038a28e958feeb received from {username='slave'} at 127.0.0.1:52376
I20250109 08:58:03.074846 21673 tablet_copy_service.cc:434] P 787ddcc824c2490a808fc5a53452dc5e: ending tablet copy session 88935ab835544e9889d05a6587d8cf21-1b582ab8d8f044d9b9038a28e958feeb on tablet 1b582ab8d8f044d9b9038a28e958feeb with peer 88935ab835544e9889d05a6587d8cf21
I20250109 08:58:03.075314 22827 tablet_copy_client.cc:806] T 71ac0243d72845659291184835652dde P 88935ab835544e9889d05a6587d8cf21: tablet copy: Starting download of 0 data blocks...
I20250109 08:58:03.075894 22827 tablet_copy_client.cc:670] T 71ac0243d72845659291184835652dde P 88935ab835544e9889d05a6587d8cf21: tablet copy: Starting download of 1 WAL segments...
I20250109 08:58:03.083271 22827 tablet_copy_client.cc:538] T 71ac0243d72845659291184835652dde P 88935ab835544e9889d05a6587d8cf21: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250109 08:58:03.083813 22831 tablet_copy_client.cc:806] T d8493108cba04514bd2236192cef1079 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Starting download of 0 data blocks...
I20250109 08:58:03.084321 22831 tablet_copy_client.cc:670] T d8493108cba04514bd2236192cef1079 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Starting download of 1 WAL segments...
I20250109 08:58:03.091269 22827 tablet_bootstrap.cc:492] T 71ac0243d72845659291184835652dde P 88935ab835544e9889d05a6587d8cf21: Bootstrap starting.
I20250109 08:58:03.095110 22831 tablet_copy_client.cc:538] T d8493108cba04514bd2236192cef1079 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250109 08:58:03.108317 22831 tablet_bootstrap.cc:492] T d8493108cba04514bd2236192cef1079 P 88935ab835544e9889d05a6587d8cf21: Bootstrap starting.
I20250109 08:58:03.119577 22827 tablet_bootstrap.cc:492] T 71ac0243d72845659291184835652dde P 88935ab835544e9889d05a6587d8cf21: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250109 08:58:03.124107 22827 tablet_bootstrap.cc:492] T 71ac0243d72845659291184835652dde P 88935ab835544e9889d05a6587d8cf21: Bootstrap complete.
I20250109 08:58:03.124800 22827 ts_tablet_manager.cc:1397] T 71ac0243d72845659291184835652dde P 88935ab835544e9889d05a6587d8cf21: Time spent bootstrapping tablet: real 0.034s user 0.020s sys 0.008s
I20250109 08:58:03.127553 22827 raft_consensus.cc:357] T 71ac0243d72845659291184835652dde P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:58:03.129796 22827 raft_consensus.cc:738] T 71ac0243d72845659291184835652dde P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 88935ab835544e9889d05a6587d8cf21, State: Initialized, Role: LEARNER
I20250109 08:58:03.141746 22827 consensus_queue.cc:260] T 71ac0243d72845659291184835652dde P 88935ab835544e9889d05a6587d8cf21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:58:03.157536 22827 ts_tablet_manager.cc:1428] T 71ac0243d72845659291184835652dde P 88935ab835544e9889d05a6587d8cf21: Time spent starting tablet: real 0.032s user 0.006s sys 0.001s
I20250109 08:58:03.162263 21523 tablet_copy_service.cc:342] P 8bf6e26e98b14566899578a990b27d47: Request end of tablet copy session 88935ab835544e9889d05a6587d8cf21-71ac0243d72845659291184835652dde received from {username='slave'} at 127.0.0.1:41812
I20250109 08:58:03.162706 21523 tablet_copy_service.cc:434] P 8bf6e26e98b14566899578a990b27d47: ending tablet copy session 88935ab835544e9889d05a6587d8cf21-71ac0243d72845659291184835652dde on tablet 71ac0243d72845659291184835652dde with peer 88935ab835544e9889d05a6587d8cf21
I20250109 08:58:03.173209 22831 tablet_bootstrap.cc:492] T d8493108cba04514bd2236192cef1079 P 88935ab835544e9889d05a6587d8cf21: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250109 08:58:03.173920 22831 tablet_bootstrap.cc:492] T d8493108cba04514bd2236192cef1079 P 88935ab835544e9889d05a6587d8cf21: Bootstrap complete.
I20250109 08:58:03.174506 22831 ts_tablet_manager.cc:1397] T d8493108cba04514bd2236192cef1079 P 88935ab835544e9889d05a6587d8cf21: Time spent bootstrapping tablet: real 0.066s user 0.021s sys 0.024s
I20250109 08:58:03.176978 22831 raft_consensus.cc:357] T d8493108cba04514bd2236192cef1079 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:58:03.177577 22831 raft_consensus.cc:738] T d8493108cba04514bd2236192cef1079 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 88935ab835544e9889d05a6587d8cf21, State: Initialized, Role: LEARNER
I20250109 08:58:03.178125 22831 consensus_queue.cc:260] T d8493108cba04514bd2236192cef1079 P 88935ab835544e9889d05a6587d8cf21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:58:03.180655 22831 ts_tablet_manager.cc:1428] T d8493108cba04514bd2236192cef1079 P 88935ab835544e9889d05a6587d8cf21: Time spent starting tablet: real 0.006s user 0.002s sys 0.006s
I20250109 08:58:03.182157 21673 tablet_copy_service.cc:342] P 787ddcc824c2490a808fc5a53452dc5e: Request end of tablet copy session 88935ab835544e9889d05a6587d8cf21-d8493108cba04514bd2236192cef1079 received from {username='slave'} at 127.0.0.1:52376
I20250109 08:58:03.182520 21673 tablet_copy_service.cc:434] P 787ddcc824c2490a808fc5a53452dc5e: ending tablet copy session 88935ab835544e9889d05a6587d8cf21-d8493108cba04514bd2236192cef1079 on tablet d8493108cba04514bd2236192cef1079 with peer 88935ab835544e9889d05a6587d8cf21
I20250109 08:58:03.224916 21512 raft_consensus.cc:1270] T a3aa271f2f0144eea9261e0e07ab833b P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:58:03.226361 22172 raft_consensus.cc:1212] T f5f85ad7c7c94acfaf9d550d55fce48c P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20250109 08:58:03.226594 22653 consensus_queue.cc:1035] T a3aa271f2f0144eea9261e0e07ab833b P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:58:03.258399 21588 raft_consensus.cc:1270] T a3aa271f2f0144eea9261e0e07ab833b P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:58:03.259822 22665 consensus_queue.cc:1035] T a3aa271f2f0144eea9261e0e07ab833b P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:58:03.268723 21656 raft_consensus.cc:1270] T 610441ef4b1d40c284eff3a714e4214e P 787ddcc824c2490a808fc5a53452dc5e [term 3 FOLLOWER]: Refusing update from remote peer 8bf6e26e98b14566899578a990b27d47: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20250109 08:58:03.270627 22814 consensus_queue.cc:1035] T 610441ef4b1d40c284eff3a714e4214e P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:58:03.271273 22177 raft_consensus.cc:1212] T 1b582ab8d8f044d9b9038a28e958feeb P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20250109 08:58:03.308187 21588 raft_consensus.cc:1270] T 610441ef4b1d40c284eff3a714e4214e P a2b1a692435241ee988e0bd1661cd4c3 [term 3 FOLLOWER]: Refusing update from remote peer 8bf6e26e98b14566899578a990b27d47: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20250109 08:58:03.315469 22814 consensus_queue.cc:1035] T 610441ef4b1d40c284eff3a714e4214e P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:58:03.337891 21656 raft_consensus.cc:1270] T 306c4892bbff4dbfbe5a0645294aa627 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Refusing update from remote peer 88935ab835544e9889d05a6587d8cf21: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250109 08:58:03.338964 22689 consensus_queue.cc:1035] T 306c4892bbff4dbfbe5a0645294aa627 P 88935ab835544e9889d05a6587d8cf21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250109 08:58:03.349882 21588 raft_consensus.cc:1270] T 306c4892bbff4dbfbe5a0645294aa627 P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Refusing update from remote peer 88935ab835544e9889d05a6587d8cf21: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250109 08:58:03.350931 22787 consensus_queue.cc:1035] T 306c4892bbff4dbfbe5a0645294aa627 P 88935ab835544e9889d05a6587d8cf21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250109 08:58:03.446202 21657 raft_consensus.cc:1270] T eb4516b9a16041c2adf34be520768f3f P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Refusing update from remote peer a2b1a692435241ee988e0bd1661cd4c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250109 08:58:03.448290 22743 consensus_queue.cc:1035] T eb4516b9a16041c2adf34be520768f3f P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250109 08:58:03.461031 22172 raft_consensus.cc:1270] T eb4516b9a16041c2adf34be520768f3f P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Refusing update from remote peer a2b1a692435241ee988e0bd1661cd4c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250109 08:58:03.462138 22816 consensus_queue.cc:1035] T eb4516b9a16041c2adf34be520768f3f P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250109 08:58:03.500341 22172 raft_consensus.cc:1212] T 71ac0243d72845659291184835652dde P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20250109 08:58:03.500960 22177 raft_consensus.cc:1212] T d8493108cba04514bd2236192cef1079 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20250109 08:58:03.688570 20363 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250109 08:58:03.695097 22665 raft_consensus.cc:1059] T 1b582ab8d8f044d9b9038a28e958feeb P 787ddcc824c2490a808fc5a53452dc5e: attempting to promote NON_VOTER 88935ab835544e9889d05a6587d8cf21 to VOTER
I20250109 08:58:03.697703 22665 consensus_queue.cc:237] T 1b582ab8d8f044d9b9038a28e958feeb P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:03.705408 21588 raft_consensus.cc:1270] T 1b582ab8d8f044d9b9038a28e958feeb P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:58:03.706185 21514 raft_consensus.cc:1270] T 1b582ab8d8f044d9b9038a28e958feeb P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:58:03.705408 22177 raft_consensus.cc:1270] T 1b582ab8d8f044d9b9038a28e958feeb P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:58:03.707373 22750 consensus_queue.cc:1035] T 1b582ab8d8f044d9b9038a28e958feeb P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:58:03.708320 22665 consensus_queue.cc:1035] T 1b582ab8d8f044d9b9038a28e958feeb P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20250109 08:58:03.710388 22653 consensus_queue.cc:1035] T 1b582ab8d8f044d9b9038a28e958feeb P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:58:03.739960 22665 raft_consensus.cc:2949] T 1b582ab8d8f044d9b9038a28e958feeb P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:03.747315 21512 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "37280454805d47f98a415a8f0c16d869"
dest_uuid: "8bf6e26e98b14566899578a990b27d47"
mode: GRACEFUL
new_leader_uuid: "787ddcc824c2490a808fc5a53452dc5e"
from {username='slave'} at 127.0.0.1:39274
I20250109 08:58:03.747871 21512 raft_consensus.cc:604] T 37280454805d47f98a415a8f0c16d869 P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Received request to transfer leadership to TS 787ddcc824c2490a808fc5a53452dc5e
I20250109 08:58:03.750166 21657 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "56bbc306275e4a409e028a7ca1dd1602"
dest_uuid: "787ddcc824c2490a808fc5a53452dc5e"
mode: GRACEFUL
new_leader_uuid: "a2b1a692435241ee988e0bd1661cd4c3"
from {username='slave'} at 127.0.0.1:55054
I20250109 08:58:03.750715 21657 raft_consensus.cc:604] T 56bbc306275e4a409e028a7ca1dd1602 P 787ddcc824c2490a808fc5a53452dc5e [term 2 LEADER]: Received request to transfer leadership to TS a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:58:03.750808 22177 raft_consensus.cc:2949] T 1b582ab8d8f044d9b9038a28e958feeb P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:03.750808 21588 raft_consensus.cc:2949] T 1b582ab8d8f044d9b9038a28e958feeb P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:03.757831 21589 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "97304b06c7fa40ef8345a72c796c47f8"
dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3"
mode: GRACEFUL
new_leader_uuid: "8bf6e26e98b14566899578a990b27d47"
from {username='slave'} at 127.0.0.1:46832
I20250109 08:58:03.758365 21589 raft_consensus.cc:604] T 97304b06c7fa40ef8345a72c796c47f8 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 LEADER]: Received request to transfer leadership to TS 8bf6e26e98b14566899578a990b27d47
I20250109 08:58:03.759553 20363 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 3
I20250109 08:58:03.761188 20363 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250109 08:58:03.761982 20363 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250109 08:58:03.762776 21512 raft_consensus.cc:2949] T 1b582ab8d8f044d9b9038a28e958feeb P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:03.774658 21412 catalog_manager.cc:5526] T 1b582ab8d8f044d9b9038a28e958feeb P 88935ab835544e9889d05a6587d8cf21 reported cstate change: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "787ddcc824c2490a808fc5a53452dc5e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:03.807559 21656 consensus_queue.cc:237] T 1b582ab8d8f044d9b9038a28e958feeb P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:03.813355 22653 raft_consensus.cc:988] T 56bbc306275e4a409e028a7ca1dd1602 P 787ddcc824c2490a808fc5a53452dc5e: : Instructing follower a2b1a692435241ee988e0bd1661cd4c3 to start an election
I20250109 08:58:03.813841 22665 raft_consensus.cc:1076] T 56bbc306275e4a409e028a7ca1dd1602 P 787ddcc824c2490a808fc5a53452dc5e [term 2 LEADER]: Signalling peer a2b1a692435241ee988e0bd1661cd4c3 to start an election
I20250109 08:58:03.815725 21588 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "56bbc306275e4a409e028a7ca1dd1602"
dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3"
from {username='slave'} at 127.0.0.1:34746
I20250109 08:58:03.816350 21588 raft_consensus.cc:491] T 56bbc306275e4a409e028a7ca1dd1602 P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:58:03.816787 21588 raft_consensus.cc:3054] T 56bbc306275e4a409e028a7ca1dd1602 P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Advancing to term 3
I20250109 08:58:03.820030 22177 raft_consensus.cc:1270] T 1b582ab8d8f044d9b9038a28e958feeb P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250109 08:58:03.823577 22750 consensus_queue.cc:1035] T 1b582ab8d8f044d9b9038a28e958feeb P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250109 08:58:03.831821 22743 raft_consensus.cc:1059] T f5f85ad7c7c94acfaf9d550d55fce48c P a2b1a692435241ee988e0bd1661cd4c3: attempting to promote NON_VOTER 88935ab835544e9889d05a6587d8cf21 to VOTER
I20250109 08:58:03.834316 22743 consensus_queue.cc:237] T f5f85ad7c7c94acfaf9d550d55fce48c P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:03.837028 21588 raft_consensus.cc:513] T 56bbc306275e4a409e028a7ca1dd1602 P a2b1a692435241ee988e0bd1661cd4c3 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:58:03.840139 21512 raft_consensus.cc:1270] T 1b582ab8d8f044d9b9038a28e958feeb P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250109 08:58:03.844821 22653 consensus_queue.cc:1035] T 1b582ab8d8f044d9b9038a28e958feeb P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250109 08:58:03.847019 22665 raft_consensus.cc:2949] T 1b582ab8d8f044d9b9038a28e958feeb P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:03.844269 21660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56bbc306275e4a409e028a7ca1dd1602" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "787ddcc824c2490a808fc5a53452dc5e"
I20250109 08:58:03.851351 21660 raft_consensus.cc:3049] T 56bbc306275e4a409e028a7ca1dd1602 P 787ddcc824c2490a808fc5a53452dc5e [term 2 LEADER]: Stepping down as leader of term 2
I20250109 08:58:03.851661 21660 raft_consensus.cc:738] T 56bbc306275e4a409e028a7ca1dd1602 P 787ddcc824c2490a808fc5a53452dc5e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Running, Role: LEADER
I20250109 08:58:03.855657 22172 raft_consensus.cc:2949] T 1b582ab8d8f044d9b9038a28e958feeb P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:03.858071 21512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56bbc306275e4a409e028a7ca1dd1602" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8bf6e26e98b14566899578a990b27d47"
I20250109 08:58:03.858742 21512 raft_consensus.cc:3054] T 56bbc306275e4a409e028a7ca1dd1602 P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Advancing to term 3
I20250109 08:58:03.852324 21660 consensus_queue.cc:260] T 56bbc306275e4a409e028a7ca1dd1602 P 787ddcc824c2490a808fc5a53452dc5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:58:03.859956 21660 raft_consensus.cc:3054] T 56bbc306275e4a409e028a7ca1dd1602 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Advancing to term 3
I20250109 08:58:03.841023 21588 leader_election.cc:290] T 56bbc306275e4a409e028a7ca1dd1602 P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 3 election: Requested vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:58:03.870779 21509 raft_consensus.cc:2949] T 1b582ab8d8f044d9b9038a28e958feeb P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:03.875234 21512 raft_consensus.cc:2463] T 56bbc306275e4a409e028a7ca1dd1602 P 8bf6e26e98b14566899578a990b27d47 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2b1a692435241ee988e0bd1661cd4c3 in term 3.
I20250109 08:58:03.876541 21552 leader_election.cc:304] T 56bbc306275e4a409e028a7ca1dd1602 P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3; no voters:
I20250109 08:58:03.880153 22773 raft_consensus.cc:2798] T 56bbc306275e4a409e028a7ca1dd1602 P a2b1a692435241ee988e0bd1661cd4c3 [term 3 FOLLOWER]: Leader election won for term 3
I20250109 08:58:03.880283 21660 raft_consensus.cc:2463] T 56bbc306275e4a409e028a7ca1dd1602 P 787ddcc824c2490a808fc5a53452dc5e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2b1a692435241ee988e0bd1661cd4c3 in term 3.
I20250109 08:58:03.885082 22177 raft_consensus.cc:1270] T f5f85ad7c7c94acfaf9d550d55fce48c P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Refusing update from remote peer a2b1a692435241ee988e0bd1661cd4c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:58:03.886224 21656 raft_consensus.cc:1270] T f5f85ad7c7c94acfaf9d550d55fce48c P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Refusing update from remote peer a2b1a692435241ee988e0bd1661cd4c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:58:03.890081 22816 consensus_queue.cc:1035] T f5f85ad7c7c94acfaf9d550d55fce48c P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:58:03.892540 21396 catalog_manager.cc:5039] ChangeConfig:REMOVE_PEER RPC for tablet 1b582ab8d8f044d9b9038a28e958feeb with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250109 08:58:03.899356 22816 consensus_queue.cc:1035] T f5f85ad7c7c94acfaf9d550d55fce48c P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:58:03.906878 21411 catalog_manager.cc:5526] T 1b582ab8d8f044d9b9038a28e958feeb P 88935ab835544e9889d05a6587d8cf21 reported cstate change: config changed from index 3 to 4, VOTER a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194) evicted. New cstate: current_term: 1 leader_uuid: "787ddcc824c2490a808fc5a53452dc5e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:03.914965 22773 raft_consensus.cc:695] T 56bbc306275e4a409e028a7ca1dd1602 P a2b1a692435241ee988e0bd1661cd4c3 [term 3 LEADER]: Becoming Leader. State: Replica: a2b1a692435241ee988e0bd1661cd4c3, State: Running, Role: LEADER
I20250109 08:58:03.915959 22773 consensus_queue.cc:237] T 56bbc306275e4a409e028a7ca1dd1602 P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:58:03.920074 22660 raft_consensus.cc:1059] T 71ac0243d72845659291184835652dde P 8bf6e26e98b14566899578a990b27d47: attempting to promote NON_VOTER 88935ab835544e9889d05a6587d8cf21 to VOTER
I20250109 08:58:03.922607 22660 consensus_queue.cc:237] T 71ac0243d72845659291184835652dde P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:03.927423 21570 tablet_service.cc:1514] Processing DeleteTablet for tablet 1b582ab8d8f044d9b9038a28e958feeb with delete_type TABLET_DATA_TOMBSTONED (TS a2b1a692435241ee988e0bd1661cd4c3 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:34732
I20250109 08:58:03.935432 21412 catalog_manager.cc:5526] T 56bbc306275e4a409e028a7ca1dd1602 P a2b1a692435241ee988e0bd1661cd4c3 reported cstate change: term changed from 2 to 3, leader changed from 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195) to a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194). New cstate: current_term: 3 leader_uuid: "a2b1a692435241ee988e0bd1661cd4c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:03.942283 21509 raft_consensus.cc:1270] T f5f85ad7c7c94acfaf9d550d55fce48c P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Refusing update from remote peer a2b1a692435241ee988e0bd1661cd4c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:58:03.955874 22857 raft_consensus.cc:988] T 97304b06c7fa40ef8345a72c796c47f8 P a2b1a692435241ee988e0bd1661cd4c3: : Instructing follower 8bf6e26e98b14566899578a990b27d47 to start an election
I20250109 08:58:03.956357 22816 raft_consensus.cc:1076] T 97304b06c7fa40ef8345a72c796c47f8 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 LEADER]: Signalling peer 8bf6e26e98b14566899578a990b27d47 to start an election
I20250109 08:58:03.961000 22743 consensus_queue.cc:1035] T f5f85ad7c7c94acfaf9d550d55fce48c P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:58:03.974435 22176 raft_consensus.cc:1270] T 71ac0243d72845659291184835652dde P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Refusing update from remote peer 8bf6e26e98b14566899578a990b27d47: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:58:03.974519 21657 raft_consensus.cc:1270] T 71ac0243d72845659291184835652dde P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Refusing update from remote peer 8bf6e26e98b14566899578a990b27d47: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:58:03.976481 22861 tablet_replica.cc:331] T 1b582ab8d8f044d9b9038a28e958feeb P a2b1a692435241ee988e0bd1661cd4c3: stopping tablet replica
I20250109 08:58:03.977602 22660 consensus_queue.cc:1035] T 71ac0243d72845659291184835652dde P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20250109 08:58:03.985059 22861 raft_consensus.cc:2238] T 1b582ab8d8f044d9b9038a28e958feeb P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:03.985853 22861 raft_consensus.cc:2267] T 1b582ab8d8f044d9b9038a28e958feeb P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:03.985693 22743 raft_consensus.cc:2949] T f5f85ad7c7c94acfaf9d550d55fce48c P a2b1a692435241ee988e0bd1661cd4c3 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:03.988029 21514 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "97304b06c7fa40ef8345a72c796c47f8"
dest_uuid: "8bf6e26e98b14566899578a990b27d47"
from {username='slave'} at 127.0.0.1:39250
I20250109 08:58:03.988620 21514 raft_consensus.cc:491] T 97304b06c7fa40ef8345a72c796c47f8 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:58:03.988960 21514 raft_consensus.cc:3054] T 97304b06c7fa40ef8345a72c796c47f8 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:03.989894 22861 ts_tablet_manager.cc:1905] T 1b582ab8d8f044d9b9038a28e958feeb P a2b1a692435241ee988e0bd1661cd4c3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250109 08:58:03.994966 21514 raft_consensus.cc:513] T 97304b06c7fa40ef8345a72c796c47f8 P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:03.996907 21514 leader_election.cc:290] T 97304b06c7fa40ef8345a72c796c47f8 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 2 election: Requested vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 88935ab835544e9889d05a6587d8cf21 (127.19.226.196:44363)
I20250109 08:58:04.002020 21657 raft_consensus.cc:2949] T f5f85ad7c7c94acfaf9d550d55fce48c P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:04.003015 22172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97304b06c7fa40ef8345a72c796c47f8" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "88935ab835544e9889d05a6587d8cf21"
I20250109 08:58:04.005447 21589 raft_consensus.cc:1270] T 71ac0243d72845659291184835652dde P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Refusing update from remote peer 8bf6e26e98b14566899578a990b27d47: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:58:04.005476 22172 raft_consensus.cc:3054] T 97304b06c7fa40ef8345a72c796c47f8 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:04.006528 22861 ts_tablet_manager.cc:1918] T 1b582ab8d8f044d9b9038a28e958feeb P a2b1a692435241ee988e0bd1661cd4c3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20250109 08:58:04.007213 22861 log.cc:1198] T 1b582ab8d8f044d9b9038a28e958feeb P a2b1a692435241ee988e0bd1661cd4c3: Deleting WAL directory at /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736412987538469-20363-0/minicluster-data/ts-1-root/wals/1b582ab8d8f044d9b9038a28e958feeb
I20250109 08:58:04.008510 22660 consensus_queue.cc:1035] T 71ac0243d72845659291184835652dde P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:58:04.003392 21583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97304b06c7fa40ef8345a72c796c47f8" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3"
I20250109 08:58:04.010846 21583 raft_consensus.cc:3049] T 97304b06c7fa40ef8345a72c796c47f8 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 LEADER]: Stepping down as leader of term 1
I20250109 08:58:04.011132 21583 raft_consensus.cc:738] T 97304b06c7fa40ef8345a72c796c47f8 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a2b1a692435241ee988e0bd1661cd4c3, State: Running, Role: LEADER
I20250109 08:58:04.011816 21583 consensus_queue.cc:260] T 97304b06c7fa40ef8345a72c796c47f8 P a2b1a692435241ee988e0bd1661cd4c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:04.013015 21583 raft_consensus.cc:3054] T 97304b06c7fa40ef8345a72c796c47f8 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:04.013767 21397 catalog_manager.cc:4872] TS a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): tablet 1b582ab8d8f044d9b9038a28e958feeb (table test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]) successfully deleted
I20250109 08:58:04.018846 22172 raft_consensus.cc:2463] T 97304b06c7fa40ef8345a72c796c47f8 P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8bf6e26e98b14566899578a990b27d47 in term 2.
I20250109 08:58:04.014569 22177 raft_consensus.cc:2949] T f5f85ad7c7c94acfaf9d550d55fce48c P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:04.027575 21583 raft_consensus.cc:2463] T 97304b06c7fa40ef8345a72c796c47f8 P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8bf6e26e98b14566899578a990b27d47 in term 2.
I20250109 08:58:04.030726 21478 leader_election.cc:304] T 97304b06c7fa40ef8345a72c796c47f8 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88935ab835544e9889d05a6587d8cf21, 8bf6e26e98b14566899578a990b27d47; no voters:
I20250109 08:58:04.031479 22738 raft_consensus.cc:2798] T 97304b06c7fa40ef8345a72c796c47f8 P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:58:04.032009 22738 raft_consensus.cc:695] T 97304b06c7fa40ef8345a72c796c47f8 P 8bf6e26e98b14566899578a990b27d47 [term 2 LEADER]: Becoming Leader. State: Replica: 8bf6e26e98b14566899578a990b27d47, State: Running, Role: LEADER
I20250109 08:58:04.034183 22738 consensus_queue.cc:237] T 97304b06c7fa40ef8345a72c796c47f8 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:04.034469 22660 consensus_queue.cc:1035] T 71ac0243d72845659291184835652dde P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:58:04.039077 21512 raft_consensus.cc:2949] T f5f85ad7c7c94acfaf9d550d55fce48c P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:04.049489 22814 raft_consensus.cc:2949] T 71ac0243d72845659291184835652dde P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:04.058437 21412 catalog_manager.cc:5526] T f5f85ad7c7c94acfaf9d550d55fce48c P a2b1a692435241ee988e0bd1661cd4c3 reported cstate change: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a2b1a692435241ee988e0bd1661cd4c3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250109 08:58:04.073873 22665 raft_consensus.cc:1059] T d8493108cba04514bd2236192cef1079 P 787ddcc824c2490a808fc5a53452dc5e: attempting to promote NON_VOTER 88935ab835544e9889d05a6587d8cf21 to VOTER
I20250109 08:58:04.076051 21657 raft_consensus.cc:2949] T 71ac0243d72845659291184835652dde P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:04.082041 22665 consensus_queue.cc:237] T d8493108cba04514bd2236192cef1079 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:04.091483 21588 raft_consensus.cc:2949] T 71ac0243d72845659291184835652dde P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:04.092547 21411 catalog_manager.cc:5526] T 97304b06c7fa40ef8345a72c796c47f8 P 8bf6e26e98b14566899578a990b27d47 reported cstate change: term changed from 1 to 2, leader changed from a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194) to 8bf6e26e98b14566899578a990b27d47 (127.19.226.193). New cstate: current_term: 2 leader_uuid: "8bf6e26e98b14566899578a990b27d47" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:04.097909 22177 raft_consensus.cc:2949] T 71ac0243d72845659291184835652dde P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:04.113842 21583 raft_consensus.cc:1270] T d8493108cba04514bd2236192cef1079 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:58:04.114709 22177 raft_consensus.cc:1270] T d8493108cba04514bd2236192cef1079 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:58:04.116891 21514 raft_consensus.cc:1270] T d8493108cba04514bd2236192cef1079 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:58:04.118901 22750 consensus_queue.cc:1035] T d8493108cba04514bd2236192cef1079 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:58:04.120671 22750 consensus_queue.cc:1035] T d8493108cba04514bd2236192cef1079 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:58:04.131409 22665 consensus_queue.cc:1035] T d8493108cba04514bd2236192cef1079 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:58:04.134716 21409 catalog_manager.cc:5526] T 71ac0243d72845659291184835652dde P 787ddcc824c2490a808fc5a53452dc5e reported cstate change: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8bf6e26e98b14566899578a990b27d47" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:04.140352 22665 raft_consensus.cc:2949] T d8493108cba04514bd2236192cef1079 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:04.146257 21588 consensus_queue.cc:237] T f5f85ad7c7c94acfaf9d550d55fce48c P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:04.145287 22177 raft_consensus.cc:2949] T d8493108cba04514bd2236192cef1079 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:04.152694 22172 raft_consensus.cc:1270] T f5f85ad7c7c94acfaf9d550d55fce48c P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Refusing update from remote peer a2b1a692435241ee988e0bd1661cd4c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250109 08:58:04.155368 22743 consensus_queue.cc:1035] T f5f85ad7c7c94acfaf9d550d55fce48c P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250109 08:58:04.162644 21512 consensus_queue.cc:237] T 71ac0243d72845659291184835652dde P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:04.168583 22856 raft_consensus.cc:2949] T f5f85ad7c7c94acfaf9d550d55fce48c P a2b1a692435241ee988e0bd1661cd4c3 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8bf6e26e98b14566899578a990b27d47 (127.19.226.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:04.173056 21514 raft_consensus.cc:2949] T d8493108cba04514bd2236192cef1079 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:04.175534 22172 raft_consensus.cc:2949] T f5f85ad7c7c94acfaf9d550d55fce48c P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8bf6e26e98b14566899578a990b27d47 (127.19.226.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:04.184113 21583 raft_consensus.cc:2949] T d8493108cba04514bd2236192cef1079 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:04.194744 21409 catalog_manager.cc:5526] T f5f85ad7c7c94acfaf9d550d55fce48c P 88935ab835544e9889d05a6587d8cf21 reported cstate change: config changed from index 3 to 4, VOTER 8bf6e26e98b14566899578a990b27d47 (127.19.226.193) evicted. New cstate: current_term: 1 leader_uuid: "a2b1a692435241ee988e0bd1661cd4c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:04.207477 21503 tablet_service.cc:1514] Processing DeleteTablet for tablet f5f85ad7c7c94acfaf9d550d55fce48c with delete_type TABLET_DATA_TOMBSTONED (TS 8bf6e26e98b14566899578a990b27d47 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:39242
I20250109 08:58:04.211047 22870 raft_consensus.cc:988] T 37280454805d47f98a415a8f0c16d869 P 8bf6e26e98b14566899578a990b27d47: : Instructing follower 787ddcc824c2490a808fc5a53452dc5e to start an election
I20250109 08:58:04.211733 22863 raft_consensus.cc:1076] T 37280454805d47f98a415a8f0c16d869 P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Signalling peer 787ddcc824c2490a808fc5a53452dc5e to start an election
I20250109 08:58:04.214944 21657 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "37280454805d47f98a415a8f0c16d869"
dest_uuid: "787ddcc824c2490a808fc5a53452dc5e"
from {username='slave'} at 127.0.0.1:58440
I20250109 08:58:04.215771 21657 raft_consensus.cc:491] T 37280454805d47f98a415a8f0c16d869 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:58:04.217406 21657 raft_consensus.cc:3054] T 37280454805d47f98a415a8f0c16d869 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:04.217492 21589 raft_consensus.cc:1270] T 71ac0243d72845659291184835652dde P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Refusing update from remote peer 8bf6e26e98b14566899578a990b27d47: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250109 08:58:04.220512 22814 consensus_queue.cc:1035] T 71ac0243d72845659291184835652dde P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250109 08:58:04.190919 21411 catalog_manager.cc:5526] T d8493108cba04514bd2236192cef1079 P 787ddcc824c2490a808fc5a53452dc5e reported cstate change: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "787ddcc824c2490a808fc5a53452dc5e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250109 08:58:04.224511 22876 tablet_replica.cc:331] T f5f85ad7c7c94acfaf9d550d55fce48c P 8bf6e26e98b14566899578a990b27d47: stopping tablet replica
I20250109 08:58:04.226083 21397 catalog_manager.cc:5039] ChangeConfig:REMOVE_PEER RPC for tablet f5f85ad7c7c94acfaf9d550d55fce48c with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250109 08:58:04.218381 22177 raft_consensus.cc:1270] T 71ac0243d72845659291184835652dde P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Refusing update from remote peer 8bf6e26e98b14566899578a990b27d47: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250109 08:58:04.227510 22738 raft_consensus.cc:2949] T 71ac0243d72845659291184835652dde P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:04.231307 21660 raft_consensus.cc:1270] T f5f85ad7c7c94acfaf9d550d55fce48c P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Refusing update from remote peer a2b1a692435241ee988e0bd1661cd4c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250109 08:58:04.232726 22743 consensus_queue.cc:1035] T f5f85ad7c7c94acfaf9d550d55fce48c P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250109 08:58:04.233044 21657 raft_consensus.cc:513] T 37280454805d47f98a415a8f0c16d869 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:04.235565 21514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37280454805d47f98a415a8f0c16d869" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "8bf6e26e98b14566899578a990b27d47"
I20250109 08:58:04.236802 21514 raft_consensus.cc:3049] T 37280454805d47f98a415a8f0c16d869 P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Stepping down as leader of term 1
I20250109 08:58:04.237144 21514 raft_consensus.cc:738] T 37280454805d47f98a415a8f0c16d869 P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8bf6e26e98b14566899578a990b27d47, State: Running, Role: LEADER
I20250109 08:58:04.237879 21514 consensus_queue.cc:260] T 37280454805d47f98a415a8f0c16d869 P 8bf6e26e98b14566899578a990b27d47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:04.236331 21657 leader_election.cc:290] T 37280454805d47f98a415a8f0c16d869 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 2 election: Requested vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), 88935ab835544e9889d05a6587d8cf21 (127.19.226.196:44363)
I20250109 08:58:04.239771 21514 raft_consensus.cc:3054] T 37280454805d47f98a415a8f0c16d869 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:04.241835 21660 raft_consensus.cc:2949] T f5f85ad7c7c94acfaf9d550d55fce48c P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8bf6e26e98b14566899578a990b27d47 (127.19.226.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:04.245398 22870 consensus_queue.cc:1035] T 71ac0243d72845659291184835652dde P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250109 08:58:04.246049 21514 raft_consensus.cc:2463] T 37280454805d47f98a415a8f0c16d869 P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 2.
I20250109 08:58:04.247059 21626 leader_election.cc:304] T 37280454805d47f98a415a8f0c16d869 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, 8bf6e26e98b14566899578a990b27d47; no voters:
I20250109 08:58:04.247875 22665 raft_consensus.cc:2798] T 37280454805d47f98a415a8f0c16d869 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:58:04.248495 22665 raft_consensus.cc:695] T 37280454805d47f98a415a8f0c16d869 P 787ddcc824c2490a808fc5a53452dc5e [term 2 LEADER]: Becoming Leader. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Running, Role: LEADER
I20250109 08:58:04.250417 22172 raft_consensus.cc:2949] T 71ac0243d72845659291184835652dde P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:04.252844 22665 consensus_queue.cc:237] T 37280454805d47f98a415a8f0c16d869 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:04.257941 22177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37280454805d47f98a415a8f0c16d869" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "88935ab835544e9889d05a6587d8cf21"
I20250109 08:58:04.259799 22177 raft_consensus.cc:3054] T 37280454805d47f98a415a8f0c16d869 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:04.262298 21396 catalog_manager.cc:5039] ChangeConfig:REMOVE_PEER RPC for tablet 71ac0243d72845659291184835652dde with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250109 08:58:04.266113 21583 raft_consensus.cc:2949] T 71ac0243d72845659291184835652dde P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:04.270936 22876 raft_consensus.cc:2238] T f5f85ad7c7c94acfaf9d550d55fce48c P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:04.271724 22876 raft_consensus.cc:2267] T f5f85ad7c7c94acfaf9d550d55fce48c P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:04.272228 21657 consensus_queue.cc:237] T d8493108cba04514bd2236192cef1079 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:04.275184 22876 ts_tablet_manager.cc:1905] T f5f85ad7c7c94acfaf9d550d55fce48c P 8bf6e26e98b14566899578a990b27d47: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250109 08:58:04.280606 22172 raft_consensus.cc:1270] T d8493108cba04514bd2236192cef1079 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250109 08:58:04.280841 21512 raft_consensus.cc:1270] T d8493108cba04514bd2236192cef1079 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250109 08:58:04.282016 22872 consensus_queue.cc:1035] T d8493108cba04514bd2236192cef1079 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250109 08:58:04.282867 22653 consensus_queue.cc:1035] T d8493108cba04514bd2236192cef1079 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250109 08:58:04.288887 22653 raft_consensus.cc:2949] T d8493108cba04514bd2236192cef1079 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:04.279028 21412 catalog_manager.cc:5526] T 71ac0243d72845659291184835652dde P 8bf6e26e98b14566899578a990b27d47 reported cstate change: config changed from index 3 to 4, VOTER 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195) evicted. New cstate: current_term: 1 leader_uuid: "8bf6e26e98b14566899578a990b27d47" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:04.291927 22176 raft_consensus.cc:2949] T d8493108cba04514bd2236192cef1079 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:04.300288 21512 raft_consensus.cc:2949] T d8493108cba04514bd2236192cef1079 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:04.303956 22177 raft_consensus.cc:2463] T 37280454805d47f98a415a8f0c16d869 P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 2.
I20250109 08:58:04.315156 22876 ts_tablet_manager.cc:1918] T f5f85ad7c7c94acfaf9d550d55fce48c P 8bf6e26e98b14566899578a990b27d47: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20250109 08:58:04.315698 22876 log.cc:1198] T f5f85ad7c7c94acfaf9d550d55fce48c P 8bf6e26e98b14566899578a990b27d47: Deleting WAL directory at /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736412987538469-20363-0/minicluster-data/ts-0-root/wals/f5f85ad7c7c94acfaf9d550d55fce48c
I20250109 08:58:04.316466 21396 catalog_manager.cc:5039] ChangeConfig:REMOVE_PEER RPC for tablet d8493108cba04514bd2236192cef1079 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250109 08:58:04.317596 21396 catalog_manager.cc:4872] TS 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): tablet f5f85ad7c7c94acfaf9d550d55fce48c (table test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]) successfully deleted
I20250109 08:58:04.321722 21653 tablet_service.cc:1514] Processing DeleteTablet for tablet 71ac0243d72845659291184835652dde with delete_type TABLET_DATA_TOMBSTONED (TS 787ddcc824c2490a808fc5a53452dc5e not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:58436
I20250109 08:58:04.330073 21410 catalog_manager.cc:5526] T d8493108cba04514bd2236192cef1079 P 88935ab835544e9889d05a6587d8cf21 reported cstate change: config changed from index 3 to 4, VOTER a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194) evicted. New cstate: current_term: 1 leader_uuid: "787ddcc824c2490a808fc5a53452dc5e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:04.339819 21570 tablet_service.cc:1514] Processing DeleteTablet for tablet d8493108cba04514bd2236192cef1079 with delete_type TABLET_DATA_TOMBSTONED (TS a2b1a692435241ee988e0bd1661cd4c3 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:34732
I20250109 08:58:04.340893 22861 tablet_replica.cc:331] T d8493108cba04514bd2236192cef1079 P a2b1a692435241ee988e0bd1661cd4c3: stopping tablet replica
I20250109 08:58:04.340633 21412 catalog_manager.cc:5526] T 37280454805d47f98a415a8f0c16d869 P 787ddcc824c2490a808fc5a53452dc5e reported cstate change: term changed from 1 to 2, leader changed from 8bf6e26e98b14566899578a990b27d47 (127.19.226.193) to 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195). New cstate: current_term: 2 leader_uuid: "787ddcc824c2490a808fc5a53452dc5e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:04.342039 22861 raft_consensus.cc:2238] T d8493108cba04514bd2236192cef1079 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:04.342821 22861 raft_consensus.cc:2267] T d8493108cba04514bd2236192cef1079 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:04.345142 22878 tablet_replica.cc:331] T 71ac0243d72845659291184835652dde P 787ddcc824c2490a808fc5a53452dc5e: stopping tablet replica
I20250109 08:58:04.346277 22861 ts_tablet_manager.cc:1905] T d8493108cba04514bd2236192cef1079 P a2b1a692435241ee988e0bd1661cd4c3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250109 08:58:04.348687 22878 raft_consensus.cc:2238] T 71ac0243d72845659291184835652dde P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:04.349457 22878 raft_consensus.cc:2267] T 71ac0243d72845659291184835652dde P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:04.356222 22878 ts_tablet_manager.cc:1905] T 71ac0243d72845659291184835652dde P 787ddcc824c2490a808fc5a53452dc5e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250109 08:58:04.364085 22861 ts_tablet_manager.cc:1918] T d8493108cba04514bd2236192cef1079 P a2b1a692435241ee988e0bd1661cd4c3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20250109 08:58:04.364593 22861 log.cc:1198] T d8493108cba04514bd2236192cef1079 P a2b1a692435241ee988e0bd1661cd4c3: Deleting WAL directory at /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736412987538469-20363-0/minicluster-data/ts-1-root/wals/d8493108cba04514bd2236192cef1079
I20250109 08:58:04.365741 21397 catalog_manager.cc:4872] TS a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): tablet d8493108cba04514bd2236192cef1079 (table test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]) successfully deleted
I20250109 08:58:04.374297 22878 ts_tablet_manager.cc:1918] T 71ac0243d72845659291184835652dde P 787ddcc824c2490a808fc5a53452dc5e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20250109 08:58:04.374660 22878 log.cc:1198] T 71ac0243d72845659291184835652dde P 787ddcc824c2490a808fc5a53452dc5e: Deleting WAL directory at /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736412987538469-20363-0/minicluster-data/ts-2-root/wals/71ac0243d72845659291184835652dde
I20250109 08:58:04.375903 21396 catalog_manager.cc:4872] TS 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731): tablet 71ac0243d72845659291184835652dde (table test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]) successfully deleted
I20250109 08:58:04.480247 21512 raft_consensus.cc:1270] T 56bbc306275e4a409e028a7ca1dd1602 P 8bf6e26e98b14566899578a990b27d47 [term 3 FOLLOWER]: Refusing update from remote peer a2b1a692435241ee988e0bd1661cd4c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20250109 08:58:04.481406 22856 consensus_queue.cc:1035] T 56bbc306275e4a409e028a7ca1dd1602 P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:58:04.487908 21660 raft_consensus.cc:1270] T 56bbc306275e4a409e028a7ca1dd1602 P 787ddcc824c2490a808fc5a53452dc5e [term 3 FOLLOWER]: Refusing update from remote peer a2b1a692435241ee988e0bd1661cd4c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20250109 08:58:04.491878 22773 consensus_queue.cc:1035] T 56bbc306275e4a409e028a7ca1dd1602 P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:58:04.614535 22176 raft_consensus.cc:1270] T 97304b06c7fa40ef8345a72c796c47f8 P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Refusing update from remote peer 8bf6e26e98b14566899578a990b27d47: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250109 08:58:04.615998 21583 raft_consensus.cc:1270] T 97304b06c7fa40ef8345a72c796c47f8 P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Refusing update from remote peer 8bf6e26e98b14566899578a990b27d47: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250109 08:58:04.616461 22863 consensus_queue.cc:1035] T 97304b06c7fa40ef8345a72c796c47f8 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250109 08:58:04.618016 22738 consensus_queue.cc:1035] T 97304b06c7fa40ef8345a72c796c47f8 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250109 08:58:04.708441 22176 raft_consensus.cc:1270] T 37280454805d47f98a415a8f0c16d869 P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250109 08:58:04.708441 21512 raft_consensus.cc:1270] T 37280454805d47f98a415a8f0c16d869 P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250109 08:58:04.709817 22653 consensus_queue.cc:1035] T 37280454805d47f98a415a8f0c16d869 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250109 08:58:04.710458 22665 consensus_queue.cc:1035] T 37280454805d47f98a415a8f0c16d869 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250109 08:58:04.838869 20363 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250109 08:58:04.875629 21512 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "79c1ed8ab551478291f1f24cc940cb27"
dest_uuid: "8bf6e26e98b14566899578a990b27d47"
mode: GRACEFUL
new_leader_uuid: "787ddcc824c2490a808fc5a53452dc5e"
from {username='slave'} at 127.0.0.1:39274
I20250109 08:58:04.876214 21512 raft_consensus.cc:604] T 79c1ed8ab551478291f1f24cc940cb27 P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Received request to transfer leadership to TS 787ddcc824c2490a808fc5a53452dc5e
I20250109 08:58:04.878369 21660 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "90a4a2381cbe40f0b204be325ee9fea1"
dest_uuid: "787ddcc824c2490a808fc5a53452dc5e"
mode: GRACEFUL
new_leader_uuid: "88935ab835544e9889d05a6587d8cf21"
from {username='slave'} at 127.0.0.1:55054
I20250109 08:58:04.878839 21660 raft_consensus.cc:604] T 90a4a2381cbe40f0b204be325ee9fea1 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Received request to transfer leadership to TS 88935ab835544e9889d05a6587d8cf21
I20250109 08:58:04.881136 21583 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "c99aacce09d44f889be5480854a22760"
dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3"
mode: GRACEFUL
new_leader_uuid: "8bf6e26e98b14566899578a990b27d47"
from {username='slave'} at 127.0.0.1:46832
I20250109 08:58:04.881661 21583 raft_consensus.cc:604] T c99aacce09d44f889be5480854a22760 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 LEADER]: Received request to transfer leadership to TS 8bf6e26e98b14566899578a990b27d47
I20250109 08:58:04.882707 20363 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 3
I20250109 08:58:04.884009 20363 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250109 08:58:04.884631 20363 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250109 08:58:05.018628 22863 raft_consensus.cc:988] T 79c1ed8ab551478291f1f24cc940cb27 P 8bf6e26e98b14566899578a990b27d47: : Instructing follower 787ddcc824c2490a808fc5a53452dc5e to start an election
I20250109 08:58:05.019178 22870 raft_consensus.cc:1076] T 79c1ed8ab551478291f1f24cc940cb27 P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Signalling peer 787ddcc824c2490a808fc5a53452dc5e to start an election
I20250109 08:58:05.020396 21660 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "79c1ed8ab551478291f1f24cc940cb27"
dest_uuid: "787ddcc824c2490a808fc5a53452dc5e"
from {username='slave'} at 127.0.0.1:58440
I20250109 08:58:05.020881 21660 raft_consensus.cc:491] T 79c1ed8ab551478291f1f24cc940cb27 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:58:05.021128 21660 raft_consensus.cc:3054] T 79c1ed8ab551478291f1f24cc940cb27 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:05.026002 21660 raft_consensus.cc:513] T 79c1ed8ab551478291f1f24cc940cb27 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:58:05.027698 21660 leader_election.cc:290] T 79c1ed8ab551478291f1f24cc940cb27 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 2 election: Requested vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
I20250109 08:58:05.029137 21512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79c1ed8ab551478291f1f24cc940cb27" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8bf6e26e98b14566899578a990b27d47"
I20250109 08:58:05.029771 21512 raft_consensus.cc:3049] T 79c1ed8ab551478291f1f24cc940cb27 P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Stepping down as leader of term 1
I20250109 08:58:05.030090 21512 raft_consensus.cc:738] T 79c1ed8ab551478291f1f24cc940cb27 P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8bf6e26e98b14566899578a990b27d47, State: Running, Role: LEADER
I20250109 08:58:05.030422 21583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79c1ed8ab551478291f1f24cc940cb27" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3"
I20250109 08:58:05.031019 21583 raft_consensus.cc:3054] T 79c1ed8ab551478291f1f24cc940cb27 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:05.030927 21512 consensus_queue.cc:260] T 79c1ed8ab551478291f1f24cc940cb27 P 8bf6e26e98b14566899578a990b27d47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:58:05.032200 21512 raft_consensus.cc:3054] T 79c1ed8ab551478291f1f24cc940cb27 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:05.035926 21583 raft_consensus.cc:2463] T 79c1ed8ab551478291f1f24cc940cb27 P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 2.
I20250109 08:58:05.036402 21512 raft_consensus.cc:2463] T 79c1ed8ab551478291f1f24cc940cb27 P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 2.
I20250109 08:58:05.036831 21627 leader_election.cc:304] T 79c1ed8ab551478291f1f24cc940cb27 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, a2b1a692435241ee988e0bd1661cd4c3; no voters:
I20250109 08:58:05.037448 22750 raft_consensus.cc:2798] T 79c1ed8ab551478291f1f24cc940cb27 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:58:05.037840 22750 raft_consensus.cc:695] T 79c1ed8ab551478291f1f24cc940cb27 P 787ddcc824c2490a808fc5a53452dc5e [term 2 LEADER]: Becoming Leader. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Running, Role: LEADER
I20250109 08:58:05.038534 22750 consensus_queue.cc:237] T 79c1ed8ab551478291f1f24cc940cb27 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:58:05.050184 21410 catalog_manager.cc:5526] T 79c1ed8ab551478291f1f24cc940cb27 P 787ddcc824c2490a808fc5a53452dc5e reported cstate change: term changed from 1 to 2, leader changed from 8bf6e26e98b14566899578a990b27d47 (127.19.226.193) to 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195). New cstate: current_term: 2 leader_uuid: "787ddcc824c2490a808fc5a53452dc5e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: HEALTHY } } }
I20250109 08:58:05.270768 22773 raft_consensus.cc:988] T c99aacce09d44f889be5480854a22760 P a2b1a692435241ee988e0bd1661cd4c3: : Instructing follower 8bf6e26e98b14566899578a990b27d47 to start an election
I20250109 08:58:05.271243 22856 raft_consensus.cc:1076] T c99aacce09d44f889be5480854a22760 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 LEADER]: Signalling peer 8bf6e26e98b14566899578a990b27d47 to start an election
I20250109 08:58:05.272485 21512 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "c99aacce09d44f889be5480854a22760"
dest_uuid: "8bf6e26e98b14566899578a990b27d47"
from {username='slave'} at 127.0.0.1:39250
I20250109 08:58:05.272920 21512 raft_consensus.cc:491] T c99aacce09d44f889be5480854a22760 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:58:05.273190 21512 raft_consensus.cc:3054] T c99aacce09d44f889be5480854a22760 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:05.277490 21512 raft_consensus.cc:513] T c99aacce09d44f889be5480854a22760 P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:58:05.278975 21512 leader_election.cc:290] T c99aacce09d44f889be5480854a22760 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 2 election: Requested vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:58:05.279538 21584 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c99aacce09d44f889be5480854a22760" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3"
I20250109 08:58:05.280129 21584 raft_consensus.cc:3049] T c99aacce09d44f889be5480854a22760 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 LEADER]: Stepping down as leader of term 1
I20250109 08:58:05.280453 21584 raft_consensus.cc:738] T c99aacce09d44f889be5480854a22760 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a2b1a692435241ee988e0bd1661cd4c3, State: Running, Role: LEADER
I20250109 08:58:05.280668 21657 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c99aacce09d44f889be5480854a22760" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "787ddcc824c2490a808fc5a53452dc5e"
I20250109 08:58:05.281191 21657 raft_consensus.cc:3054] T c99aacce09d44f889be5480854a22760 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:05.281136 21584 consensus_queue.cc:260] T c99aacce09d44f889be5480854a22760 P a2b1a692435241ee988e0bd1661cd4c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:58:05.282459 21584 raft_consensus.cc:3054] T c99aacce09d44f889be5480854a22760 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:05.287607 21657 raft_consensus.cc:2463] T c99aacce09d44f889be5480854a22760 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8bf6e26e98b14566899578a990b27d47 in term 2.
I20250109 08:58:05.288645 21477 leader_election.cc:304] T c99aacce09d44f889be5480854a22760 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, 8bf6e26e98b14566899578a990b27d47; no voters:
I20250109 08:58:05.289180 21584 raft_consensus.cc:2463] T c99aacce09d44f889be5480854a22760 P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8bf6e26e98b14566899578a990b27d47 in term 2.
I20250109 08:58:05.289558 22863 raft_consensus.cc:2798] T c99aacce09d44f889be5480854a22760 P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:58:05.290028 22863 raft_consensus.cc:695] T c99aacce09d44f889be5480854a22760 P 8bf6e26e98b14566899578a990b27d47 [term 2 LEADER]: Becoming Leader. State: Replica: 8bf6e26e98b14566899578a990b27d47, State: Running, Role: LEADER
I20250109 08:58:05.290742 22863 consensus_queue.cc:237] T c99aacce09d44f889be5480854a22760 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:58:05.295336 22889 raft_consensus.cc:988] T 90a4a2381cbe40f0b204be325ee9fea1 P 787ddcc824c2490a808fc5a53452dc5e: : Instructing follower 88935ab835544e9889d05a6587d8cf21 to start an election
I20250109 08:58:05.295832 22750 raft_consensus.cc:1076] T 90a4a2381cbe40f0b204be325ee9fea1 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Signalling peer 88935ab835544e9889d05a6587d8cf21 to start an election
I20250109 08:58:05.297024 22177 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "90a4a2381cbe40f0b204be325ee9fea1"
dest_uuid: "88935ab835544e9889d05a6587d8cf21"
from {username='slave'} at 127.0.0.1:60458
I20250109 08:58:05.297412 22177 raft_consensus.cc:491] T 90a4a2381cbe40f0b204be325ee9fea1 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:58:05.297658 22177 raft_consensus.cc:3054] T 90a4a2381cbe40f0b204be325ee9fea1 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:05.303344 22177 raft_consensus.cc:513] T 90a4a2381cbe40f0b204be325ee9fea1 P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:05.305429 22177 leader_election.cc:290] T 90a4a2381cbe40f0b204be325ee9fea1 P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 election: Requested vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:58:05.306736 21657 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90a4a2381cbe40f0b204be325ee9fea1" candidate_uuid: "88935ab835544e9889d05a6587d8cf21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "787ddcc824c2490a808fc5a53452dc5e"
I20250109 08:58:05.307230 21657 raft_consensus.cc:3049] T 90a4a2381cbe40f0b204be325ee9fea1 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Stepping down as leader of term 1
I20250109 08:58:05.307545 21657 raft_consensus.cc:738] T 90a4a2381cbe40f0b204be325ee9fea1 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Running, Role: LEADER
I20250109 08:58:05.308187 21657 consensus_queue.cc:260] T 90a4a2381cbe40f0b204be325ee9fea1 P 787ddcc824c2490a808fc5a53452dc5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:05.309504 21657 raft_consensus.cc:3054] T 90a4a2381cbe40f0b204be325ee9fea1 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:05.313078 21584 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90a4a2381cbe40f0b204be325ee9fea1" candidate_uuid: "88935ab835544e9889d05a6587d8cf21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3"
I20250109 08:58:05.313814 21584 raft_consensus.cc:3054] T 90a4a2381cbe40f0b204be325ee9fea1 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:05.315347 21657 raft_consensus.cc:2463] T 90a4a2381cbe40f0b204be325ee9fea1 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88935ab835544e9889d05a6587d8cf21 in term 2.
I20250109 08:58:05.315599 21410 catalog_manager.cc:5526] T c99aacce09d44f889be5480854a22760 P 8bf6e26e98b14566899578a990b27d47 reported cstate change: term changed from 1 to 2, leader changed from a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194) to 8bf6e26e98b14566899578a990b27d47 (127.19.226.193). New cstate: current_term: 2 leader_uuid: "8bf6e26e98b14566899578a990b27d47" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:05.316583 22141 leader_election.cc:304] T 90a4a2381cbe40f0b204be325ee9fea1 P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, 88935ab835544e9889d05a6587d8cf21; no voters:
I20250109 08:58:05.319281 22787 raft_consensus.cc:2798] T 90a4a2381cbe40f0b204be325ee9fea1 P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:58:05.319729 22787 raft_consensus.cc:695] T 90a4a2381cbe40f0b204be325ee9fea1 P 88935ab835544e9889d05a6587d8cf21 [term 2 LEADER]: Becoming Leader. State: Replica: 88935ab835544e9889d05a6587d8cf21, State: Running, Role: LEADER
I20250109 08:58:05.319980 21584 raft_consensus.cc:2463] T 90a4a2381cbe40f0b204be325ee9fea1 P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88935ab835544e9889d05a6587d8cf21 in term 2.
I20250109 08:58:05.321461 22787 consensus_queue.cc:237] T 90a4a2381cbe40f0b204be325ee9fea1 P 88935ab835544e9889d05a6587d8cf21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:05.330626 21410 catalog_manager.cc:5526] T 90a4a2381cbe40f0b204be325ee9fea1 P 88935ab835544e9889d05a6587d8cf21 reported cstate change: term changed from 1 to 2, leader changed from 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195) to 88935ab835544e9889d05a6587d8cf21 (127.19.226.196). New cstate: current_term: 2 leader_uuid: "88935ab835544e9889d05a6587d8cf21" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250109 08:58:05.394277 21584 consensus_queue.cc:237] T 1d3d5d2a2c88417a97c9d57262a356dd P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
W20250109 08:58:05.400481 21553 consensus_peers.cc:487] T 1d3d5d2a2c88417a97c9d57262a356dd P a2b1a692435241ee988e0bd1661cd4c3 -> Peer 88935ab835544e9889d05a6587d8cf21 (127.19.226.196:44363): Couldn't send request to peer 88935ab835544e9889d05a6587d8cf21. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1d3d5d2a2c88417a97c9d57262a356dd. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:05.401033 21512 raft_consensus.cc:1270] T 1d3d5d2a2c88417a97c9d57262a356dd P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Refusing update from remote peer a2b1a692435241ee988e0bd1661cd4c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:58:05.401278 21657 raft_consensus.cc:1270] T 1d3d5d2a2c88417a97c9d57262a356dd P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Refusing update from remote peer a2b1a692435241ee988e0bd1661cd4c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:58:05.402091 22856 consensus_queue.cc:1035] T 1d3d5d2a2c88417a97c9d57262a356dd P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:58:05.402794 22773 consensus_queue.cc:1035] T 1d3d5d2a2c88417a97c9d57262a356dd P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:58:05.409262 22743 raft_consensus.cc:2949] T 1d3d5d2a2c88417a97c9d57262a356dd P a2b1a692435241ee988e0bd1661cd4c3 [term 2 LEADER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:58:05.410858 21657 raft_consensus.cc:2949] T 1d3d5d2a2c88417a97c9d57262a356dd P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:58:05.410773 21512 raft_consensus.cc:2949] T 1d3d5d2a2c88417a97c9d57262a356dd P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:58:05.421654 21583 consensus_queue.cc:237] T 7781c00473024307987096b596da12ac P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:58:05.425945 21410 catalog_manager.cc:5526] T 1d3d5d2a2c88417a97c9d57262a356dd P a2b1a692435241ee988e0bd1661cd4c3 reported cstate change: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New cstate: current_term: 2 leader_uuid: "a2b1a692435241ee988e0bd1661cd4c3" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:05.431890 21657 raft_consensus.cc:1270] T 7781c00473024307987096b596da12ac P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Refusing update from remote peer a2b1a692435241ee988e0bd1661cd4c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250109 08:58:05.433216 22773 consensus_queue.cc:1035] T 7781c00473024307987096b596da12ac P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20250109 08:58:05.438484 21553 consensus_peers.cc:487] T 7781c00473024307987096b596da12ac P a2b1a692435241ee988e0bd1661cd4c3 -> Peer 88935ab835544e9889d05a6587d8cf21 (127.19.226.196:44363): Couldn't send request to peer 88935ab835544e9889d05a6587d8cf21. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7781c00473024307987096b596da12ac. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:05.455964 22773 raft_consensus.cc:2949] T 7781c00473024307987096b596da12ac P a2b1a692435241ee988e0bd1661cd4c3 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:58:05.457111 21512 raft_consensus.cc:1270] T 7781c00473024307987096b596da12ac P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Refusing update from remote peer a2b1a692435241ee988e0bd1661cd4c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250109 08:58:05.457260 21660 raft_consensus.cc:2949] T 7781c00473024307987096b596da12ac P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:58:05.458420 22743 consensus_queue.cc:1035] T 7781c00473024307987096b596da12ac P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:58:05.463950 21512 raft_consensus.cc:2949] T 7781c00473024307987096b596da12ac P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:58:05.472230 21409 catalog_manager.cc:5526] T 7781c00473024307987096b596da12ac P a2b1a692435241ee988e0bd1661cd4c3 reported cstate change: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New cstate: current_term: 1 leader_uuid: "a2b1a692435241ee988e0bd1661cd4c3" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:05.474226 21583 consensus_queue.cc:237] T 56bbc306275e4a409e028a7ca1dd1602 P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
W20250109 08:58:05.481536 21553 consensus_peers.cc:487] T 56bbc306275e4a409e028a7ca1dd1602 P a2b1a692435241ee988e0bd1661cd4c3 -> Peer 88935ab835544e9889d05a6587d8cf21 (127.19.226.196:44363): Couldn't send request to peer 88935ab835544e9889d05a6587d8cf21. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 56bbc306275e4a409e028a7ca1dd1602. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:05.481989 21512 raft_consensus.cc:1270] T 56bbc306275e4a409e028a7ca1dd1602 P 8bf6e26e98b14566899578a990b27d47 [term 3 FOLLOWER]: Refusing update from remote peer a2b1a692435241ee988e0bd1661cd4c3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20250109 08:58:05.482815 21660 raft_consensus.cc:1270] T 56bbc306275e4a409e028a7ca1dd1602 P 787ddcc824c2490a808fc5a53452dc5e [term 3 FOLLOWER]: Refusing update from remote peer a2b1a692435241ee988e0bd1661cd4c3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20250109 08:58:05.483413 22856 consensus_queue.cc:1035] T 56bbc306275e4a409e028a7ca1dd1602 P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250109 08:58:05.484323 22773 consensus_queue.cc:1035] T 56bbc306275e4a409e028a7ca1dd1602 P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250109 08:58:05.492305 22773 raft_consensus.cc:2949] T 56bbc306275e4a409e028a7ca1dd1602 P a2b1a692435241ee988e0bd1661cd4c3 [term 3 LEADER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:58:05.494182 21514 raft_consensus.cc:2949] T 56bbc306275e4a409e028a7ca1dd1602 P 8bf6e26e98b14566899578a990b27d47 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:58:05.502223 21660 raft_consensus.cc:2949] T 56bbc306275e4a409e028a7ca1dd1602 P 787ddcc824c2490a808fc5a53452dc5e [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:58:05.507993 21657 consensus_queue.cc:237] T 5ac5ffe3cd65463fb448e819e3978841 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:58:05.514700 21409 catalog_manager.cc:5526] T 56bbc306275e4a409e028a7ca1dd1602 P a2b1a692435241ee988e0bd1661cd4c3 reported cstate change: config changed from index -1 to 4, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New cstate: current_term: 3 leader_uuid: "a2b1a692435241ee988e0bd1661cd4c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:05.521724 21514 raft_consensus.cc:1270] T 5ac5ffe3cd65463fb448e819e3978841 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20250109 08:58:05.522094 21627 consensus_peers.cc:487] T 5ac5ffe3cd65463fb448e819e3978841 P 787ddcc824c2490a808fc5a53452dc5e -> Peer 88935ab835544e9889d05a6587d8cf21 (127.19.226.196:44363): Couldn't send request to peer 88935ab835544e9889d05a6587d8cf21. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5ac5ffe3cd65463fb448e819e3978841. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:05.523504 22750 consensus_queue.cc:1035] T 5ac5ffe3cd65463fb448e819e3978841 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:58:05.523866 21584 raft_consensus.cc:1270] T 5ac5ffe3cd65463fb448e819e3978841 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250109 08:58:05.525945 22889 consensus_queue.cc:1035] T 5ac5ffe3cd65463fb448e819e3978841 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:58:05.547539 22904 raft_consensus.cc:2949] T 5ac5ffe3cd65463fb448e819e3978841 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:58:05.548902 21584 raft_consensus.cc:2949] T 5ac5ffe3cd65463fb448e819e3978841 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:58:05.559957 21514 raft_consensus.cc:2949] T 5ac5ffe3cd65463fb448e819e3978841 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:58:05.566274 21412 catalog_manager.cc:5526] T 5ac5ffe3cd65463fb448e819e3978841 P 787ddcc824c2490a808fc5a53452dc5e reported cstate change: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New cstate: current_term: 1 leader_uuid: "787ddcc824c2490a808fc5a53452dc5e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:05.577492 21583 raft_consensus.cc:1270] T 79c1ed8ab551478291f1f24cc940cb27 P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:58:05.578768 22750 consensus_queue.cc:1035] T 79c1ed8ab551478291f1f24cc940cb27 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20250109 08:58:05.599550 21514 raft_consensus.cc:1270] T 79c1ed8ab551478291f1f24cc940cb27 P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:58:05.601274 22889 consensus_queue.cc:1035] T 79c1ed8ab551478291f1f24cc940cb27 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:58:05.783677 21583 raft_consensus.cc:1270] T 90a4a2381cbe40f0b204be325ee9fea1 P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Refusing update from remote peer 88935ab835544e9889d05a6587d8cf21: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250109 08:58:05.784832 22787 consensus_queue.cc:1035] T 90a4a2381cbe40f0b204be325ee9fea1 P 88935ab835544e9889d05a6587d8cf21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250109 08:58:05.796291 21660 raft_consensus.cc:1270] T 90a4a2381cbe40f0b204be325ee9fea1 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Refusing update from remote peer 88935ab835544e9889d05a6587d8cf21: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250109 08:58:05.800889 22689 consensus_queue.cc:1035] T 90a4a2381cbe40f0b204be325ee9fea1 P 88935ab835544e9889d05a6587d8cf21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250109 08:58:05.815459 21588 raft_consensus.cc:1270] T c99aacce09d44f889be5480854a22760 P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Refusing update from remote peer 8bf6e26e98b14566899578a990b27d47: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:58:05.819969 22863 consensus_queue.cc:1035] T c99aacce09d44f889be5480854a22760 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:58:05.833267 21657 raft_consensus.cc:1270] T c99aacce09d44f889be5480854a22760 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Refusing update from remote peer 8bf6e26e98b14566899578a990b27d47: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:58:05.834604 22870 consensus_queue.cc:1035] T c99aacce09d44f889be5480854a22760 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:58:05.920599 20363 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250109 08:58:05.972741 21513 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "af330dcd4767423ea4c179be7f88d0c4"
dest_uuid: "8bf6e26e98b14566899578a990b27d47"
mode: GRACEFUL
new_leader_uuid: "787ddcc824c2490a808fc5a53452dc5e"
from {username='slave'} at 127.0.0.1:39274
I20250109 08:58:05.973445 21513 raft_consensus.cc:604] T af330dcd4767423ea4c179be7f88d0c4 P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Received request to transfer leadership to TS 787ddcc824c2490a808fc5a53452dc5e
I20250109 08:58:05.975975 21660 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "d355d166de534fb4a50c9f48f2df7469"
dest_uuid: "787ddcc824c2490a808fc5a53452dc5e"
mode: GRACEFUL
new_leader_uuid: "a2b1a692435241ee988e0bd1661cd4c3"
from {username='slave'} at 127.0.0.1:55054
I20250109 08:58:05.976562 21660 raft_consensus.cc:604] T d355d166de534fb4a50c9f48f2df7469 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Received request to transfer leadership to TS a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:58:05.977660 20363 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 2
I20250109 08:58:05.979372 20363 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250109 08:58:05.982342 20363 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250109 08:58:06.001394 22923 ts_tablet_manager.cc:927] T 1d3d5d2a2c88417a97c9d57262a356dd P 88935ab835544e9889d05a6587d8cf21: Initiating tablet copy from peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
I20250109 08:58:06.004685 22923 tablet_copy_client.cc:323] T 1d3d5d2a2c88417a97c9d57262a356dd P 88935ab835544e9889d05a6587d8cf21: tablet copy: Beginning tablet copy session from remote peer at address 127.19.226.194:45527
I20250109 08:58:06.006156 21599 tablet_copy_service.cc:140] P a2b1a692435241ee988e0bd1661cd4c3: Received BeginTabletCopySession request for tablet 1d3d5d2a2c88417a97c9d57262a356dd from peer 88935ab835544e9889d05a6587d8cf21 ({username='slave'} at 127.0.0.1:33652)
I20250109 08:58:06.006593 21599 tablet_copy_service.cc:161] P a2b1a692435241ee988e0bd1661cd4c3: Beginning new tablet copy session on tablet 1d3d5d2a2c88417a97c9d57262a356dd from peer 88935ab835544e9889d05a6587d8cf21 at {username='slave'} at 127.0.0.1:33652: session id = 88935ab835544e9889d05a6587d8cf21-1d3d5d2a2c88417a97c9d57262a356dd
I20250109 08:58:06.013315 21599 tablet_copy_source_session.cc:215] T 1d3d5d2a2c88417a97c9d57262a356dd P a2b1a692435241ee988e0bd1661cd4c3: Tablet Copy: opened 0 blocks and 1 log segments
I20250109 08:58:06.017016 22923 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d3d5d2a2c88417a97c9d57262a356dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:06.032709 22923 tablet_copy_client.cc:806] T 1d3d5d2a2c88417a97c9d57262a356dd P 88935ab835544e9889d05a6587d8cf21: tablet copy: Starting download of 0 data blocks...
I20250109 08:58:06.033442 22923 tablet_copy_client.cc:670] T 1d3d5d2a2c88417a97c9d57262a356dd P 88935ab835544e9889d05a6587d8cf21: tablet copy: Starting download of 1 WAL segments...
I20250109 08:58:06.035140 22925 ts_tablet_manager.cc:927] T 7781c00473024307987096b596da12ac P 88935ab835544e9889d05a6587d8cf21: Initiating tablet copy from peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
I20250109 08:58:06.044337 22923 tablet_copy_client.cc:538] T 1d3d5d2a2c88417a97c9d57262a356dd P 88935ab835544e9889d05a6587d8cf21: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250109 08:58:06.051169 22927 ts_tablet_manager.cc:927] T 56bbc306275e4a409e028a7ca1dd1602 P 88935ab835544e9889d05a6587d8cf21: Initiating tablet copy from peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
I20250109 08:58:06.055282 22925 tablet_copy_client.cc:323] T 7781c00473024307987096b596da12ac P 88935ab835544e9889d05a6587d8cf21: tablet copy: Beginning tablet copy session from remote peer at address 127.19.226.194:45527
I20250109 08:58:06.056999 21599 tablet_copy_service.cc:140] P a2b1a692435241ee988e0bd1661cd4c3: Received BeginTabletCopySession request for tablet 7781c00473024307987096b596da12ac from peer 88935ab835544e9889d05a6587d8cf21 ({username='slave'} at 127.0.0.1:33652)
I20250109 08:58:06.057406 22927 tablet_copy_client.cc:323] T 56bbc306275e4a409e028a7ca1dd1602 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Beginning tablet copy session from remote peer at address 127.19.226.194:45527
I20250109 08:58:06.057533 21599 tablet_copy_service.cc:161] P a2b1a692435241ee988e0bd1661cd4c3: Beginning new tablet copy session on tablet 7781c00473024307987096b596da12ac from peer 88935ab835544e9889d05a6587d8cf21 at {username='slave'} at 127.0.0.1:33652: session id = 88935ab835544e9889d05a6587d8cf21-7781c00473024307987096b596da12ac
I20250109 08:58:06.058966 21598 tablet_copy_service.cc:140] P a2b1a692435241ee988e0bd1661cd4c3: Received BeginTabletCopySession request for tablet 56bbc306275e4a409e028a7ca1dd1602 from peer 88935ab835544e9889d05a6587d8cf21 ({username='slave'} at 127.0.0.1:33652)
I20250109 08:58:06.059468 21598 tablet_copy_service.cc:161] P a2b1a692435241ee988e0bd1661cd4c3: Beginning new tablet copy session on tablet 56bbc306275e4a409e028a7ca1dd1602 from peer 88935ab835544e9889d05a6587d8cf21 at {username='slave'} at 127.0.0.1:33652: session id = 88935ab835544e9889d05a6587d8cf21-56bbc306275e4a409e028a7ca1dd1602
I20250109 08:58:06.060637 22923 tablet_bootstrap.cc:492] T 1d3d5d2a2c88417a97c9d57262a356dd P 88935ab835544e9889d05a6587d8cf21: Bootstrap starting.
I20250109 08:58:06.064693 21599 tablet_copy_source_session.cc:215] T 7781c00473024307987096b596da12ac P a2b1a692435241ee988e0bd1661cd4c3: Tablet Copy: opened 0 blocks and 1 log segments
I20250109 08:58:06.067791 22925 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7781c00473024307987096b596da12ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:06.069551 21598 tablet_copy_source_session.cc:215] T 56bbc306275e4a409e028a7ca1dd1602 P a2b1a692435241ee988e0bd1661cd4c3: Tablet Copy: opened 0 blocks and 1 log segments
I20250109 08:58:06.079926 22928 ts_tablet_manager.cc:927] T 5ac5ffe3cd65463fb448e819e3978841 P 88935ab835544e9889d05a6587d8cf21: Initiating tablet copy from peer 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:58:06.088704 22927 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56bbc306275e4a409e028a7ca1dd1602. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:06.112668 22923 tablet_bootstrap.cc:492] T 1d3d5d2a2c88417a97c9d57262a356dd P 88935ab835544e9889d05a6587d8cf21: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250109 08:58:06.113739 22923 tablet_bootstrap.cc:492] T 1d3d5d2a2c88417a97c9d57262a356dd P 88935ab835544e9889d05a6587d8cf21: Bootstrap complete.
I20250109 08:58:06.114472 22923 ts_tablet_manager.cc:1397] T 1d3d5d2a2c88417a97c9d57262a356dd P 88935ab835544e9889d05a6587d8cf21: Time spent bootstrapping tablet: real 0.058s user 0.023s sys 0.014s
I20250109 08:58:06.117465 22923 raft_consensus.cc:357] T 1d3d5d2a2c88417a97c9d57262a356dd P 88935ab835544e9889d05a6587d8cf21 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:58:06.118575 22923 raft_consensus.cc:738] T 1d3d5d2a2c88417a97c9d57262a356dd P 88935ab835544e9889d05a6587d8cf21 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 88935ab835544e9889d05a6587d8cf21, State: Initialized, Role: LEARNER
I20250109 08:58:06.119314 22923 consensus_queue.cc:260] T 1d3d5d2a2c88417a97c9d57262a356dd P 88935ab835544e9889d05a6587d8cf21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:58:06.130286 22923 ts_tablet_manager.cc:1428] T 1d3d5d2a2c88417a97c9d57262a356dd P 88935ab835544e9889d05a6587d8cf21: Time spent starting tablet: real 0.015s user 0.008s sys 0.000s
I20250109 08:58:06.132040 21598 tablet_copy_service.cc:342] P a2b1a692435241ee988e0bd1661cd4c3: Request end of tablet copy session 88935ab835544e9889d05a6587d8cf21-1d3d5d2a2c88417a97c9d57262a356dd received from {username='slave'} at 127.0.0.1:33652
I20250109 08:58:06.132393 21598 tablet_copy_service.cc:434] P a2b1a692435241ee988e0bd1661cd4c3: ending tablet copy session 88935ab835544e9889d05a6587d8cf21-1d3d5d2a2c88417a97c9d57262a356dd on tablet 1d3d5d2a2c88417a97c9d57262a356dd with peer 88935ab835544e9889d05a6587d8cf21
I20250109 08:58:06.141130 22928 tablet_copy_client.cc:323] T 5ac5ffe3cd65463fb448e819e3978841 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Beginning tablet copy session from remote peer at address 127.19.226.195:35731
I20250109 08:58:06.141477 22925 tablet_copy_client.cc:806] T 7781c00473024307987096b596da12ac P 88935ab835544e9889d05a6587d8cf21: tablet copy: Starting download of 0 data blocks...
I20250109 08:58:06.142027 22925 tablet_copy_client.cc:670] T 7781c00473024307987096b596da12ac P 88935ab835544e9889d05a6587d8cf21: tablet copy: Starting download of 1 WAL segments...
I20250109 08:58:06.142434 21673 tablet_copy_service.cc:140] P 787ddcc824c2490a808fc5a53452dc5e: Received BeginTabletCopySession request for tablet 5ac5ffe3cd65463fb448e819e3978841 from peer 88935ab835544e9889d05a6587d8cf21 ({username='slave'} at 127.0.0.1:52376)
I20250109 08:58:06.142913 21673 tablet_copy_service.cc:161] P 787ddcc824c2490a808fc5a53452dc5e: Beginning new tablet copy session on tablet 5ac5ffe3cd65463fb448e819e3978841 from peer 88935ab835544e9889d05a6587d8cf21 at {username='slave'} at 127.0.0.1:52376: session id = 88935ab835544e9889d05a6587d8cf21-5ac5ffe3cd65463fb448e819e3978841
I20250109 08:58:06.146232 22925 tablet_copy_client.cc:538] T 7781c00473024307987096b596da12ac P 88935ab835544e9889d05a6587d8cf21: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250109 08:58:06.148484 22927 tablet_copy_client.cc:806] T 56bbc306275e4a409e028a7ca1dd1602 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Starting download of 0 data blocks...
I20250109 08:58:06.149453 21673 tablet_copy_source_session.cc:215] T 5ac5ffe3cd65463fb448e819e3978841 P 787ddcc824c2490a808fc5a53452dc5e: Tablet Copy: opened 0 blocks and 1 log segments
I20250109 08:58:06.149684 22927 tablet_copy_client.cc:670] T 56bbc306275e4a409e028a7ca1dd1602 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Starting download of 1 WAL segments...
I20250109 08:58:06.153362 22928 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ac5ffe3cd65463fb448e819e3978841. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:06.153514 22927 tablet_copy_client.cc:538] T 56bbc306275e4a409e028a7ca1dd1602 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250109 08:58:06.158764 22925 tablet_bootstrap.cc:492] T 7781c00473024307987096b596da12ac P 88935ab835544e9889d05a6587d8cf21: Bootstrap starting.
I20250109 08:58:06.161896 22927 tablet_bootstrap.cc:492] T 56bbc306275e4a409e028a7ca1dd1602 P 88935ab835544e9889d05a6587d8cf21: Bootstrap starting.
I20250109 08:58:06.178052 22928 tablet_copy_client.cc:806] T 5ac5ffe3cd65463fb448e819e3978841 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Starting download of 0 data blocks...
I20250109 08:58:06.178593 22928 tablet_copy_client.cc:670] T 5ac5ffe3cd65463fb448e819e3978841 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Starting download of 1 WAL segments...
I20250109 08:58:06.185158 22928 tablet_copy_client.cc:538] T 5ac5ffe3cd65463fb448e819e3978841 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250109 08:58:06.187549 22927 tablet_bootstrap.cc:492] T 56bbc306275e4a409e028a7ca1dd1602 P 88935ab835544e9889d05a6587d8cf21: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250109 08:58:06.188408 22927 tablet_bootstrap.cc:492] T 56bbc306275e4a409e028a7ca1dd1602 P 88935ab835544e9889d05a6587d8cf21: Bootstrap complete.
I20250109 08:58:06.189040 22927 ts_tablet_manager.cc:1397] T 56bbc306275e4a409e028a7ca1dd1602 P 88935ab835544e9889d05a6587d8cf21: Time spent bootstrapping tablet: real 0.027s user 0.015s sys 0.008s
I20250109 08:58:06.189498 22925 tablet_bootstrap.cc:492] T 7781c00473024307987096b596da12ac P 88935ab835544e9889d05a6587d8cf21: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250109 08:58:06.190320 22925 tablet_bootstrap.cc:492] T 7781c00473024307987096b596da12ac P 88935ab835544e9889d05a6587d8cf21: Bootstrap complete.
I20250109 08:58:06.190923 22925 ts_tablet_manager.cc:1397] T 7781c00473024307987096b596da12ac P 88935ab835544e9889d05a6587d8cf21: Time spent bootstrapping tablet: real 0.032s user 0.020s sys 0.005s
I20250109 08:58:06.192866 22927 raft_consensus.cc:357] T 56bbc306275e4a409e028a7ca1dd1602 P 88935ab835544e9889d05a6587d8cf21 [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:58:06.193759 22927 raft_consensus.cc:738] T 56bbc306275e4a409e028a7ca1dd1602 P 88935ab835544e9889d05a6587d8cf21 [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: 88935ab835544e9889d05a6587d8cf21, State: Initialized, Role: LEARNER
I20250109 08:58:06.194157 22928 tablet_bootstrap.cc:492] T 5ac5ffe3cd65463fb448e819e3978841 P 88935ab835544e9889d05a6587d8cf21: Bootstrap starting.
I20250109 08:58:06.193616 22925 raft_consensus.cc:357] T 7781c00473024307987096b596da12ac P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:58:06.194620 22925 raft_consensus.cc:738] T 7781c00473024307987096b596da12ac P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 88935ab835544e9889d05a6587d8cf21, State: Initialized, Role: LEARNER
I20250109 08:58:06.194852 22927 consensus_queue.cc:260] T 56bbc306275e4a409e028a7ca1dd1602 P 88935ab835544e9889d05a6587d8cf21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:58:06.195436 22925 consensus_queue.cc:260] T 7781c00473024307987096b596da12ac P 88935ab835544e9889d05a6587d8cf21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:58:06.197649 22927 ts_tablet_manager.cc:1428] T 56bbc306275e4a409e028a7ca1dd1602 P 88935ab835544e9889d05a6587d8cf21: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20250109 08:58:06.197899 22925 ts_tablet_manager.cc:1428] T 7781c00473024307987096b596da12ac P 88935ab835544e9889d05a6587d8cf21: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20250109 08:58:06.199391 21598 tablet_copy_service.cc:342] P a2b1a692435241ee988e0bd1661cd4c3: Request end of tablet copy session 88935ab835544e9889d05a6587d8cf21-56bbc306275e4a409e028a7ca1dd1602 received from {username='slave'} at 127.0.0.1:33652
I20250109 08:58:06.199779 21598 tablet_copy_service.cc:434] P a2b1a692435241ee988e0bd1661cd4c3: ending tablet copy session 88935ab835544e9889d05a6587d8cf21-56bbc306275e4a409e028a7ca1dd1602 on tablet 56bbc306275e4a409e028a7ca1dd1602 with peer 88935ab835544e9889d05a6587d8cf21
I20250109 08:58:06.200397 21599 tablet_copy_service.cc:342] P a2b1a692435241ee988e0bd1661cd4c3: Request end of tablet copy session 88935ab835544e9889d05a6587d8cf21-7781c00473024307987096b596da12ac received from {username='slave'} at 127.0.0.1:33652
I20250109 08:58:06.200773 21599 tablet_copy_service.cc:434] P a2b1a692435241ee988e0bd1661cd4c3: ending tablet copy session 88935ab835544e9889d05a6587d8cf21-7781c00473024307987096b596da12ac on tablet 7781c00473024307987096b596da12ac with peer 88935ab835544e9889d05a6587d8cf21
I20250109 08:58:06.201458 22889 raft_consensus.cc:988] T d355d166de534fb4a50c9f48f2df7469 P 787ddcc824c2490a808fc5a53452dc5e: : Instructing follower a2b1a692435241ee988e0bd1661cd4c3 to start an election
I20250109 08:58:06.201906 22889 raft_consensus.cc:1076] T d355d166de534fb4a50c9f48f2df7469 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Signalling peer a2b1a692435241ee988e0bd1661cd4c3 to start an election
I20250109 08:58:06.205144 21583 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d355d166de534fb4a50c9f48f2df7469"
dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3"
from {username='slave'} at 127.0.0.1:34746
I20250109 08:58:06.205622 21583 raft_consensus.cc:491] T d355d166de534fb4a50c9f48f2df7469 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:58:06.205968 21583 raft_consensus.cc:3054] T d355d166de534fb4a50c9f48f2df7469 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:06.213331 21583 raft_consensus.cc:513] T d355d166de534fb4a50c9f48f2df7469 P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:58:06.215030 21583 leader_election.cc:290] T d355d166de534fb4a50c9f48f2df7469 P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 2 election: Requested vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:58:06.215806 21513 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d355d166de534fb4a50c9f48f2df7469" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8bf6e26e98b14566899578a990b27d47"
I20250109 08:58:06.216270 21513 raft_consensus.cc:3054] T d355d166de534fb4a50c9f48f2df7469 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:06.216701 21660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d355d166de534fb4a50c9f48f2df7469" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "787ddcc824c2490a808fc5a53452dc5e"
I20250109 08:58:06.217235 21660 raft_consensus.cc:3049] T d355d166de534fb4a50c9f48f2df7469 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Stepping down as leader of term 1
I20250109 08:58:06.217487 21660 raft_consensus.cc:738] T d355d166de534fb4a50c9f48f2df7469 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Running, Role: LEADER
I20250109 08:58:06.218118 21660 consensus_queue.cc:260] T d355d166de534fb4a50c9f48f2df7469 P 787ddcc824c2490a808fc5a53452dc5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:58:06.219069 21660 raft_consensus.cc:3054] T d355d166de534fb4a50c9f48f2df7469 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:06.231881 21660 raft_consensus.cc:2463] T d355d166de534fb4a50c9f48f2df7469 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2b1a692435241ee988e0bd1661cd4c3 in term 2.
I20250109 08:58:06.234227 21552 leader_election.cc:304] T d355d166de534fb4a50c9f48f2df7469 P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, a2b1a692435241ee988e0bd1661cd4c3; no voters:
I20250109 08:58:06.235028 22900 raft_consensus.cc:2798] T d355d166de534fb4a50c9f48f2df7469 P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:58:06.235530 22900 raft_consensus.cc:695] T d355d166de534fb4a50c9f48f2df7469 P a2b1a692435241ee988e0bd1661cd4c3 [term 2 LEADER]: Becoming Leader. State: Replica: a2b1a692435241ee988e0bd1661cd4c3, State: Running, Role: LEADER
I20250109 08:58:06.236303 22900 consensus_queue.cc:237] T d355d166de534fb4a50c9f48f2df7469 P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:58:06.241518 22928 tablet_bootstrap.cc:492] T 5ac5ffe3cd65463fb448e819e3978841 P 88935ab835544e9889d05a6587d8cf21: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250109 08:58:06.242305 22928 tablet_bootstrap.cc:492] T 5ac5ffe3cd65463fb448e819e3978841 P 88935ab835544e9889d05a6587d8cf21: Bootstrap complete.
I20250109 08:58:06.242918 22928 ts_tablet_manager.cc:1397] T 5ac5ffe3cd65463fb448e819e3978841 P 88935ab835544e9889d05a6587d8cf21: Time spent bootstrapping tablet: real 0.049s user 0.029s sys 0.007s
I20250109 08:58:06.245606 22928 raft_consensus.cc:357] T 5ac5ffe3cd65463fb448e819e3978841 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:58:06.246438 22928 raft_consensus.cc:738] T 5ac5ffe3cd65463fb448e819e3978841 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 88935ab835544e9889d05a6587d8cf21, State: Initialized, Role: LEARNER
I20250109 08:58:06.247001 22928 consensus_queue.cc:260] T 5ac5ffe3cd65463fb448e819e3978841 P 88935ab835544e9889d05a6587d8cf21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:58:06.249614 22928 ts_tablet_manager.cc:1428] T 5ac5ffe3cd65463fb448e819e3978841 P 88935ab835544e9889d05a6587d8cf21: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
I20250109 08:58:06.251196 21673 tablet_copy_service.cc:342] P 787ddcc824c2490a808fc5a53452dc5e: Request end of tablet copy session 88935ab835544e9889d05a6587d8cf21-5ac5ffe3cd65463fb448e819e3978841 received from {username='slave'} at 127.0.0.1:52376
I20250109 08:58:06.251616 21673 tablet_copy_service.cc:434] P 787ddcc824c2490a808fc5a53452dc5e: ending tablet copy session 88935ab835544e9889d05a6587d8cf21-5ac5ffe3cd65463fb448e819e3978841 on tablet 5ac5ffe3cd65463fb448e819e3978841 with peer 88935ab835544e9889d05a6587d8cf21
I20250109 08:58:06.251072 21410 catalog_manager.cc:5526] T d355d166de534fb4a50c9f48f2df7469 P a2b1a692435241ee988e0bd1661cd4c3 reported cstate change: term changed from 1 to 2, leader changed from 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195) to a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194). New cstate: current_term: 2 leader_uuid: "a2b1a692435241ee988e0bd1661cd4c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:06.257400 21513 raft_consensus.cc:2463] T d355d166de534fb4a50c9f48f2df7469 P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2b1a692435241ee988e0bd1661cd4c3 in term 2.
I20250109 08:58:06.384997 22176 raft_consensus.cc:1212] T 1d3d5d2a2c88417a97c9d57262a356dd P 88935ab835544e9889d05a6587d8cf21 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.1->[2.2-2.2] Dedup: 2.2->[]
I20250109 08:58:06.464020 22176 raft_consensus.cc:1212] T 7781c00473024307987096b596da12ac P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20250109 08:58:06.503902 22870 raft_consensus.cc:988] T af330dcd4767423ea4c179be7f88d0c4 P 8bf6e26e98b14566899578a990b27d47: : Instructing follower 787ddcc824c2490a808fc5a53452dc5e to start an election
I20250109 08:58:06.504434 22870 raft_consensus.cc:1076] T af330dcd4767423ea4c179be7f88d0c4 P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Signalling peer 787ddcc824c2490a808fc5a53452dc5e to start an election
I20250109 08:58:06.505749 21660 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "af330dcd4767423ea4c179be7f88d0c4"
dest_uuid: "787ddcc824c2490a808fc5a53452dc5e"
from {username='slave'} at 127.0.0.1:58440
I20250109 08:58:06.506184 21660 raft_consensus.cc:491] T af330dcd4767423ea4c179be7f88d0c4 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:58:06.506449 21660 raft_consensus.cc:3054] T af330dcd4767423ea4c179be7f88d0c4 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:06.511785 21660 raft_consensus.cc:513] T af330dcd4767423ea4c179be7f88d0c4 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:58:06.513674 21660 leader_election.cc:290] T af330dcd4767423ea4c179be7f88d0c4 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 2 election: Requested vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
I20250109 08:58:06.513921 21513 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af330dcd4767423ea4c179be7f88d0c4" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8bf6e26e98b14566899578a990b27d47"
I20250109 08:58:06.514156 21583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af330dcd4767423ea4c179be7f88d0c4" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3"
I20250109 08:58:06.514514 21513 raft_consensus.cc:3049] T af330dcd4767423ea4c179be7f88d0c4 P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Stepping down as leader of term 1
I20250109 08:58:06.514645 21583 raft_consensus.cc:3054] T af330dcd4767423ea4c179be7f88d0c4 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:06.514861 21513 raft_consensus.cc:738] T af330dcd4767423ea4c179be7f88d0c4 P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8bf6e26e98b14566899578a990b27d47, State: Running, Role: LEADER
I20250109 08:58:06.515534 21513 consensus_queue.cc:260] T af330dcd4767423ea4c179be7f88d0c4 P 8bf6e26e98b14566899578a990b27d47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:58:06.516633 21513 raft_consensus.cc:3054] T af330dcd4767423ea4c179be7f88d0c4 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:06.519758 21583 raft_consensus.cc:2463] T af330dcd4767423ea4c179be7f88d0c4 P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 2.
I20250109 08:58:06.521395 21627 leader_election.cc:304] T af330dcd4767423ea4c179be7f88d0c4 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, a2b1a692435241ee988e0bd1661cd4c3; no voters:
I20250109 08:58:06.521741 21513 raft_consensus.cc:2463] T af330dcd4767423ea4c179be7f88d0c4 P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 2.
I20250109 08:58:06.522140 22750 raft_consensus.cc:2798] T af330dcd4767423ea4c179be7f88d0c4 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:58:06.522568 22750 raft_consensus.cc:695] T af330dcd4767423ea4c179be7f88d0c4 P 787ddcc824c2490a808fc5a53452dc5e [term 2 LEADER]: Becoming Leader. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Running, Role: LEADER
I20250109 08:58:06.523272 22750 consensus_queue.cc:237] T af330dcd4767423ea4c179be7f88d0c4 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:58:06.535921 21410 catalog_manager.cc:5526] T af330dcd4767423ea4c179be7f88d0c4 P 787ddcc824c2490a808fc5a53452dc5e reported cstate change: term changed from 1 to 2, leader changed from 8bf6e26e98b14566899578a990b27d47 (127.19.226.193) to 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195). New cstate: current_term: 2 leader_uuid: "787ddcc824c2490a808fc5a53452dc5e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: HEALTHY } } }
I20250109 08:58:06.628127 22176 raft_consensus.cc:1212] T 56bbc306275e4a409e028a7ca1dd1602 P 88935ab835544e9889d05a6587d8cf21 [term 3 LEARNER]: Deduplicated request from leader. Original: 3.3->[3.4-3.4] Dedup: 3.4->[]
I20250109 08:58:06.641990 22176 raft_consensus.cc:1212] T 5ac5ffe3cd65463fb448e819e3978841 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20250109 08:58:06.657686 21514 raft_consensus.cc:1270] T d355d166de534fb4a50c9f48f2df7469 P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Refusing update from remote peer a2b1a692435241ee988e0bd1661cd4c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:58:06.658912 22856 consensus_queue.cc:1035] T d355d166de534fb4a50c9f48f2df7469 P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:58:06.672184 21656 raft_consensus.cc:1270] T d355d166de534fb4a50c9f48f2df7469 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Refusing update from remote peer a2b1a692435241ee988e0bd1661cd4c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:58:06.674074 22856 consensus_queue.cc:1035] T d355d166de534fb4a50c9f48f2df7469 P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:58:06.819453 22773 raft_consensus.cc:1059] T 1d3d5d2a2c88417a97c9d57262a356dd P a2b1a692435241ee988e0bd1661cd4c3: attempting to promote NON_VOTER 88935ab835544e9889d05a6587d8cf21 to VOTER
I20250109 08:58:06.821715 22773 consensus_queue.cc:237] T 1d3d5d2a2c88417a97c9d57262a356dd P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 0, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:06.831012 22176 raft_consensus.cc:1270] T 1d3d5d2a2c88417a97c9d57262a356dd P 88935ab835544e9889d05a6587d8cf21 [term 2 LEARNER]: Refusing update from remote peer a2b1a692435241ee988e0bd1661cd4c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20250109 08:58:06.831190 21660 raft_consensus.cc:1270] T 1d3d5d2a2c88417a97c9d57262a356dd P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Refusing update from remote peer a2b1a692435241ee988e0bd1661cd4c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20250109 08:58:06.832311 21514 raft_consensus.cc:1270] T 1d3d5d2a2c88417a97c9d57262a356dd P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Refusing update from remote peer a2b1a692435241ee988e0bd1661cd4c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20250109 08:58:06.833451 22900 consensus_queue.cc:1035] T 1d3d5d2a2c88417a97c9d57262a356dd P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:58:06.834581 22856 consensus_queue.cc:1035] T 1d3d5d2a2c88417a97c9d57262a356dd P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:58:06.836228 22900 consensus_queue.cc:1035] T 1d3d5d2a2c88417a97c9d57262a356dd P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:58:06.859730 22773 raft_consensus.cc:2949] T 1d3d5d2a2c88417a97c9d57262a356dd P a2b1a692435241ee988e0bd1661cd4c3 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:06.864317 21660 raft_consensus.cc:2949] T 1d3d5d2a2c88417a97c9d57262a356dd P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:06.872298 22176 raft_consensus.cc:2949] T 1d3d5d2a2c88417a97c9d57262a356dd P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:06.875205 21513 raft_consensus.cc:2949] T 1d3d5d2a2c88417a97c9d57262a356dd P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:06.877354 21412 catalog_manager.cc:5526] T 1d3d5d2a2c88417a97c9d57262a356dd P a2b1a692435241ee988e0bd1661cd4c3 reported cstate change: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "a2b1a692435241ee988e0bd1661cd4c3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250109 08:58:06.914052 21583 consensus_queue.cc:237] T 1d3d5d2a2c88417a97c9d57262a356dd P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:06.930562 21660 raft_consensus.cc:1270] T 1d3d5d2a2c88417a97c9d57262a356dd P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Refusing update from remote peer a2b1a692435241ee988e0bd1661cd4c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20250109 08:58:06.933080 22946 consensus_queue.cc:1035] T 1d3d5d2a2c88417a97c9d57262a356dd P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20250109 08:58:06.933833 22174 raft_consensus.cc:1270] T 1d3d5d2a2c88417a97c9d57262a356dd P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Refusing update from remote peer a2b1a692435241ee988e0bd1661cd4c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20250109 08:58:06.945858 21584 raft_consensus.cc:1270] T af330dcd4767423ea4c179be7f88d0c4 P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:58:06.946648 22900 raft_consensus.cc:2949] T 1d3d5d2a2c88417a97c9d57262a356dd P a2b1a692435241ee988e0bd1661cd4c3 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER 8bf6e26e98b14566899578a990b27d47 (127.19.226.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:06.953001 21656 raft_consensus.cc:2949] T 1d3d5d2a2c88417a97c9d57262a356dd P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER 8bf6e26e98b14566899578a990b27d47 (127.19.226.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:06.955519 22946 consensus_queue.cc:1035] T 1d3d5d2a2c88417a97c9d57262a356dd P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250109 08:58:06.955885 22889 consensus_queue.cc:1035] T af330dcd4767423ea4c179be7f88d0c4 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:58:06.957657 22946 raft_consensus.cc:1059] T 7781c00473024307987096b596da12ac P a2b1a692435241ee988e0bd1661cd4c3: attempting to promote NON_VOTER 88935ab835544e9889d05a6587d8cf21 to VOTER
I20250109 08:58:06.957943 21397 catalog_manager.cc:5039] ChangeConfig:REMOVE_PEER RPC for tablet 1d3d5d2a2c88417a97c9d57262a356dd with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250109 08:58:06.960042 22946 consensus_queue.cc:237] T 7781c00473024307987096b596da12ac P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:06.962137 22177 raft_consensus.cc:2949] T 1d3d5d2a2c88417a97c9d57262a356dd P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER 8bf6e26e98b14566899578a990b27d47 (127.19.226.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:06.971032 21660 raft_consensus.cc:1270] T 7781c00473024307987096b596da12ac P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Refusing update from remote peer a2b1a692435241ee988e0bd1661cd4c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:58:06.971066 22176 raft_consensus.cc:1270] T 7781c00473024307987096b596da12ac P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Refusing update from remote peer a2b1a692435241ee988e0bd1661cd4c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:58:06.973160 22946 consensus_queue.cc:1035] T 7781c00473024307987096b596da12ac P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:58:06.974987 21513 raft_consensus.cc:1270] T 7781c00473024307987096b596da12ac P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Refusing update from remote peer a2b1a692435241ee988e0bd1661cd4c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:58:06.980538 22900 consensus_queue.cc:1035] T 7781c00473024307987096b596da12ac P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:58:06.992898 21410 catalog_manager.cc:5526] T 1d3d5d2a2c88417a97c9d57262a356dd P 787ddcc824c2490a808fc5a53452dc5e reported cstate change: config changed from index 3 to 4, VOTER 8bf6e26e98b14566899578a990b27d47 (127.19.226.193) evicted. New cstate: current_term: 2 leader_uuid: "a2b1a692435241ee988e0bd1661cd4c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:07.002236 22900 consensus_queue.cc:1035] T 7781c00473024307987096b596da12ac P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20250109 08:58:07.018800 21503 tablet_service.cc:1514] Processing DeleteTablet for tablet 1d3d5d2a2c88417a97c9d57262a356dd with delete_type TABLET_DATA_TOMBSTONED (TS 8bf6e26e98b14566899578a990b27d47 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:39242
I20250109 08:58:07.021348 22955 tablet_replica.cc:331] T 1d3d5d2a2c88417a97c9d57262a356dd P 8bf6e26e98b14566899578a990b27d47: stopping tablet replica
I20250109 08:58:07.027498 21513 raft_consensus.cc:1270] T af330dcd4767423ea4c179be7f88d0c4 P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:58:07.028601 22750 consensus_queue.cc:1035] T af330dcd4767423ea4c179be7f88d0c4 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:58:07.033538 22773 raft_consensus.cc:2949] T 7781c00473024307987096b596da12ac P a2b1a692435241ee988e0bd1661cd4c3 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:07.036232 20363 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250109 08:58:07.034348 21514 raft_consensus.cc:2949] T 7781c00473024307987096b596da12ac P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:07.041119 22955 raft_consensus.cc:2238] T 1d3d5d2a2c88417a97c9d57262a356dd P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:07.041939 22955 raft_consensus.cc:2267] T 1d3d5d2a2c88417a97c9d57262a356dd P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:07.044945 22955 ts_tablet_manager.cc:1905] T 1d3d5d2a2c88417a97c9d57262a356dd P 8bf6e26e98b14566899578a990b27d47: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250109 08:58:07.034348 21656 raft_consensus.cc:2949] T 7781c00473024307987096b596da12ac P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:07.050401 22856 raft_consensus.cc:1059] T 56bbc306275e4a409e028a7ca1dd1602 P a2b1a692435241ee988e0bd1661cd4c3: attempting to promote NON_VOTER 88935ab835544e9889d05a6587d8cf21 to VOTER
I20250109 08:58:07.053037 22856 consensus_queue.cc:237] T 56bbc306275e4a409e028a7ca1dd1602 P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 2, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:07.061002 22176 raft_consensus.cc:2949] T 7781c00473024307987096b596da12ac P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:07.062101 22955 ts_tablet_manager.cc:1918] T 1d3d5d2a2c88417a97c9d57262a356dd P 8bf6e26e98b14566899578a990b27d47: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.3
I20250109 08:58:07.062600 22955 log.cc:1198] T 1d3d5d2a2c88417a97c9d57262a356dd P 8bf6e26e98b14566899578a990b27d47: Deleting WAL directory at /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736412987538469-20363-0/minicluster-data/ts-0-root/wals/1d3d5d2a2c88417a97c9d57262a356dd
I20250109 08:58:07.064105 21396 catalog_manager.cc:4872] TS 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): tablet 1d3d5d2a2c88417a97c9d57262a356dd (table test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]) successfully deleted
I20250109 08:58:07.068079 21412 catalog_manager.cc:5526] T 7781c00473024307987096b596da12ac P a2b1a692435241ee988e0bd1661cd4c3 reported cstate change: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a2b1a692435241ee988e0bd1661cd4c3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250109 08:58:07.116001 21588 consensus_queue.cc:237] T 7781c00473024307987096b596da12ac P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:07.123381 21657 raft_consensus.cc:1270] T 7781c00473024307987096b596da12ac P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Refusing update from remote peer a2b1a692435241ee988e0bd1661cd4c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250109 08:58:07.127894 22946 consensus_queue.cc:1035] T 7781c00473024307987096b596da12ac P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20250109 08:58:07.130864 22176 raft_consensus.cc:1270] T 7781c00473024307987096b596da12ac P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Refusing update from remote peer a2b1a692435241ee988e0bd1661cd4c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250109 08:58:07.132493 22959 consensus_queue.cc:1035] T 7781c00473024307987096b596da12ac P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250109 08:58:07.139593 22176 raft_consensus.cc:1270] T 56bbc306275e4a409e028a7ca1dd1602 P 88935ab835544e9889d05a6587d8cf21 [term 3 LEARNER]: Refusing update from remote peer a2b1a692435241ee988e0bd1661cd4c3: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20250109 08:58:07.140398 21513 raft_consensus.cc:1270] T 56bbc306275e4a409e028a7ca1dd1602 P 8bf6e26e98b14566899578a990b27d47 [term 3 FOLLOWER]: Refusing update from remote peer a2b1a692435241ee988e0bd1661cd4c3: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20250109 08:58:07.144735 22900 consensus_queue.cc:1035] T 56bbc306275e4a409e028a7ca1dd1602 P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250109 08:58:07.151628 21657 raft_consensus.cc:1270] T 56bbc306275e4a409e028a7ca1dd1602 P 787ddcc824c2490a808fc5a53452dc5e [term 3 FOLLOWER]: Refusing update from remote peer a2b1a692435241ee988e0bd1661cd4c3: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20250109 08:58:07.153591 22900 raft_consensus.cc:2949] T 7781c00473024307987096b596da12ac P a2b1a692435241ee988e0bd1661cd4c3 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8bf6e26e98b14566899578a990b27d47 (127.19.226.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:07.155097 22773 consensus_queue.cc:1035] T 56bbc306275e4a409e028a7ca1dd1602 P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250109 08:58:07.161410 22750 raft_consensus.cc:1059] T 5ac5ffe3cd65463fb448e819e3978841 P 787ddcc824c2490a808fc5a53452dc5e: attempting to promote NON_VOTER 88935ab835544e9889d05a6587d8cf21 to VOTER
I20250109 08:58:07.161820 22959 consensus_queue.cc:1035] T 56bbc306275e4a409e028a7ca1dd1602 P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250109 08:58:07.163564 22750 consensus_queue.cc:237] T 5ac5ffe3cd65463fb448e819e3978841 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:07.167991 22176 raft_consensus.cc:2949] T 7781c00473024307987096b596da12ac P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8bf6e26e98b14566899578a990b27d47 (127.19.226.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:07.176004 21397 catalog_manager.cc:5039] ChangeConfig:REMOVE_PEER RPC for tablet 7781c00473024307987096b596da12ac with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250109 08:58:07.164755 21656 raft_consensus.cc:2949] T 7781c00473024307987096b596da12ac P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8bf6e26e98b14566899578a990b27d47 (127.19.226.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:07.183668 21410 catalog_manager.cc:5526] T 7781c00473024307987096b596da12ac P 88935ab835544e9889d05a6587d8cf21 reported cstate change: config changed from index 3 to 4, VOTER 8bf6e26e98b14566899578a990b27d47 (127.19.226.193) evicted. New cstate: current_term: 1 leader_uuid: "a2b1a692435241ee988e0bd1661cd4c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:07.185681 22177 raft_consensus.cc:2949] T 56bbc306275e4a409e028a7ca1dd1602 P 88935ab835544e9889d05a6587d8cf21 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:07.178431 22959 raft_consensus.cc:2949] T 56bbc306275e4a409e028a7ca1dd1602 P a2b1a692435241ee988e0bd1661cd4c3 [term 3 LEADER]: Committing config change with OpId 3.5: config changed from index 4 to 5, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:07.204102 21503 tablet_service.cc:1514] Processing DeleteTablet for tablet 7781c00473024307987096b596da12ac with delete_type TABLET_DATA_TOMBSTONED (TS 8bf6e26e98b14566899578a990b27d47 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:39242
I20250109 08:58:07.205030 22955 tablet_replica.cc:331] T 7781c00473024307987096b596da12ac P 8bf6e26e98b14566899578a990b27d47: stopping tablet replica
I20250109 08:58:07.206099 22955 raft_consensus.cc:2238] T 7781c00473024307987096b596da12ac P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:07.206820 22955 raft_consensus.cc:2267] T 7781c00473024307987096b596da12ac P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:07.210510 22955 ts_tablet_manager.cc:1905] T 7781c00473024307987096b596da12ac P 8bf6e26e98b14566899578a990b27d47: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250109 08:58:07.217980 21410 catalog_manager.cc:5526] T 56bbc306275e4a409e028a7ca1dd1602 P 88935ab835544e9889d05a6587d8cf21 reported cstate change: config changed from index 4 to 5, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "a2b1a692435241ee988e0bd1661cd4c3" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:07.219957 21588 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "363fa693a5114f2bb22f4ab67722a1c6"
dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3"
mode: GRACEFUL
new_leader_uuid: "8bf6e26e98b14566899578a990b27d47"
from {username='slave'} at 127.0.0.1:46832
I20250109 08:58:07.220603 21588 raft_consensus.cc:604] T 363fa693a5114f2bb22f4ab67722a1c6 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 LEADER]: Received request to transfer leadership to TS 8bf6e26e98b14566899578a990b27d47
I20250109 08:58:07.223541 21658 raft_consensus.cc:2949] T 56bbc306275e4a409e028a7ca1dd1602 P 787ddcc824c2490a808fc5a53452dc5e [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:07.226814 21513 raft_consensus.cc:1270] T 5ac5ffe3cd65463fb448e819e3978841 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:58:07.229151 22177 raft_consensus.cc:1270] T 5ac5ffe3cd65463fb448e819e3978841 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:58:07.229557 21588 raft_consensus.cc:1270] T 5ac5ffe3cd65463fb448e819e3978841 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:58:07.235045 21656 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "37280454805d47f98a415a8f0c16d869"
dest_uuid: "787ddcc824c2490a808fc5a53452dc5e"
mode: GRACEFUL
new_leader_uuid: "88935ab835544e9889d05a6587d8cf21"
from {username='slave'} at 127.0.0.1:55054
I20250109 08:58:07.235478 21656 raft_consensus.cc:604] T 37280454805d47f98a415a8f0c16d869 P 787ddcc824c2490a808fc5a53452dc5e [term 2 LEADER]: Received request to transfer leadership to TS 88935ab835544e9889d05a6587d8cf21
I20250109 08:58:07.228130 22904 consensus_queue.cc:1035] T 5ac5ffe3cd65463fb448e819e3978841 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.012s
I20250109 08:58:07.247048 21514 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "d6f988a231a74f6fa112da9a527d4a0b"
dest_uuid: "8bf6e26e98b14566899578a990b27d47"
mode: GRACEFUL
new_leader_uuid: "a2b1a692435241ee988e0bd1661cd4c3"
from {username='slave'} at 127.0.0.1:39274
I20250109 08:58:07.248687 22889 consensus_queue.cc:1035] T 5ac5ffe3cd65463fb448e819e3978841 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:58:07.248304 21514 raft_consensus.cc:604] T d6f988a231a74f6fa112da9a527d4a0b P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Received request to transfer leadership to TS a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:58:07.251892 22750 consensus_queue.cc:1035] T 5ac5ffe3cd65463fb448e819e3978841 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:58:07.253702 20363 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 3
I20250109 08:58:07.255227 20363 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250109 08:58:07.255802 20363 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250109 08:58:07.284191 21514 raft_consensus.cc:2949] T 56bbc306275e4a409e028a7ca1dd1602 P 8bf6e26e98b14566899578a990b27d47 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:07.291735 21584 consensus_queue.cc:237] T 56bbc306275e4a409e028a7ca1dd1602 P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:07.315340 21512 raft_consensus.cc:1270] T 56bbc306275e4a409e028a7ca1dd1602 P 8bf6e26e98b14566899578a990b27d47 [term 3 FOLLOWER]: Refusing update from remote peer a2b1a692435241ee988e0bd1661cd4c3: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20250109 08:58:07.316957 22900 consensus_queue.cc:1035] T 56bbc306275e4a409e028a7ca1dd1602 P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20250109 08:58:07.329721 22900 raft_consensus.cc:2949] T 56bbc306275e4a409e028a7ca1dd1602 P a2b1a692435241ee988e0bd1661cd4c3 [term 3 LEADER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:07.331029 22750 raft_consensus.cc:988] T 37280454805d47f98a415a8f0c16d869 P 787ddcc824c2490a808fc5a53452dc5e: : Instructing follower 88935ab835544e9889d05a6587d8cf21 to start an election
I20250109 08:58:07.331475 22750 raft_consensus.cc:1076] T 37280454805d47f98a415a8f0c16d869 P 787ddcc824c2490a808fc5a53452dc5e [term 2 LEADER]: Signalling peer 88935ab835544e9889d05a6587d8cf21 to start an election
I20250109 08:58:07.331661 21588 raft_consensus.cc:2949] T 5ac5ffe3cd65463fb448e819e3978841 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:07.333863 22964 raft_consensus.cc:2949] T 5ac5ffe3cd65463fb448e819e3978841 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:07.335630 22176 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "37280454805d47f98a415a8f0c16d869"
dest_uuid: "88935ab835544e9889d05a6587d8cf21"
from {username='slave'} at 127.0.0.1:60458
I20250109 08:58:07.336166 22176 raft_consensus.cc:491] T 37280454805d47f98a415a8f0c16d869 P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:58:07.336499 22176 raft_consensus.cc:3054] T 37280454805d47f98a415a8f0c16d869 P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Advancing to term 3
I20250109 08:58:07.337261 21512 raft_consensus.cc:2949] T 56bbc306275e4a409e028a7ca1dd1602 P 8bf6e26e98b14566899578a990b27d47 [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:07.342250 22176 raft_consensus.cc:513] T 37280454805d47f98a415a8f0c16d869 P 88935ab835544e9889d05a6587d8cf21 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:07.345265 22177 raft_consensus.cc:1270] T 56bbc306275e4a409e028a7ca1dd1602 P 88935ab835544e9889d05a6587d8cf21 [term 3 FOLLOWER]: Refusing update from remote peer a2b1a692435241ee988e0bd1661cd4c3: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20250109 08:58:07.346494 21656 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37280454805d47f98a415a8f0c16d869" candidate_uuid: "88935ab835544e9889d05a6587d8cf21" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "787ddcc824c2490a808fc5a53452dc5e"
I20250109 08:58:07.345793 22174 raft_consensus.cc:2949] T 5ac5ffe3cd65463fb448e819e3978841 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:07.347098 21656 raft_consensus.cc:3049] T 37280454805d47f98a415a8f0c16d869 P 787ddcc824c2490a808fc5a53452dc5e [term 2 LEADER]: Stepping down as leader of term 2
I20250109 08:58:07.347402 21656 raft_consensus.cc:738] T 37280454805d47f98a415a8f0c16d869 P 787ddcc824c2490a808fc5a53452dc5e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Running, Role: LEADER
I20250109 08:58:07.351138 21412 catalog_manager.cc:5526] T 5ac5ffe3cd65463fb448e819e3978841 P 787ddcc824c2490a808fc5a53452dc5e reported cstate change: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "787ddcc824c2490a808fc5a53452dc5e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250109 08:58:07.351409 21513 raft_consensus.cc:2949] T 5ac5ffe3cd65463fb448e819e3978841 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:07.352413 21656 consensus_queue.cc:260] T 37280454805d47f98a415a8f0c16d869 P 787ddcc824c2490a808fc5a53452dc5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:07.353325 22773 consensus_queue.cc:1035] T 56bbc306275e4a409e028a7ca1dd1602 P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20250109 08:58:07.353466 21656 raft_consensus.cc:3054] T 37280454805d47f98a415a8f0c16d869 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Advancing to term 3
I20250109 08:58:07.355703 22176 leader_election.cc:290] T 37280454805d47f98a415a8f0c16d869 P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 3 election: Requested vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:58:07.366060 21656 raft_consensus.cc:2463] T 37280454805d47f98a415a8f0c16d869 P 787ddcc824c2490a808fc5a53452dc5e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88935ab835544e9889d05a6587d8cf21 in term 3.
I20250109 08:58:07.365751 22177 raft_consensus.cc:2949] T 56bbc306275e4a409e028a7ca1dd1602 P 88935ab835544e9889d05a6587d8cf21 [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:07.373274 21397 catalog_manager.cc:5039] ChangeConfig:REMOVE_PEER RPC for tablet 56bbc306275e4a409e028a7ca1dd1602 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250109 08:58:07.366439 21514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37280454805d47f98a415a8f0c16d869" candidate_uuid: "88935ab835544e9889d05a6587d8cf21" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "8bf6e26e98b14566899578a990b27d47"
I20250109 08:58:07.374269 21514 raft_consensus.cc:3054] T 37280454805d47f98a415a8f0c16d869 P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Advancing to term 3
I20250109 08:58:07.374549 22955 ts_tablet_manager.cc:1918] T 7781c00473024307987096b596da12ac P 8bf6e26e98b14566899578a990b27d47: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20250109 08:58:07.379832 22955 log.cc:1198] T 7781c00473024307987096b596da12ac P 8bf6e26e98b14566899578a990b27d47: Deleting WAL directory at /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736412987538469-20363-0/minicluster-data/ts-0-root/wals/7781c00473024307987096b596da12ac
I20250109 08:58:07.381449 21396 catalog_manager.cc:4872] TS 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): tablet 7781c00473024307987096b596da12ac (table test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]) successfully deleted
I20250109 08:58:07.375207 22141 leader_election.cc:304] T 37280454805d47f98a415a8f0c16d869 P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, 88935ab835544e9889d05a6587d8cf21; no voters:
I20250109 08:58:07.382905 22916 raft_consensus.cc:2798] T 37280454805d47f98a415a8f0c16d869 P 88935ab835544e9889d05a6587d8cf21 [term 3 FOLLOWER]: Leader election won for term 3
I20250109 08:58:07.383345 22916 raft_consensus.cc:695] T 37280454805d47f98a415a8f0c16d869 P 88935ab835544e9889d05a6587d8cf21 [term 3 LEADER]: Becoming Leader. State: Replica: 88935ab835544e9889d05a6587d8cf21, State: Running, Role: LEADER
I20250109 08:58:07.384176 22916 consensus_queue.cc:237] T 37280454805d47f98a415a8f0c16d869 P 88935ab835544e9889d05a6587d8cf21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:07.387784 22959 raft_consensus.cc:988] T 363fa693a5114f2bb22f4ab67722a1c6 P a2b1a692435241ee988e0bd1661cd4c3: : Instructing follower 8bf6e26e98b14566899578a990b27d47 to start an election
I20250109 08:58:07.388278 22773 raft_consensus.cc:1076] T 363fa693a5114f2bb22f4ab67722a1c6 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 LEADER]: Signalling peer 8bf6e26e98b14566899578a990b27d47 to start an election
I20250109 08:58:07.390440 21513 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "363fa693a5114f2bb22f4ab67722a1c6"
dest_uuid: "8bf6e26e98b14566899578a990b27d47"
from {username='slave'} at 127.0.0.1:39250
I20250109 08:58:07.391690 21513 raft_consensus.cc:491] T 363fa693a5114f2bb22f4ab67722a1c6 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:58:07.392976 21513 raft_consensus.cc:3054] T 363fa693a5114f2bb22f4ab67722a1c6 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:07.391325 21410 catalog_manager.cc:5526] T 56bbc306275e4a409e028a7ca1dd1602 P a2b1a692435241ee988e0bd1661cd4c3 reported cstate change: config changed from index 5 to 6, VOTER 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195) evicted. New cstate: current_term: 3 leader_uuid: "a2b1a692435241ee988e0bd1661cd4c3" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:07.406672 21653 tablet_service.cc:1514] Processing DeleteTablet for tablet 56bbc306275e4a409e028a7ca1dd1602 with delete_type TABLET_DATA_TOMBSTONED (TS 787ddcc824c2490a808fc5a53452dc5e not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:58436
I20250109 08:58:07.407948 21656 consensus_queue.cc:237] T 5ac5ffe3cd65463fb448e819e3978841 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:07.416078 22970 tablet_replica.cc:331] T 56bbc306275e4a409e028a7ca1dd1602 P 787ddcc824c2490a808fc5a53452dc5e: stopping tablet replica
I20250109 08:58:07.417430 21514 raft_consensus.cc:2463] T 37280454805d47f98a415a8f0c16d869 P 8bf6e26e98b14566899578a990b27d47 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88935ab835544e9889d05a6587d8cf21 in term 3.
I20250109 08:58:07.420398 21513 raft_consensus.cc:513] T 363fa693a5114f2bb22f4ab67722a1c6 P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:58:07.417168 22970 raft_consensus.cc:2238] T 56bbc306275e4a409e028a7ca1dd1602 P 787ddcc824c2490a808fc5a53452dc5e [term 3 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:07.445322 22970 raft_consensus.cc:2267] T 56bbc306275e4a409e028a7ca1dd1602 P 787ddcc824c2490a808fc5a53452dc5e [term 3 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:07.445732 21509 raft_consensus.cc:1270] T 5ac5ffe3cd65463fb448e819e3978841 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250109 08:58:07.446875 21513 leader_election.cc:290] T 363fa693a5114f2bb22f4ab67722a1c6 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 2 election: Requested vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:58:07.444689 21409 catalog_manager.cc:5526] T 37280454805d47f98a415a8f0c16d869 P 88935ab835544e9889d05a6587d8cf21 reported cstate change: term changed from 2 to 3, leader changed from 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195) to 88935ab835544e9889d05a6587d8cf21 (127.19.226.196). New cstate: current_term: 3 leader_uuid: "88935ab835544e9889d05a6587d8cf21" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250109 08:58:07.448837 22970 ts_tablet_manager.cc:1905] T 56bbc306275e4a409e028a7ca1dd1602 P 787ddcc824c2490a808fc5a53452dc5e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250109 08:58:07.447680 22965 consensus_queue.cc:1035] T 5ac5ffe3cd65463fb448e819e3978841 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250109 08:58:07.454468 21584 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "363fa693a5114f2bb22f4ab67722a1c6" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3"
I20250109 08:58:07.455019 21584 raft_consensus.cc:3049] T 363fa693a5114f2bb22f4ab67722a1c6 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 LEADER]: Stepping down as leader of term 1
I20250109 08:58:07.455282 21584 raft_consensus.cc:738] T 363fa693a5114f2bb22f4ab67722a1c6 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a2b1a692435241ee988e0bd1661cd4c3, State: Running, Role: LEADER
I20250109 08:58:07.455940 21584 consensus_queue.cc:260] T 363fa693a5114f2bb22f4ab67722a1c6 P a2b1a692435241ee988e0bd1661cd4c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:58:07.462829 22904 raft_consensus.cc:2949] T 5ac5ffe3cd65463fb448e819e3978841 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:07.464201 21584 raft_consensus.cc:3054] T 363fa693a5114f2bb22f4ab67722a1c6 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:07.466421 22970 ts_tablet_manager.cc:1918] T 56bbc306275e4a409e028a7ca1dd1602 P 787ddcc824c2490a808fc5a53452dc5e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.5
I20250109 08:58:07.466812 22970 log.cc:1198] T 56bbc306275e4a409e028a7ca1dd1602 P 787ddcc824c2490a808fc5a53452dc5e: Deleting WAL directory at /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736412987538469-20363-0/minicluster-data/ts-2-root/wals/56bbc306275e4a409e028a7ca1dd1602
I20250109 08:58:07.469902 21396 catalog_manager.cc:4872] TS 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731): tablet 56bbc306275e4a409e028a7ca1dd1602 (table test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]) successfully deleted
I20250109 08:58:07.425050 22174 raft_consensus.cc:1270] T 5ac5ffe3cd65463fb448e819e3978841 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250109 08:58:07.475190 21509 raft_consensus.cc:2949] T 5ac5ffe3cd65463fb448e819e3978841 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:07.478715 21396 catalog_manager.cc:5039] ChangeConfig:REMOVE_PEER RPC for tablet 5ac5ffe3cd65463fb448e819e3978841 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
W20250109 08:58:07.480010 22969 log.cc:926] Time spent T 37280454805d47f98a415a8f0c16d869 P 88935ab835544e9889d05a6587d8cf21: Append to log took a long time: real 0.058s user 0.000s sys 0.000s
I20250109 08:58:07.483227 21584 raft_consensus.cc:2463] T 363fa693a5114f2bb22f4ab67722a1c6 P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8bf6e26e98b14566899578a990b27d47 in term 2.
I20250109 08:58:07.484396 21478 leader_election.cc:304] T 363fa693a5114f2bb22f4ab67722a1c6 P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3; no voters:
I20250109 08:58:07.485783 22863 raft_consensus.cc:2798] T 363fa693a5114f2bb22f4ab67722a1c6 P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:58:07.486048 21409 catalog_manager.cc:5526] T 5ac5ffe3cd65463fb448e819e3978841 P 787ddcc824c2490a808fc5a53452dc5e reported cstate change: config changed from index 3 to 4, VOTER a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194) evicted. New cstate: current_term: 1 leader_uuid: "787ddcc824c2490a808fc5a53452dc5e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:07.487016 22863 raft_consensus.cc:695] T 363fa693a5114f2bb22f4ab67722a1c6 P 8bf6e26e98b14566899578a990b27d47 [term 2 LEADER]: Becoming Leader. State: Replica: 8bf6e26e98b14566899578a990b27d47, State: Running, Role: LEADER
I20250109 08:58:07.488356 22863 consensus_queue.cc:237] T 363fa693a5114f2bb22f4ab67722a1c6 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:58:07.489017 21656 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "363fa693a5114f2bb22f4ab67722a1c6" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "787ddcc824c2490a808fc5a53452dc5e"
I20250109 08:58:07.489609 21656 raft_consensus.cc:3054] T 363fa693a5114f2bb22f4ab67722a1c6 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:07.491248 22965 consensus_queue.cc:1035] T 5ac5ffe3cd65463fb448e819e3978841 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250109 08:58:07.495800 21656 raft_consensus.cc:2463] T 363fa693a5114f2bb22f4ab67722a1c6 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8bf6e26e98b14566899578a990b27d47 in term 2.
I20250109 08:58:07.497345 22174 raft_consensus.cc:2949] T 5ac5ffe3cd65463fb448e819e3978841 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:07.507910 21570 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ac5ffe3cd65463fb448e819e3978841 with delete_type TABLET_DATA_TOMBSTONED (TS a2b1a692435241ee988e0bd1661cd4c3 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:34732
I20250109 08:58:07.521080 22972 tablet_replica.cc:331] T 5ac5ffe3cd65463fb448e819e3978841 P a2b1a692435241ee988e0bd1661cd4c3: stopping tablet replica
I20250109 08:58:07.520772 21409 catalog_manager.cc:5526] T 363fa693a5114f2bb22f4ab67722a1c6 P 8bf6e26e98b14566899578a990b27d47 reported cstate change: term changed from 1 to 2, leader changed from a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194) to 8bf6e26e98b14566899578a990b27d47 (127.19.226.193). New cstate: current_term: 2 leader_uuid: "8bf6e26e98b14566899578a990b27d47" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:07.522048 22972 raft_consensus.cc:2238] T 5ac5ffe3cd65463fb448e819e3978841 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:07.522689 22972 raft_consensus.cc:2267] T 5ac5ffe3cd65463fb448e819e3978841 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:07.526189 22972 ts_tablet_manager.cc:1905] T 5ac5ffe3cd65463fb448e819e3978841 P a2b1a692435241ee988e0bd1661cd4c3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250109 08:58:07.541169 22972 ts_tablet_manager.cc:1918] T 5ac5ffe3cd65463fb448e819e3978841 P a2b1a692435241ee988e0bd1661cd4c3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20250109 08:58:07.541507 22972 log.cc:1198] T 5ac5ffe3cd65463fb448e819e3978841 P a2b1a692435241ee988e0bd1661cd4c3: Deleting WAL directory at /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736412987538469-20363-0/minicluster-data/ts-1-root/wals/5ac5ffe3cd65463fb448e819e3978841
I20250109 08:58:07.542815 21397 catalog_manager.cc:4872] TS a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): tablet 5ac5ffe3cd65463fb448e819e3978841 (table test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]) successfully deleted
I20250109 08:58:07.673457 22958 raft_consensus.cc:988] T d6f988a231a74f6fa112da9a527d4a0b P 8bf6e26e98b14566899578a990b27d47: : Instructing follower a2b1a692435241ee988e0bd1661cd4c3 to start an election
I20250109 08:58:07.673877 22863 raft_consensus.cc:1076] T d6f988a231a74f6fa112da9a527d4a0b P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Signalling peer a2b1a692435241ee988e0bd1661cd4c3 to start an election
I20250109 08:58:07.674998 21584 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d6f988a231a74f6fa112da9a527d4a0b"
dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3"
from {username='slave'} at 127.0.0.1:34754
I20250109 08:58:07.675485 21584 raft_consensus.cc:491] T d6f988a231a74f6fa112da9a527d4a0b P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:58:07.675740 21584 raft_consensus.cc:3054] T d6f988a231a74f6fa112da9a527d4a0b P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:07.680043 21584 raft_consensus.cc:513] T d6f988a231a74f6fa112da9a527d4a0b P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:07.681521 21584 leader_election.cc:290] T d6f988a231a74f6fa112da9a527d4a0b P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 2 election: Requested vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), 88935ab835544e9889d05a6587d8cf21 (127.19.226.196:44363)
I20250109 08:58:07.682261 21514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6f988a231a74f6fa112da9a527d4a0b" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "8bf6e26e98b14566899578a990b27d47"
I20250109 08:58:07.682608 22174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6f988a231a74f6fa112da9a527d4a0b" candidate_uuid: "a2b1a692435241ee988e0bd1661cd4c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "88935ab835544e9889d05a6587d8cf21"
I20250109 08:58:07.682873 21514 raft_consensus.cc:3049] T d6f988a231a74f6fa112da9a527d4a0b P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Stepping down as leader of term 1
I20250109 08:58:07.683233 21514 raft_consensus.cc:738] T d6f988a231a74f6fa112da9a527d4a0b P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8bf6e26e98b14566899578a990b27d47, State: Running, Role: LEADER
I20250109 08:58:07.683312 22174 raft_consensus.cc:3054] T d6f988a231a74f6fa112da9a527d4a0b P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:07.683893 21514 consensus_queue.cc:260] T d6f988a231a74f6fa112da9a527d4a0b P 8bf6e26e98b14566899578a990b27d47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:07.684859 21514 raft_consensus.cc:3054] T d6f988a231a74f6fa112da9a527d4a0b P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:07.688316 22174 raft_consensus.cc:2463] T d6f988a231a74f6fa112da9a527d4a0b P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2b1a692435241ee988e0bd1661cd4c3 in term 2.
I20250109 08:58:07.689640 21553 leader_election.cc:304] T d6f988a231a74f6fa112da9a527d4a0b P a2b1a692435241ee988e0bd1661cd4c3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88935ab835544e9889d05a6587d8cf21, a2b1a692435241ee988e0bd1661cd4c3; no voters:
I20250109 08:58:07.689973 21514 raft_consensus.cc:2463] T d6f988a231a74f6fa112da9a527d4a0b P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2b1a692435241ee988e0bd1661cd4c3 in term 2.
I20250109 08:58:07.690286 22773 raft_consensus.cc:2798] T d6f988a231a74f6fa112da9a527d4a0b P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:58:07.691473 22773 raft_consensus.cc:695] T d6f988a231a74f6fa112da9a527d4a0b P a2b1a692435241ee988e0bd1661cd4c3 [term 2 LEADER]: Becoming Leader. State: Replica: a2b1a692435241ee988e0bd1661cd4c3, State: Running, Role: LEADER
I20250109 08:58:07.692202 22773 consensus_queue.cc:237] T d6f988a231a74f6fa112da9a527d4a0b P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:07.700484 21409 catalog_manager.cc:5526] T d6f988a231a74f6fa112da9a527d4a0b P a2b1a692435241ee988e0bd1661cd4c3 reported cstate change: term changed from 1 to 2, leader changed from 8bf6e26e98b14566899578a990b27d47 (127.19.226.193) to a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194). New cstate: current_term: 2 leader_uuid: "a2b1a692435241ee988e0bd1661cd4c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:07.882627 21584 raft_consensus.cc:1270] T 363fa693a5114f2bb22f4ab67722a1c6 P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Refusing update from remote peer 8bf6e26e98b14566899578a990b27d47: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:58:07.883586 22958 consensus_queue.cc:1035] T 363fa693a5114f2bb22f4ab67722a1c6 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:58:07.891825 21656 raft_consensus.cc:1270] T 363fa693a5114f2bb22f4ab67722a1c6 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Refusing update from remote peer 8bf6e26e98b14566899578a990b27d47: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:58:07.892990 22870 consensus_queue.cc:1035] T 363fa693a5114f2bb22f4ab67722a1c6 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:58:07.913506 21513 raft_consensus.cc:1270] T 37280454805d47f98a415a8f0c16d869 P 8bf6e26e98b14566899578a990b27d47 [term 3 FOLLOWER]: Refusing update from remote peer 88935ab835544e9889d05a6587d8cf21: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20250109 08:58:07.914469 22787 consensus_queue.cc:1035] T 37280454805d47f98a415a8f0c16d869 P 88935ab835544e9889d05a6587d8cf21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20250109 08:58:07.927446 21656 raft_consensus.cc:1270] T 37280454805d47f98a415a8f0c16d869 P 787ddcc824c2490a808fc5a53452dc5e [term 3 FOLLOWER]: Refusing update from remote peer 88935ab835544e9889d05a6587d8cf21: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20250109 08:58:07.928493 22787 consensus_queue.cc:1035] T 37280454805d47f98a415a8f0c16d869 P 88935ab835544e9889d05a6587d8cf21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20250109 08:58:08.292531 21514 raft_consensus.cc:1270] T d6f988a231a74f6fa112da9a527d4a0b P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Refusing update from remote peer a2b1a692435241ee988e0bd1661cd4c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250109 08:58:08.293643 22900 consensus_queue.cc:1035] T d6f988a231a74f6fa112da9a527d4a0b P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250109 08:58:08.303419 22177 raft_consensus.cc:1270] T d6f988a231a74f6fa112da9a527d4a0b P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Refusing update from remote peer a2b1a692435241ee988e0bd1661cd4c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250109 08:58:08.309037 22900 consensus_queue.cc:1035] T d6f988a231a74f6fa112da9a527d4a0b P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250109 08:58:08.473852 20363 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250109 08:58:08.511811 21514 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "af273bab41cc4a9da780147eebfd52ec"
dest_uuid: "8bf6e26e98b14566899578a990b27d47"
mode: GRACEFUL
new_leader_uuid: "787ddcc824c2490a808fc5a53452dc5e"
from {username='slave'} at 127.0.0.1:39274
I20250109 08:58:08.512705 21514 raft_consensus.cc:604] T af273bab41cc4a9da780147eebfd52ec P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Received request to transfer leadership to TS 787ddcc824c2490a808fc5a53452dc5e
I20250109 08:58:08.515154 21588 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "e03846e461974d0d8944c344e715b86a"
dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3"
mode: GRACEFUL
new_leader_uuid: "8bf6e26e98b14566899578a990b27d47"
from {username='slave'} at 127.0.0.1:46832
I20250109 08:58:08.515573 21588 raft_consensus.cc:604] T e03846e461974d0d8944c344e715b86a P a2b1a692435241ee988e0bd1661cd4c3 [term 1 LEADER]: Received request to transfer leadership to TS 8bf6e26e98b14566899578a990b27d47
I20250109 08:58:08.516520 20363 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 2
I20250109 08:58:08.517580 20363 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250109 08:58:08.518023 20363 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250109 08:58:08.583904 21588 consensus_queue.cc:237] T 0e06021cd6624e50af2c914399ebd6c4 P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:58:08.588649 21656 raft_consensus.cc:1270] T 0e06021cd6624e50af2c914399ebd6c4 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Refusing update from remote peer a2b1a692435241ee988e0bd1661cd4c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250109 08:58:08.590004 21509 raft_consensus.cc:1270] T 0e06021cd6624e50af2c914399ebd6c4 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Refusing update from remote peer a2b1a692435241ee988e0bd1661cd4c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20250109 08:58:08.590636 21553 consensus_peers.cc:487] T 0e06021cd6624e50af2c914399ebd6c4 P a2b1a692435241ee988e0bd1661cd4c3 -> Peer 88935ab835544e9889d05a6587d8cf21 (127.19.226.196:44363): Couldn't send request to peer 88935ab835544e9889d05a6587d8cf21. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0e06021cd6624e50af2c914399ebd6c4. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:08.590855 22773 consensus_queue.cc:1035] T 0e06021cd6624e50af2c914399ebd6c4 P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:58:08.591650 22900 consensus_queue.cc:1035] T 0e06021cd6624e50af2c914399ebd6c4 P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:58:08.598966 22985 raft_consensus.cc:2949] T 0e06021cd6624e50af2c914399ebd6c4 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:58:08.600512 21656 raft_consensus.cc:2949] T 0e06021cd6624e50af2c914399ebd6c4 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:58:08.600512 21509 raft_consensus.cc:2949] T 0e06021cd6624e50af2c914399ebd6c4 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:58:08.618983 21409 catalog_manager.cc:5526] T 0e06021cd6624e50af2c914399ebd6c4 P a2b1a692435241ee988e0bd1661cd4c3 reported cstate change: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New cstate: current_term: 1 leader_uuid: "a2b1a692435241ee988e0bd1661cd4c3" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:08.620121 21588 consensus_queue.cc:237] T 3b5049c00b0b4dd89c49fa5d13db169e P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:58:08.637321 21656 raft_consensus.cc:1270] T 3b5049c00b0b4dd89c49fa5d13db169e P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Refusing update from remote peer a2b1a692435241ee988e0bd1661cd4c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20250109 08:58:08.637815 21553 consensus_peers.cc:487] T 3b5049c00b0b4dd89c49fa5d13db169e P a2b1a692435241ee988e0bd1661cd4c3 -> Peer 88935ab835544e9889d05a6587d8cf21 (127.19.226.196:44363): Couldn't send request to peer 88935ab835544e9889d05a6587d8cf21. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3b5049c00b0b4dd89c49fa5d13db169e. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:08.638768 22900 consensus_queue.cc:1035] T 3b5049c00b0b4dd89c49fa5d13db169e P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:58:08.637321 21509 raft_consensus.cc:1270] T 3b5049c00b0b4dd89c49fa5d13db169e P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Refusing update from remote peer a2b1a692435241ee988e0bd1661cd4c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250109 08:58:08.657255 22900 consensus_queue.cc:1035] T 3b5049c00b0b4dd89c49fa5d13db169e P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20250109 08:58:08.660156 22986 raft_consensus.cc:2949] T 3b5049c00b0b4dd89c49fa5d13db169e P a2b1a692435241ee988e0bd1661cd4c3 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:58:08.663249 21660 raft_consensus.cc:2949] T 3b5049c00b0b4dd89c49fa5d13db169e P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:58:08.668790 21509 raft_consensus.cc:2949] T 3b5049c00b0b4dd89c49fa5d13db169e P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:58:08.680928 21412 catalog_manager.cc:5526] T 3b5049c00b0b4dd89c49fa5d13db169e P a2b1a692435241ee988e0bd1661cd4c3 reported cstate change: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New cstate: current_term: 1 leader_uuid: "a2b1a692435241ee988e0bd1661cd4c3" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:08.685341 21656 consensus_queue.cc:237] T 7c19ed9fdde5463f84e43bea7b860d03 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:58:08.693313 21514 raft_consensus.cc:1270] T 7c19ed9fdde5463f84e43bea7b860d03 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250109 08:58:08.693861 21588 raft_consensus.cc:1270] T 7c19ed9fdde5463f84e43bea7b860d03 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20250109 08:58:08.694496 21627 consensus_peers.cc:487] T 7c19ed9fdde5463f84e43bea7b860d03 P 787ddcc824c2490a808fc5a53452dc5e -> Peer 88935ab835544e9889d05a6587d8cf21 (127.19.226.196:44363): Couldn't send request to peer 88935ab835544e9889d05a6587d8cf21. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7c19ed9fdde5463f84e43bea7b860d03. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:08.695050 22965 consensus_queue.cc:1035] T 7c19ed9fdde5463f84e43bea7b860d03 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:58:08.696554 22965 consensus_queue.cc:1035] T 7c19ed9fdde5463f84e43bea7b860d03 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:58:08.709249 22982 raft_consensus.cc:2949] T 7c19ed9fdde5463f84e43bea7b860d03 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:58:08.711076 21588 raft_consensus.cc:2949] T 7c19ed9fdde5463f84e43bea7b860d03 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:58:08.712127 21514 raft_consensus.cc:2949] T 7c19ed9fdde5463f84e43bea7b860d03 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:58:08.729687 21412 catalog_manager.cc:5526] T 7c19ed9fdde5463f84e43bea7b860d03 P 787ddcc824c2490a808fc5a53452dc5e reported cstate change: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New cstate: current_term: 1 leader_uuid: "787ddcc824c2490a808fc5a53452dc5e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:08.732682 21584 consensus_queue.cc:237] T d355d166de534fb4a50c9f48f2df7469 P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:58:08.749809 21656 raft_consensus.cc:1270] T d355d166de534fb4a50c9f48f2df7469 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Refusing update from remote peer a2b1a692435241ee988e0bd1661cd4c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20250109 08:58:08.750231 21509 raft_consensus.cc:1270] T d355d166de534fb4a50c9f48f2df7469 P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Refusing update from remote peer a2b1a692435241ee988e0bd1661cd4c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
W20250109 08:58:08.750353 21553 consensus_peers.cc:487] T d355d166de534fb4a50c9f48f2df7469 P a2b1a692435241ee988e0bd1661cd4c3 -> Peer 88935ab835544e9889d05a6587d8cf21 (127.19.226.196:44363): Couldn't send request to peer 88935ab835544e9889d05a6587d8cf21. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d355d166de534fb4a50c9f48f2df7469. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:08.751264 22986 consensus_queue.cc:1035] T d355d166de534fb4a50c9f48f2df7469 P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:58:08.753950 22986 consensus_queue.cc:1035] T d355d166de534fb4a50c9f48f2df7469 P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:58:08.757894 22992 raft_consensus.cc:2949] T d355d166de534fb4a50c9f48f2df7469 P a2b1a692435241ee988e0bd1661cd4c3 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:58:08.759287 21656 raft_consensus.cc:2949] T d355d166de534fb4a50c9f48f2df7469 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:58:08.779455 21412 catalog_manager.cc:5526] T d355d166de534fb4a50c9f48f2df7469 P a2b1a692435241ee988e0bd1661cd4c3 reported cstate change: config changed from index -1 to 3, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New cstate: current_term: 2 leader_uuid: "a2b1a692435241ee988e0bd1661cd4c3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:08.795836 21514 raft_consensus.cc:2949] T d355d166de534fb4a50c9f48f2df7469 P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:58:08.886073 22870 raft_consensus.cc:988] T af273bab41cc4a9da780147eebfd52ec P 8bf6e26e98b14566899578a990b27d47: : Instructing follower 787ddcc824c2490a808fc5a53452dc5e to start an election
I20250109 08:58:08.887070 22870 raft_consensus.cc:1076] T af273bab41cc4a9da780147eebfd52ec P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Signalling peer 787ddcc824c2490a808fc5a53452dc5e to start an election
I20250109 08:58:08.902191 21658 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "af273bab41cc4a9da780147eebfd52ec"
dest_uuid: "787ddcc824c2490a808fc5a53452dc5e"
from {username='slave'} at 127.0.0.1:58440
I20250109 08:58:08.910678 21658 raft_consensus.cc:491] T af273bab41cc4a9da780147eebfd52ec P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:58:08.911338 21658 raft_consensus.cc:3054] T af273bab41cc4a9da780147eebfd52ec P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:08.923936 22900 raft_consensus.cc:988] T e03846e461974d0d8944c344e715b86a P a2b1a692435241ee988e0bd1661cd4c3: : Instructing follower 8bf6e26e98b14566899578a990b27d47 to start an election
I20250109 08:58:08.925765 22900 raft_consensus.cc:1076] T e03846e461974d0d8944c344e715b86a P a2b1a692435241ee988e0bd1661cd4c3 [term 1 LEADER]: Signalling peer 8bf6e26e98b14566899578a990b27d47 to start an election
I20250109 08:58:08.927901 21513 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "e03846e461974d0d8944c344e715b86a"
dest_uuid: "8bf6e26e98b14566899578a990b27d47"
from {username='slave'} at 127.0.0.1:39250
I20250109 08:58:08.928509 21513 raft_consensus.cc:491] T e03846e461974d0d8944c344e715b86a P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:58:08.928866 21513 raft_consensus.cc:3054] T e03846e461974d0d8944c344e715b86a P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:08.927356 21658 raft_consensus.cc:513] T af273bab41cc4a9da780147eebfd52ec P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:58:08.937865 21658 leader_election.cc:290] T af273bab41cc4a9da780147eebfd52ec P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 2 election: Requested vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
I20250109 08:58:08.938925 21509 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af273bab41cc4a9da780147eebfd52ec" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8bf6e26e98b14566899578a990b27d47"
I20250109 08:58:08.939800 21509 raft_consensus.cc:3049] T af273bab41cc4a9da780147eebfd52ec P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Stepping down as leader of term 1
I20250109 08:58:08.940117 21509 raft_consensus.cc:738] T af273bab41cc4a9da780147eebfd52ec P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8bf6e26e98b14566899578a990b27d47, State: Running, Role: LEADER
I20250109 08:58:08.941020 21588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af273bab41cc4a9da780147eebfd52ec" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3"
I20250109 08:58:08.941007 21509 consensus_queue.cc:260] T af273bab41cc4a9da780147eebfd52ec P 8bf6e26e98b14566899578a990b27d47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:58:08.941870 21588 raft_consensus.cc:3054] T af273bab41cc4a9da780147eebfd52ec P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:08.942625 21509 raft_consensus.cc:3054] T af273bab41cc4a9da780147eebfd52ec P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:08.952155 21509 raft_consensus.cc:2463] T af273bab41cc4a9da780147eebfd52ec P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 2.
I20250109 08:58:08.953708 21626 leader_election.cc:304] T af273bab41cc4a9da780147eebfd52ec P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, 8bf6e26e98b14566899578a990b27d47; no voters:
I20250109 08:58:08.956022 23002 raft_consensus.cc:2798] T af273bab41cc4a9da780147eebfd52ec P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:58:08.956640 23002 raft_consensus.cc:695] T af273bab41cc4a9da780147eebfd52ec P 787ddcc824c2490a808fc5a53452dc5e [term 2 LEADER]: Becoming Leader. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Running, Role: LEADER
I20250109 08:58:08.957352 21513 raft_consensus.cc:513] T e03846e461974d0d8944c344e715b86a P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:58:08.957530 23002 consensus_queue.cc:237] T af273bab41cc4a9da780147eebfd52ec P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:58:08.959316 21513 leader_election.cc:290] T e03846e461974d0d8944c344e715b86a P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 2 election: Requested vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:58:08.960139 21588 raft_consensus.cc:2463] T af273bab41cc4a9da780147eebfd52ec P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 2.
I20250109 08:58:08.960311 21584 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e03846e461974d0d8944c344e715b86a" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a2b1a692435241ee988e0bd1661cd4c3"
I20250109 08:58:08.961021 21584 raft_consensus.cc:3049] T e03846e461974d0d8944c344e715b86a P a2b1a692435241ee988e0bd1661cd4c3 [term 1 LEADER]: Stepping down as leader of term 1
I20250109 08:58:08.961673 21584 raft_consensus.cc:738] T e03846e461974d0d8944c344e715b86a P a2b1a692435241ee988e0bd1661cd4c3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a2b1a692435241ee988e0bd1661cd4c3, State: Running, Role: LEADER
I20250109 08:58:08.962536 21584 consensus_queue.cc:260] T e03846e461974d0d8944c344e715b86a P a2b1a692435241ee988e0bd1661cd4c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:58:08.964038 21584 raft_consensus.cc:3054] T e03846e461974d0d8944c344e715b86a P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:08.965063 21660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e03846e461974d0d8944c344e715b86a" candidate_uuid: "8bf6e26e98b14566899578a990b27d47" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "787ddcc824c2490a808fc5a53452dc5e"
I20250109 08:58:08.965796 21660 raft_consensus.cc:3054] T e03846e461974d0d8944c344e715b86a P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:08.974184 21584 raft_consensus.cc:2463] T e03846e461974d0d8944c344e715b86a P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8bf6e26e98b14566899578a990b27d47 in term 2.
I20250109 08:58:08.975585 21478 leader_election.cc:304] T e03846e461974d0d8944c344e715b86a P 8bf6e26e98b14566899578a990b27d47 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3; no voters:
I20250109 08:58:08.980489 21660 raft_consensus.cc:2463] T e03846e461974d0d8944c344e715b86a P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8bf6e26e98b14566899578a990b27d47 in term 2.
I20250109 08:58:08.980798 23007 raft_consensus.cc:2798] T e03846e461974d0d8944c344e715b86a P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:58:08.981426 23007 raft_consensus.cc:695] T e03846e461974d0d8944c344e715b86a P 8bf6e26e98b14566899578a990b27d47 [term 2 LEADER]: Becoming Leader. State: Replica: 8bf6e26e98b14566899578a990b27d47, State: Running, Role: LEADER
I20250109 08:58:08.982499 23007 consensus_queue.cc:237] T e03846e461974d0d8944c344e715b86a P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:58:09.012081 21412 catalog_manager.cc:5526] T e03846e461974d0d8944c344e715b86a P 8bf6e26e98b14566899578a990b27d47 reported cstate change: term changed from 1 to 2, leader changed from a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194) to 8bf6e26e98b14566899578a990b27d47 (127.19.226.193). New cstate: current_term: 2 leader_uuid: "8bf6e26e98b14566899578a990b27d47" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:09.027504 21409 catalog_manager.cc:5526] T af273bab41cc4a9da780147eebfd52ec P 787ddcc824c2490a808fc5a53452dc5e reported cstate change: term changed from 1 to 2, leader changed from 8bf6e26e98b14566899578a990b27d47 (127.19.226.193) to 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195). New cstate: current_term: 2 leader_uuid: "787ddcc824c2490a808fc5a53452dc5e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: HEALTHY } } }
I20250109 08:58:09.052347 23011 ts_tablet_manager.cc:927] T 0e06021cd6624e50af2c914399ebd6c4 P 88935ab835544e9889d05a6587d8cf21: Initiating tablet copy from peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
I20250109 08:58:09.059576 23012 ts_tablet_manager.cc:927] T 3b5049c00b0b4dd89c49fa5d13db169e P 88935ab835544e9889d05a6587d8cf21: Initiating tablet copy from peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
I20250109 08:58:09.061131 23011 tablet_copy_client.cc:323] T 0e06021cd6624e50af2c914399ebd6c4 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Beginning tablet copy session from remote peer at address 127.19.226.194:45527
I20250109 08:58:09.066630 21599 tablet_copy_service.cc:140] P a2b1a692435241ee988e0bd1661cd4c3: Received BeginTabletCopySession request for tablet 0e06021cd6624e50af2c914399ebd6c4 from peer 88935ab835544e9889d05a6587d8cf21 ({username='slave'} at 127.0.0.1:33652)
I20250109 08:58:09.067245 21599 tablet_copy_service.cc:161] P a2b1a692435241ee988e0bd1661cd4c3: Beginning new tablet copy session on tablet 0e06021cd6624e50af2c914399ebd6c4 from peer 88935ab835544e9889d05a6587d8cf21 at {username='slave'} at 127.0.0.1:33652: session id = 88935ab835544e9889d05a6587d8cf21-0e06021cd6624e50af2c914399ebd6c4
I20250109 08:58:09.073055 23012 tablet_copy_client.cc:323] T 3b5049c00b0b4dd89c49fa5d13db169e P 88935ab835544e9889d05a6587d8cf21: tablet copy: Beginning tablet copy session from remote peer at address 127.19.226.194:45527
I20250109 08:58:09.075409 21598 tablet_copy_service.cc:140] P a2b1a692435241ee988e0bd1661cd4c3: Received BeginTabletCopySession request for tablet 3b5049c00b0b4dd89c49fa5d13db169e from peer 88935ab835544e9889d05a6587d8cf21 ({username='slave'} at 127.0.0.1:33652)
I20250109 08:58:09.075981 21598 tablet_copy_service.cc:161] P a2b1a692435241ee988e0bd1661cd4c3: Beginning new tablet copy session on tablet 3b5049c00b0b4dd89c49fa5d13db169e from peer 88935ab835544e9889d05a6587d8cf21 at {username='slave'} at 127.0.0.1:33652: session id = 88935ab835544e9889d05a6587d8cf21-3b5049c00b0b4dd89c49fa5d13db169e
I20250109 08:58:09.083865 21598 tablet_copy_source_session.cc:215] T 3b5049c00b0b4dd89c49fa5d13db169e P a2b1a692435241ee988e0bd1661cd4c3: Tablet Copy: opened 0 blocks and 1 log segments
I20250109 08:58:09.085976 21599 tablet_copy_source_session.cc:215] T 0e06021cd6624e50af2c914399ebd6c4 P a2b1a692435241ee988e0bd1661cd4c3: Tablet Copy: opened 0 blocks and 1 log segments
I20250109 08:58:09.087337 23012 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b5049c00b0b4dd89c49fa5d13db169e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:09.088935 23011 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e06021cd6624e50af2c914399ebd6c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:09.104882 23011 tablet_copy_client.cc:806] T 0e06021cd6624e50af2c914399ebd6c4 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Starting download of 0 data blocks...
I20250109 08:58:09.105551 23011 tablet_copy_client.cc:670] T 0e06021cd6624e50af2c914399ebd6c4 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Starting download of 1 WAL segments...
I20250109 08:58:09.105613 23012 tablet_copy_client.cc:806] T 3b5049c00b0b4dd89c49fa5d13db169e P 88935ab835544e9889d05a6587d8cf21: tablet copy: Starting download of 0 data blocks...
I20250109 08:58:09.106362 23012 tablet_copy_client.cc:670] T 3b5049c00b0b4dd89c49fa5d13db169e P 88935ab835544e9889d05a6587d8cf21: tablet copy: Starting download of 1 WAL segments...
I20250109 08:58:09.110445 23011 tablet_copy_client.cc:538] T 0e06021cd6624e50af2c914399ebd6c4 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250109 08:58:09.114976 23012 tablet_copy_client.cc:538] T 3b5049c00b0b4dd89c49fa5d13db169e P 88935ab835544e9889d05a6587d8cf21: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250109 08:58:09.119683 23011 tablet_bootstrap.cc:492] T 0e06021cd6624e50af2c914399ebd6c4 P 88935ab835544e9889d05a6587d8cf21: Bootstrap starting.
I20250109 08:58:09.125205 23012 tablet_bootstrap.cc:492] T 3b5049c00b0b4dd89c49fa5d13db169e P 88935ab835544e9889d05a6587d8cf21: Bootstrap starting.
I20250109 08:58:09.148528 23012 tablet_bootstrap.cc:492] T 3b5049c00b0b4dd89c49fa5d13db169e P 88935ab835544e9889d05a6587d8cf21: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250109 08:58:09.149323 23011 tablet_bootstrap.cc:492] T 0e06021cd6624e50af2c914399ebd6c4 P 88935ab835544e9889d05a6587d8cf21: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250109 08:58:09.149418 23012 tablet_bootstrap.cc:492] T 3b5049c00b0b4dd89c49fa5d13db169e P 88935ab835544e9889d05a6587d8cf21: Bootstrap complete.
I20250109 08:58:09.150255 23011 tablet_bootstrap.cc:492] T 0e06021cd6624e50af2c914399ebd6c4 P 88935ab835544e9889d05a6587d8cf21: Bootstrap complete.
I20250109 08:58:09.151032 23011 ts_tablet_manager.cc:1397] T 0e06021cd6624e50af2c914399ebd6c4 P 88935ab835544e9889d05a6587d8cf21: Time spent bootstrapping tablet: real 0.032s user 0.024s sys 0.008s
I20250109 08:58:09.150231 23012 ts_tablet_manager.cc:1397] T 3b5049c00b0b4dd89c49fa5d13db169e P 88935ab835544e9889d05a6587d8cf21: Time spent bootstrapping tablet: real 0.025s user 0.015s sys 0.006s
I20250109 08:58:09.154026 23011 raft_consensus.cc:357] T 0e06021cd6624e50af2c914399ebd6c4 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:58:09.154968 23011 raft_consensus.cc:738] T 0e06021cd6624e50af2c914399ebd6c4 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 88935ab835544e9889d05a6587d8cf21, State: Initialized, Role: LEARNER
I20250109 08:58:09.155587 23011 consensus_queue.cc:260] T 0e06021cd6624e50af2c914399ebd6c4 P 88935ab835544e9889d05a6587d8cf21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:58:09.154570 23012 raft_consensus.cc:357] T 3b5049c00b0b4dd89c49fa5d13db169e P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:58:09.156440 23012 raft_consensus.cc:738] T 3b5049c00b0b4dd89c49fa5d13db169e P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 88935ab835544e9889d05a6587d8cf21, State: Initialized, Role: LEARNER
I20250109 08:58:09.157730 23011 ts_tablet_manager.cc:1428] T 0e06021cd6624e50af2c914399ebd6c4 P 88935ab835544e9889d05a6587d8cf21: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20250109 08:58:09.157099 23012 consensus_queue.cc:260] T 3b5049c00b0b4dd89c49fa5d13db169e P 88935ab835544e9889d05a6587d8cf21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:58:09.159927 23012 ts_tablet_manager.cc:1428] T 3b5049c00b0b4dd89c49fa5d13db169e P 88935ab835544e9889d05a6587d8cf21: Time spent starting tablet: real 0.008s user 0.004s sys 0.000s
I20250109 08:58:09.160353 21598 tablet_copy_service.cc:342] P a2b1a692435241ee988e0bd1661cd4c3: Request end of tablet copy session 88935ab835544e9889d05a6587d8cf21-0e06021cd6624e50af2c914399ebd6c4 received from {username='slave'} at 127.0.0.1:33652
I20250109 08:58:09.160859 21598 tablet_copy_service.cc:434] P a2b1a692435241ee988e0bd1661cd4c3: ending tablet copy session 88935ab835544e9889d05a6587d8cf21-0e06021cd6624e50af2c914399ebd6c4 on tablet 0e06021cd6624e50af2c914399ebd6c4 with peer 88935ab835544e9889d05a6587d8cf21
I20250109 08:58:09.161471 21599 tablet_copy_service.cc:342] P a2b1a692435241ee988e0bd1661cd4c3: Request end of tablet copy session 88935ab835544e9889d05a6587d8cf21-3b5049c00b0b4dd89c49fa5d13db169e received from {username='slave'} at 127.0.0.1:33652
I20250109 08:58:09.161882 21599 tablet_copy_service.cc:434] P a2b1a692435241ee988e0bd1661cd4c3: ending tablet copy session 88935ab835544e9889d05a6587d8cf21-3b5049c00b0b4dd89c49fa5d13db169e on tablet 3b5049c00b0b4dd89c49fa5d13db169e with peer 88935ab835544e9889d05a6587d8cf21
I20250109 08:58:09.246050 23012 ts_tablet_manager.cc:927] T d355d166de534fb4a50c9f48f2df7469 P 88935ab835544e9889d05a6587d8cf21: Initiating tablet copy from peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
I20250109 08:58:09.247642 23012 tablet_copy_client.cc:323] T d355d166de534fb4a50c9f48f2df7469 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Beginning tablet copy session from remote peer at address 127.19.226.194:45527
I20250109 08:58:09.248838 21599 tablet_copy_service.cc:140] P a2b1a692435241ee988e0bd1661cd4c3: Received BeginTabletCopySession request for tablet d355d166de534fb4a50c9f48f2df7469 from peer 88935ab835544e9889d05a6587d8cf21 ({username='slave'} at 127.0.0.1:33652)
I20250109 08:58:09.249346 21599 tablet_copy_service.cc:161] P a2b1a692435241ee988e0bd1661cd4c3: Beginning new tablet copy session on tablet d355d166de534fb4a50c9f48f2df7469 from peer 88935ab835544e9889d05a6587d8cf21 at {username='slave'} at 127.0.0.1:33652: session id = 88935ab835544e9889d05a6587d8cf21-d355d166de534fb4a50c9f48f2df7469
I20250109 08:58:09.255928 21599 tablet_copy_source_session.cc:215] T d355d166de534fb4a50c9f48f2df7469 P a2b1a692435241ee988e0bd1661cd4c3: Tablet Copy: opened 0 blocks and 1 log segments
I20250109 08:58:09.258358 23012 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d355d166de534fb4a50c9f48f2df7469. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:09.271883 23012 tablet_copy_client.cc:806] T d355d166de534fb4a50c9f48f2df7469 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Starting download of 0 data blocks...
I20250109 08:58:09.272452 23012 tablet_copy_client.cc:670] T d355d166de534fb4a50c9f48f2df7469 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Starting download of 1 WAL segments...
I20250109 08:58:09.275586 23012 tablet_copy_client.cc:538] T d355d166de534fb4a50c9f48f2df7469 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250109 08:58:09.286511 23012 tablet_bootstrap.cc:492] T d355d166de534fb4a50c9f48f2df7469 P 88935ab835544e9889d05a6587d8cf21: Bootstrap starting.
I20250109 08:58:09.309022 23011 ts_tablet_manager.cc:927] T 7c19ed9fdde5463f84e43bea7b860d03 P 88935ab835544e9889d05a6587d8cf21: Initiating tablet copy from peer 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:58:09.310575 23011 tablet_copy_client.cc:323] T 7c19ed9fdde5463f84e43bea7b860d03 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Beginning tablet copy session from remote peer at address 127.19.226.195:35731
I20250109 08:58:09.312665 23012 tablet_bootstrap.cc:492] T d355d166de534fb4a50c9f48f2df7469 P 88935ab835544e9889d05a6587d8cf21: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250109 08:58:09.312804 21673 tablet_copy_service.cc:140] P 787ddcc824c2490a808fc5a53452dc5e: Received BeginTabletCopySession request for tablet 7c19ed9fdde5463f84e43bea7b860d03 from peer 88935ab835544e9889d05a6587d8cf21 ({username='slave'} at 127.0.0.1:52376)
I20250109 08:58:09.313292 21673 tablet_copy_service.cc:161] P 787ddcc824c2490a808fc5a53452dc5e: Beginning new tablet copy session on tablet 7c19ed9fdde5463f84e43bea7b860d03 from peer 88935ab835544e9889d05a6587d8cf21 at {username='slave'} at 127.0.0.1:52376: session id = 88935ab835544e9889d05a6587d8cf21-7c19ed9fdde5463f84e43bea7b860d03
I20250109 08:58:09.313603 23012 tablet_bootstrap.cc:492] T d355d166de534fb4a50c9f48f2df7469 P 88935ab835544e9889d05a6587d8cf21: Bootstrap complete.
I20250109 08:58:09.314308 23012 ts_tablet_manager.cc:1397] T d355d166de534fb4a50c9f48f2df7469 P 88935ab835544e9889d05a6587d8cf21: Time spent bootstrapping tablet: real 0.028s user 0.013s sys 0.007s
I20250109 08:58:09.317080 23012 raft_consensus.cc:357] T d355d166de534fb4a50c9f48f2df7469 P 88935ab835544e9889d05a6587d8cf21 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:58:09.317921 23012 raft_consensus.cc:738] T d355d166de534fb4a50c9f48f2df7469 P 88935ab835544e9889d05a6587d8cf21 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 88935ab835544e9889d05a6587d8cf21, State: Initialized, Role: LEARNER
I20250109 08:58:09.318542 23012 consensus_queue.cc:260] T d355d166de534fb4a50c9f48f2df7469 P 88935ab835544e9889d05a6587d8cf21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:58:09.324736 21673 tablet_copy_source_session.cc:215] T 7c19ed9fdde5463f84e43bea7b860d03 P 787ddcc824c2490a808fc5a53452dc5e: Tablet Copy: opened 0 blocks and 1 log segments
I20250109 08:58:09.325222 23012 ts_tablet_manager.cc:1428] T d355d166de534fb4a50c9f48f2df7469 P 88935ab835544e9889d05a6587d8cf21: Time spent starting tablet: real 0.011s user 0.008s sys 0.001s
I20250109 08:58:09.326987 21599 tablet_copy_service.cc:342] P a2b1a692435241ee988e0bd1661cd4c3: Request end of tablet copy session 88935ab835544e9889d05a6587d8cf21-d355d166de534fb4a50c9f48f2df7469 received from {username='slave'} at 127.0.0.1:33652
I20250109 08:58:09.327373 21599 tablet_copy_service.cc:434] P a2b1a692435241ee988e0bd1661cd4c3: ending tablet copy session 88935ab835544e9889d05a6587d8cf21-d355d166de534fb4a50c9f48f2df7469 on tablet d355d166de534fb4a50c9f48f2df7469 with peer 88935ab835544e9889d05a6587d8cf21
I20250109 08:58:09.327494 23011 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c19ed9fdde5463f84e43bea7b860d03. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:09.341058 23011 tablet_copy_client.cc:806] T 7c19ed9fdde5463f84e43bea7b860d03 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Starting download of 0 data blocks...
I20250109 08:58:09.341557 23011 tablet_copy_client.cc:670] T 7c19ed9fdde5463f84e43bea7b860d03 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Starting download of 1 WAL segments...
I20250109 08:58:09.344553 23011 tablet_copy_client.cc:538] T 7c19ed9fdde5463f84e43bea7b860d03 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250109 08:58:09.351015 23011 tablet_bootstrap.cc:492] T 7c19ed9fdde5463f84e43bea7b860d03 P 88935ab835544e9889d05a6587d8cf21: Bootstrap starting.
I20250109 08:58:09.366837 23011 tablet_bootstrap.cc:492] T 7c19ed9fdde5463f84e43bea7b860d03 P 88935ab835544e9889d05a6587d8cf21: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250109 08:58:09.367583 23011 tablet_bootstrap.cc:492] T 7c19ed9fdde5463f84e43bea7b860d03 P 88935ab835544e9889d05a6587d8cf21: Bootstrap complete.
I20250109 08:58:09.368163 23011 ts_tablet_manager.cc:1397] T 7c19ed9fdde5463f84e43bea7b860d03 P 88935ab835544e9889d05a6587d8cf21: Time spent bootstrapping tablet: real 0.017s user 0.015s sys 0.000s
I20250109 08:58:09.371124 23011 raft_consensus.cc:357] T 7c19ed9fdde5463f84e43bea7b860d03 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:58:09.371975 23011 raft_consensus.cc:738] T 7c19ed9fdde5463f84e43bea7b860d03 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 88935ab835544e9889d05a6587d8cf21, State: Initialized, Role: LEARNER
I20250109 08:58:09.372503 23011 consensus_queue.cc:260] T 7c19ed9fdde5463f84e43bea7b860d03 P 88935ab835544e9889d05a6587d8cf21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:58:09.374729 23011 ts_tablet_manager.cc:1428] T 7c19ed9fdde5463f84e43bea7b860d03 P 88935ab835544e9889d05a6587d8cf21: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20250109 08:58:09.377561 21673 tablet_copy_service.cc:342] P 787ddcc824c2490a808fc5a53452dc5e: Request end of tablet copy session 88935ab835544e9889d05a6587d8cf21-7c19ed9fdde5463f84e43bea7b860d03 received from {username='slave'} at 127.0.0.1:52376
I20250109 08:58:09.377949 21673 tablet_copy_service.cc:434] P 787ddcc824c2490a808fc5a53452dc5e: ending tablet copy session 88935ab835544e9889d05a6587d8cf21-7c19ed9fdde5463f84e43bea7b860d03 on tablet 7c19ed9fdde5463f84e43bea7b860d03 with peer 88935ab835544e9889d05a6587d8cf21
I20250109 08:58:09.401569 21509 raft_consensus.cc:1270] T af273bab41cc4a9da780147eebfd52ec P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:58:09.402840 22965 consensus_queue.cc:1035] T af273bab41cc4a9da780147eebfd52ec P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:58:09.431552 21584 raft_consensus.cc:1270] T af273bab41cc4a9da780147eebfd52ec P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:58:09.434769 22999 consensus_queue.cc:1035] T af273bab41cc4a9da780147eebfd52ec P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:58:09.452879 22176 raft_consensus.cc:1212] T 3b5049c00b0b4dd89c49fa5d13db169e P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20250109 08:58:09.563737 21657 raft_consensus.cc:1270] T e03846e461974d0d8944c344e715b86a P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Refusing update from remote peer 8bf6e26e98b14566899578a990b27d47: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:58:09.565030 22863 consensus_queue.cc:1035] T e03846e461974d0d8944c344e715b86a P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:58:09.574945 20363 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250109 08:58:09.582870 21583 raft_consensus.cc:1270] T e03846e461974d0d8944c344e715b86a P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Refusing update from remote peer 8bf6e26e98b14566899578a990b27d47: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:58:09.584321 23029 consensus_queue.cc:1035] T e03846e461974d0d8944c344e715b86a P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20250109 08:58:09.761915 22176 raft_consensus.cc:1212] T 7c19ed9fdde5463f84e43bea7b860d03 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20250109 08:58:10.158905 22178 raft_consensus.cc:1212] T d355d166de534fb4a50c9f48f2df7469 P 88935ab835544e9889d05a6587d8cf21 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3] Dedup: 2.3->[]
I20250109 08:58:10.162575 22174 raft_consensus.cc:1212] T 0e06021cd6624e50af2c914399ebd6c4 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
W20250109 08:58:10.250654 21552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.226.195:35731, user_credentials={real_user=slave}} blocked reactor thread for 95241.2us
I20250109 08:58:10.334987 23044 raft_consensus.cc:1059] T 3b5049c00b0b4dd89c49fa5d13db169e P a2b1a692435241ee988e0bd1661cd4c3: attempting to promote NON_VOTER 88935ab835544e9889d05a6587d8cf21 to VOTER
I20250109 08:58:10.338476 23044 consensus_queue.cc:237] T 3b5049c00b0b4dd89c49fa5d13db169e P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:10.425901 21509 raft_consensus.cc:1270] T 3b5049c00b0b4dd89c49fa5d13db169e P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Refusing update from remote peer a2b1a692435241ee988e0bd1661cd4c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:58:10.427381 23044 consensus_queue.cc:1035] T 3b5049c00b0b4dd89c49fa5d13db169e P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:58:10.435204 23054 raft_consensus.cc:1059] T 7c19ed9fdde5463f84e43bea7b860d03 P 787ddcc824c2490a808fc5a53452dc5e: attempting to promote NON_VOTER 88935ab835544e9889d05a6587d8cf21 to VOTER
I20250109 08:58:10.436626 22177 raft_consensus.cc:1270] T 3b5049c00b0b4dd89c49fa5d13db169e P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Refusing update from remote peer a2b1a692435241ee988e0bd1661cd4c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:58:10.448516 23054 consensus_queue.cc:237] T 7c19ed9fdde5463f84e43bea7b860d03 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:10.460471 21660 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "1b582ab8d8f044d9b9038a28e958feeb"
dest_uuid: "787ddcc824c2490a808fc5a53452dc5e"
mode: GRACEFUL
new_leader_uuid: "88935ab835544e9889d05a6587d8cf21"
from {username='slave'} at 127.0.0.1:55054
I20250109 08:58:10.461241 21660 raft_consensus.cc:604] T 1b582ab8d8f044d9b9038a28e958feeb P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Received request to transfer leadership to TS 88935ab835544e9889d05a6587d8cf21
I20250109 08:58:10.461726 21655 raft_consensus.cc:1270] T 3b5049c00b0b4dd89c49fa5d13db169e P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Refusing update from remote peer a2b1a692435241ee988e0bd1661cd4c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:58:10.468106 21510 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "206785f29c824cddbe9bbc00d365ffb3"
dest_uuid: "8bf6e26e98b14566899578a990b27d47"
mode: GRACEFUL
new_leader_uuid: "787ddcc824c2490a808fc5a53452dc5e"
from {username='slave'} at 127.0.0.1:39274
I20250109 08:58:10.468436 23040 consensus_queue.cc:1035] T 3b5049c00b0b4dd89c49fa5d13db169e P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20250109 08:58:10.468912 21510 raft_consensus.cc:604] T 206785f29c824cddbe9bbc00d365ffb3 P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Received request to transfer leadership to TS 787ddcc824c2490a808fc5a53452dc5e
I20250109 08:58:10.476838 20363 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 2
I20250109 08:58:10.478597 20363 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250109 08:58:10.479338 20363 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250109 08:58:10.487295 23044 consensus_queue.cc:1035] T 3b5049c00b0b4dd89c49fa5d13db169e P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20250109 08:58:10.501576 21589 raft_consensus.cc:1270] T 7c19ed9fdde5463f84e43bea7b860d03 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:58:10.501852 22177 raft_consensus.cc:1270] T 7c19ed9fdde5463f84e43bea7b860d03 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:58:10.503649 21509 raft_consensus.cc:1270] T 7c19ed9fdde5463f84e43bea7b860d03 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:58:10.505798 23050 consensus_queue.cc:1035] T 7c19ed9fdde5463f84e43bea7b860d03 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:58:10.510421 22965 consensus_queue.cc:1035] T 7c19ed9fdde5463f84e43bea7b860d03 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:58:10.512272 22965 consensus_queue.cc:1035] T 7c19ed9fdde5463f84e43bea7b860d03 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20250109 08:58:10.526640 23042 raft_consensus.cc:2949] T 3b5049c00b0b4dd89c49fa5d13db169e P a2b1a692435241ee988e0bd1661cd4c3 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:10.528204 21660 raft_consensus.cc:2949] T 3b5049c00b0b4dd89c49fa5d13db169e P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:10.538082 21509 raft_consensus.cc:2949] T 3b5049c00b0b4dd89c49fa5d13db169e P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:10.546011 21412 catalog_manager.cc:5526] T 3b5049c00b0b4dd89c49fa5d13db169e P a2b1a692435241ee988e0bd1661cd4c3 reported cstate change: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a2b1a692435241ee988e0bd1661cd4c3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:10.578415 23042 raft_consensus.cc:1059] T 0e06021cd6624e50af2c914399ebd6c4 P a2b1a692435241ee988e0bd1661cd4c3: attempting to promote NON_VOTER 88935ab835544e9889d05a6587d8cf21 to VOTER
I20250109 08:58:10.580897 23042 consensus_queue.cc:237] T 0e06021cd6624e50af2c914399ebd6c4 P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:10.582963 23050 raft_consensus.cc:2949] T 7c19ed9fdde5463f84e43bea7b860d03 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:10.585018 21589 raft_consensus.cc:2949] T 7c19ed9fdde5463f84e43bea7b860d03 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:10.593894 22177 raft_consensus.cc:2949] T 7c19ed9fdde5463f84e43bea7b860d03 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:10.616617 21509 raft_consensus.cc:2949] T 7c19ed9fdde5463f84e43bea7b860d03 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:10.617698 22173 raft_consensus.cc:2949] T 3b5049c00b0b4dd89c49fa5d13db169e P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:10.622354 21660 raft_consensus.cc:1270] T 0e06021cd6624e50af2c914399ebd6c4 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Refusing update from remote peer a2b1a692435241ee988e0bd1661cd4c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:58:10.622759 22174 raft_consensus.cc:1270] T 0e06021cd6624e50af2c914399ebd6c4 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Refusing update from remote peer a2b1a692435241ee988e0bd1661cd4c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:58:10.625528 23042 consensus_queue.cc:1035] T 0e06021cd6624e50af2c914399ebd6c4 P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20250109 08:58:10.626516 23036 consensus_queue.cc:1035] T 0e06021cd6624e50af2c914399ebd6c4 P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:58:10.636610 21509 raft_consensus.cc:1270] T 0e06021cd6624e50af2c914399ebd6c4 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Refusing update from remote peer a2b1a692435241ee988e0bd1661cd4c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250109 08:58:10.641960 22965 raft_consensus.cc:988] T 1b582ab8d8f044d9b9038a28e958feeb P 787ddcc824c2490a808fc5a53452dc5e: : Instructing follower 88935ab835544e9889d05a6587d8cf21 to start an election
I20250109 08:58:10.642452 22965 raft_consensus.cc:1076] T 1b582ab8d8f044d9b9038a28e958feeb P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Signalling peer 88935ab835544e9889d05a6587d8cf21 to start an election
I20250109 08:58:10.644068 23043 consensus_queue.cc:1035] T 0e06021cd6624e50af2c914399ebd6c4 P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20250109 08:58:10.652664 22178 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "1b582ab8d8f044d9b9038a28e958feeb"
dest_uuid: "88935ab835544e9889d05a6587d8cf21"
from {username='slave'} at 127.0.0.1:60458
I20250109 08:58:10.653184 22178 raft_consensus.cc:491] T 1b582ab8d8f044d9b9038a28e958feeb P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:58:10.653466 22178 raft_consensus.cc:3054] T 1b582ab8d8f044d9b9038a28e958feeb P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:10.670871 21410 catalog_manager.cc:5526] T 7c19ed9fdde5463f84e43bea7b860d03 P 88935ab835544e9889d05a6587d8cf21 reported cstate change: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "787ddcc824c2490a808fc5a53452dc5e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:10.686017 23041 raft_consensus.cc:2949] T 0e06021cd6624e50af2c914399ebd6c4 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:10.688951 21660 raft_consensus.cc:2949] T 0e06021cd6624e50af2c914399ebd6c4 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:10.691699 22178 raft_consensus.cc:513] T 1b582ab8d8f044d9b9038a28e958feeb P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:10.694324 21514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b582ab8d8f044d9b9038a28e958feeb" candidate_uuid: "88935ab835544e9889d05a6587d8cf21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "8bf6e26e98b14566899578a990b27d47"
I20250109 08:58:10.694897 21514 raft_consensus.cc:3054] T 1b582ab8d8f044d9b9038a28e958feeb P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:10.700520 22178 leader_election.cc:290] T 1b582ab8d8f044d9b9038a28e958feeb P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 election: Requested vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:58:10.702843 21660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b582ab8d8f044d9b9038a28e958feeb" candidate_uuid: "88935ab835544e9889d05a6587d8cf21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "787ddcc824c2490a808fc5a53452dc5e"
I20250109 08:58:10.703595 21660 raft_consensus.cc:3049] T 1b582ab8d8f044d9b9038a28e958feeb P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Stepping down as leader of term 1
I20250109 08:58:10.703948 21660 raft_consensus.cc:738] T 1b582ab8d8f044d9b9038a28e958feeb P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Running, Role: LEADER
I20250109 08:58:10.713812 21660 consensus_queue.cc:260] T 1b582ab8d8f044d9b9038a28e958feeb P 787ddcc824c2490a808fc5a53452dc5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:10.715263 21660 raft_consensus.cc:3054] T 1b582ab8d8f044d9b9038a28e958feeb P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:10.721231 23042 raft_consensus.cc:1059] T d355d166de534fb4a50c9f48f2df7469 P a2b1a692435241ee988e0bd1661cd4c3: attempting to promote NON_VOTER 88935ab835544e9889d05a6587d8cf21 to VOTER
I20250109 08:58:10.722673 21660 raft_consensus.cc:2463] T 1b582ab8d8f044d9b9038a28e958feeb P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88935ab835544e9889d05a6587d8cf21 in term 2.
I20250109 08:58:10.724104 22141 leader_election.cc:304] T 1b582ab8d8f044d9b9038a28e958feeb P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, 88935ab835544e9889d05a6587d8cf21; no voters:
I20250109 08:58:10.724668 23042 consensus_queue.cc:237] T d355d166de534fb4a50c9f48f2df7469 P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:10.726579 23031 raft_consensus.cc:2798] T 1b582ab8d8f044d9b9038a28e958feeb P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:58:10.727224 23031 raft_consensus.cc:695] T 1b582ab8d8f044d9b9038a28e958feeb P 88935ab835544e9889d05a6587d8cf21 [term 2 LEADER]: Becoming Leader. State: Replica: 88935ab835544e9889d05a6587d8cf21, State: Running, Role: LEADER
I20250109 08:58:10.728102 23031 consensus_queue.cc:237] T 1b582ab8d8f044d9b9038a28e958feeb P 88935ab835544e9889d05a6587d8cf21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:10.738437 21513 raft_consensus.cc:2949] T 0e06021cd6624e50af2c914399ebd6c4 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:10.750775 21658 raft_consensus.cc:1270] T d355d166de534fb4a50c9f48f2df7469 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Refusing update from remote peer a2b1a692435241ee988e0bd1661cd4c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20250109 08:58:10.751048 21514 raft_consensus.cc:2463] T 1b582ab8d8f044d9b9038a28e958feeb P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88935ab835544e9889d05a6587d8cf21 in term 2.
I20250109 08:58:10.751943 22177 raft_consensus.cc:1270] T d355d166de534fb4a50c9f48f2df7469 P 88935ab835544e9889d05a6587d8cf21 [term 2 LEARNER]: Refusing update from remote peer a2b1a692435241ee988e0bd1661cd4c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20250109 08:58:10.753440 23041 consensus_queue.cc:1035] T d355d166de534fb4a50c9f48f2df7469 P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250109 08:58:10.754150 22173 raft_consensus.cc:2949] T 0e06021cd6624e50af2c914399ebd6c4 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:10.755122 23041 consensus_queue.cc:1035] T d355d166de534fb4a50c9f48f2df7469 P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250109 08:58:10.761433 21514 raft_consensus.cc:1270] T d355d166de534fb4a50c9f48f2df7469 P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Refusing update from remote peer a2b1a692435241ee988e0bd1661cd4c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20250109 08:58:10.772615 23042 consensus_queue.cc:1035] T d355d166de534fb4a50c9f48f2df7469 P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.006s
I20250109 08:58:10.778318 21411 catalog_manager.cc:5526] T 1b582ab8d8f044d9b9038a28e958feeb P 88935ab835544e9889d05a6587d8cf21 reported cstate change: term changed from 1 to 2, leader changed from 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195) to 88935ab835544e9889d05a6587d8cf21 (127.19.226.196). New cstate: current_term: 2 leader_uuid: "88935ab835544e9889d05a6587d8cf21" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250109 08:58:10.782732 21410 catalog_manager.cc:5526] T 0e06021cd6624e50af2c914399ebd6c4 P 787ddcc824c2490a808fc5a53452dc5e reported cstate change: config changed from index 2 to 3, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a2b1a692435241ee988e0bd1661cd4c3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:10.806155 21513 raft_consensus.cc:2949] T d355d166de534fb4a50c9f48f2df7469 P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:10.803895 23041 raft_consensus.cc:2949] T d355d166de534fb4a50c9f48f2df7469 P a2b1a692435241ee988e0bd1661cd4c3 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:10.812127 21581 consensus_queue.cc:237] T 3b5049c00b0b4dd89c49fa5d13db169e P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:10.814608 21588 consensus_queue.cc:237] T 0e06021cd6624e50af2c914399ebd6c4 P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:10.819280 21660 consensus_queue.cc:237] T 7c19ed9fdde5463f84e43bea7b860d03 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:10.833662 21655 raft_consensus.cc:1270] T 0e06021cd6624e50af2c914399ebd6c4 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Refusing update from remote peer a2b1a692435241ee988e0bd1661cd4c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250109 08:58:10.834990 23039 consensus_queue.cc:1035] T 0e06021cd6624e50af2c914399ebd6c4 P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250109 08:58:10.845875 21409 catalog_manager.cc:5526] T d355d166de534fb4a50c9f48f2df7469 P 8bf6e26e98b14566899578a990b27d47 reported cstate change: config changed from index 3 to 4, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "a2b1a692435241ee988e0bd1661cd4c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:10.849998 21513 raft_consensus.cc:1270] T 3b5049c00b0b4dd89c49fa5d13db169e P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Refusing update from remote peer a2b1a692435241ee988e0bd1661cd4c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250109 08:58:10.850157 23036 raft_consensus.cc:2949] T 0e06021cd6624e50af2c914399ebd6c4 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8bf6e26e98b14566899578a990b27d47 (127.19.226.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:10.851388 23044 consensus_queue.cc:1035] T 3b5049c00b0b4dd89c49fa5d13db169e P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250109 08:58:10.851755 21658 raft_consensus.cc:2949] T d355d166de534fb4a50c9f48f2df7469 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:10.852923 22177 raft_consensus.cc:1270] T 3b5049c00b0b4dd89c49fa5d13db169e P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Refusing update from remote peer a2b1a692435241ee988e0bd1661cd4c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250109 08:58:10.854254 23044 consensus_queue.cc:1035] T 3b5049c00b0b4dd89c49fa5d13db169e P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20250109 08:58:10.855471 21514 raft_consensus.cc:1270] T 7c19ed9fdde5463f84e43bea7b860d03 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250109 08:58:10.855974 21659 raft_consensus.cc:2949] T 0e06021cd6624e50af2c914399ebd6c4 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8bf6e26e98b14566899578a990b27d47 (127.19.226.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:10.856882 23054 consensus_queue.cc:1035] T 7c19ed9fdde5463f84e43bea7b860d03 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250109 08:58:10.857503 22173 raft_consensus.cc:1270] T 0e06021cd6624e50af2c914399ebd6c4 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Refusing update from remote peer a2b1a692435241ee988e0bd1661cd4c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250109 08:58:10.865548 22173 raft_consensus.cc:1270] T 7c19ed9fdde5463f84e43bea7b860d03 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250109 08:58:10.866209 23043 raft_consensus.cc:2949] T 3b5049c00b0b4dd89c49fa5d13db169e P a2b1a692435241ee988e0bd1661cd4c3 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:10.871230 23046 raft_consensus.cc:988] T 206785f29c824cddbe9bbc00d365ffb3 P 8bf6e26e98b14566899578a990b27d47: : Instructing follower 787ddcc824c2490a808fc5a53452dc5e to start an election
I20250109 08:58:10.871863 23045 raft_consensus.cc:1076] T 206785f29c824cddbe9bbc00d365ffb3 P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Signalling peer 787ddcc824c2490a808fc5a53452dc5e to start an election
I20250109 08:58:10.874708 22177 raft_consensus.cc:2949] T d355d166de534fb4a50c9f48f2df7469 P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:10.876879 21660 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "206785f29c824cddbe9bbc00d365ffb3"
dest_uuid: "787ddcc824c2490a808fc5a53452dc5e"
from {username='slave'} at 127.0.0.1:58440
I20250109 08:58:10.877516 21660 raft_consensus.cc:491] T 206785f29c824cddbe9bbc00d365ffb3 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:58:10.877908 21660 raft_consensus.cc:3054] T 206785f29c824cddbe9bbc00d365ffb3 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:10.878937 23039 consensus_queue.cc:1035] T 0e06021cd6624e50af2c914399ebd6c4 P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250109 08:58:10.880244 23050 consensus_queue.cc:1035] T 7c19ed9fdde5463f84e43bea7b860d03 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250109 08:58:10.881359 21513 raft_consensus.cc:2949] T 3b5049c00b0b4dd89c49fa5d13db169e P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:10.887817 22965 raft_consensus.cc:2949] T 7c19ed9fdde5463f84e43bea7b860d03 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:10.899453 22178 raft_consensus.cc:2949] T 3b5049c00b0b4dd89c49fa5d13db169e P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:10.899979 21514 raft_consensus.cc:2949] T 7c19ed9fdde5463f84e43bea7b860d03 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:10.902330 22173 raft_consensus.cc:2949] T 0e06021cd6624e50af2c914399ebd6c4 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8bf6e26e98b14566899578a990b27d47 (127.19.226.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:10.904443 22174 raft_consensus.cc:2949] T 7c19ed9fdde5463f84e43bea7b860d03 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:10.913290 21396 catalog_manager.cc:5039] ChangeConfig:REMOVE_PEER RPC for tablet 7c19ed9fdde5463f84e43bea7b860d03 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250109 08:58:10.915038 21397 catalog_manager.cc:5039] ChangeConfig:REMOVE_PEER RPC for tablet 3b5049c00b0b4dd89c49fa5d13db169e with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250109 08:58:10.915957 21588 consensus_queue.cc:237] T d355d166de534fb4a50c9f48f2df7469 P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:10.917569 21397 catalog_manager.cc:5039] ChangeConfig:REMOVE_PEER RPC for tablet 0e06021cd6624e50af2c914399ebd6c4 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250109 08:58:10.925053 21412 catalog_manager.cc:5526] T 7c19ed9fdde5463f84e43bea7b860d03 P 88935ab835544e9889d05a6587d8cf21 reported cstate change: config changed from index 3 to 4, VOTER a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194) evicted. New cstate: current_term: 1 leader_uuid: "787ddcc824c2490a808fc5a53452dc5e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:10.930577 21660 raft_consensus.cc:513] T 206785f29c824cddbe9bbc00d365ffb3 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:10.935007 21660 leader_election.cc:290] T 206785f29c824cddbe9bbc00d365ffb3 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 2 election: Requested vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), 88935ab835544e9889d05a6587d8cf21 (127.19.226.196:44363)
I20250109 08:58:10.942091 22177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "206785f29c824cddbe9bbc00d365ffb3" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "88935ab835544e9889d05a6587d8cf21"
I20250109 08:58:10.943766 21513 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "206785f29c824cddbe9bbc00d365ffb3" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "8bf6e26e98b14566899578a990b27d47"
I20250109 08:58:10.944360 21513 raft_consensus.cc:3049] T 206785f29c824cddbe9bbc00d365ffb3 P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Stepping down as leader of term 1
I20250109 08:58:10.945022 21513 raft_consensus.cc:738] T 206785f29c824cddbe9bbc00d365ffb3 P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8bf6e26e98b14566899578a990b27d47, State: Running, Role: LEADER
I20250109 08:58:10.943490 22177 raft_consensus.cc:3054] T 206785f29c824cddbe9bbc00d365ffb3 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:10.946177 21412 catalog_manager.cc:5526] T 0e06021cd6624e50af2c914399ebd6c4 P 787ddcc824c2490a808fc5a53452dc5e reported cstate change: config changed from index 3 to 4, VOTER 8bf6e26e98b14566899578a990b27d47 (127.19.226.193) evicted. New cstate: current_term: 1 leader_uuid: "a2b1a692435241ee988e0bd1661cd4c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:10.947474 21660 raft_consensus.cc:1270] T d355d166de534fb4a50c9f48f2df7469 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Refusing update from remote peer a2b1a692435241ee988e0bd1661cd4c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250109 08:58:10.949316 23043 consensus_queue.cc:1035] T d355d166de534fb4a50c9f48f2df7469 P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250109 08:58:10.952651 21570 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c19ed9fdde5463f84e43bea7b860d03 with delete_type TABLET_DATA_TOMBSTONED (TS a2b1a692435241ee988e0bd1661cd4c3 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:34732
I20250109 08:58:10.959976 23036 raft_consensus.cc:2949] T d355d166de534fb4a50c9f48f2df7469 P a2b1a692435241ee988e0bd1661cd4c3 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 8bf6e26e98b14566899578a990b27d47 (127.19.226.193) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:10.962900 23076 tablet_replica.cc:331] T 7c19ed9fdde5463f84e43bea7b860d03 P a2b1a692435241ee988e0bd1661cd4c3: stopping tablet replica
I20250109 08:58:10.963882 21409 catalog_manager.cc:5526] T 3b5049c00b0b4dd89c49fa5d13db169e P 88935ab835544e9889d05a6587d8cf21 reported cstate change: config changed from index 3 to 4, VOTER 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195) evicted. New cstate: current_term: 1 leader_uuid: "a2b1a692435241ee988e0bd1661cd4c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:10.966295 21503 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e06021cd6624e50af2c914399ebd6c4 with delete_type TABLET_DATA_TOMBSTONED (TS 8bf6e26e98b14566899578a990b27d47 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:39242
I20250109 08:58:10.967615 23076 raft_consensus.cc:2238] T 7c19ed9fdde5463f84e43bea7b860d03 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:10.968293 23076 raft_consensus.cc:2267] T 7c19ed9fdde5463f84e43bea7b860d03 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:10.972189 23076 ts_tablet_manager.cc:1905] T 7c19ed9fdde5463f84e43bea7b860d03 P a2b1a692435241ee988e0bd1661cd4c3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250109 08:58:10.971484 21663 raft_consensus.cc:2949] T d355d166de534fb4a50c9f48f2df7469 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 8bf6e26e98b14566899578a990b27d47 (127.19.226.193) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:10.945804 21513 consensus_queue.cc:260] T 206785f29c824cddbe9bbc00d365ffb3 P 8bf6e26e98b14566899578a990b27d47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:10.982000 21513 raft_consensus.cc:3054] T 206785f29c824cddbe9bbc00d365ffb3 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:10.974722 22177 raft_consensus.cc:2463] T 206785f29c824cddbe9bbc00d365ffb3 P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 2.
I20250109 08:58:10.976509 21397 catalog_manager.cc:5039] ChangeConfig:REMOVE_PEER RPC for tablet d355d166de534fb4a50c9f48f2df7469 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250109 08:58:10.986924 21627 leader_election.cc:304] T 206785f29c824cddbe9bbc00d365ffb3 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, 88935ab835544e9889d05a6587d8cf21; no voters:
I20250109 08:58:10.988207 22965 raft_consensus.cc:2798] T 206785f29c824cddbe9bbc00d365ffb3 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:58:10.989070 22965 raft_consensus.cc:695] T 206785f29c824cddbe9bbc00d365ffb3 P 787ddcc824c2490a808fc5a53452dc5e [term 2 LEADER]: Becoming Leader. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Running, Role: LEADER
I20250109 08:58:10.990072 22965 consensus_queue.cc:237] T 206785f29c824cddbe9bbc00d365ffb3 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:10.976418 22173 raft_consensus.cc:1270] T d355d166de534fb4a50c9f48f2df7469 P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Refusing update from remote peer a2b1a692435241ee988e0bd1661cd4c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250109 08:58:10.998311 21513 raft_consensus.cc:2463] T 206785f29c824cddbe9bbc00d365ffb3 P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 2.
I20250109 08:58:11.004513 23044 consensus_queue.cc:1035] T d355d166de534fb4a50c9f48f2df7469 P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250109 08:58:11.010318 22173 raft_consensus.cc:2949] T d355d166de534fb4a50c9f48f2df7469 P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 8bf6e26e98b14566899578a990b27d47 (127.19.226.193) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
I20250109 08:58:11.017980 21653 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b5049c00b0b4dd89c49fa5d13db169e with delete_type TABLET_DATA_TOMBSTONED (TS 787ddcc824c2490a808fc5a53452dc5e not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:58436
I20250109 08:58:11.042512 21410 catalog_manager.cc:5526] T d355d166de534fb4a50c9f48f2df7469 P a2b1a692435241ee988e0bd1661cd4c3 reported cstate change: config changed from index 4 to 5, VOTER 8bf6e26e98b14566899578a990b27d47 (127.19.226.193) evicted. New cstate: current_term: 2 leader_uuid: "a2b1a692435241ee988e0bd1661cd4c3" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:11.044080 21412 catalog_manager.cc:5526] T 206785f29c824cddbe9bbc00d365ffb3 P 787ddcc824c2490a808fc5a53452dc5e reported cstate change: term changed from 1 to 2, leader changed from 8bf6e26e98b14566899578a990b27d47 (127.19.226.193) to 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195). New cstate: current_term: 2 leader_uuid: "787ddcc824c2490a808fc5a53452dc5e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:11.060559 23077 tablet_replica.cc:331] T 0e06021cd6624e50af2c914399ebd6c4 P 8bf6e26e98b14566899578a990b27d47: stopping tablet replica
I20250109 08:58:11.060727 21500 tablet_service.cc:1514] Processing DeleteTablet for tablet d355d166de534fb4a50c9f48f2df7469 with delete_type TABLET_DATA_TOMBSTONED (TS 8bf6e26e98b14566899578a990b27d47 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:39242
I20250109 08:58:11.066437 23077 raft_consensus.cc:2238] T 0e06021cd6624e50af2c914399ebd6c4 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:11.069737 23077 raft_consensus.cc:2267] T 0e06021cd6624e50af2c914399ebd6c4 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:11.074544 23077 ts_tablet_manager.cc:1905] T 0e06021cd6624e50af2c914399ebd6c4 P 8bf6e26e98b14566899578a990b27d47: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250109 08:58:11.075506 23079 tablet_replica.cc:331] T 3b5049c00b0b4dd89c49fa5d13db169e P 787ddcc824c2490a808fc5a53452dc5e: stopping tablet replica
I20250109 08:58:11.078850 23076 ts_tablet_manager.cc:1918] T 7c19ed9fdde5463f84e43bea7b860d03 P a2b1a692435241ee988e0bd1661cd4c3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20250109 08:58:11.080399 23079 raft_consensus.cc:2238] T 3b5049c00b0b4dd89c49fa5d13db169e P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:11.091785 23076 log.cc:1198] T 7c19ed9fdde5463f84e43bea7b860d03 P a2b1a692435241ee988e0bd1661cd4c3: Deleting WAL directory at /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736412987538469-20363-0/minicluster-data/ts-1-root/wals/7c19ed9fdde5463f84e43bea7b860d03
I20250109 08:58:11.092263 23079 raft_consensus.cc:2267] T 3b5049c00b0b4dd89c49fa5d13db169e P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:11.094571 21397 catalog_manager.cc:4872] TS a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): tablet 7c19ed9fdde5463f84e43bea7b860d03 (table test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]) successfully deleted
I20250109 08:58:11.094877 23077 ts_tablet_manager.cc:1918] T 0e06021cd6624e50af2c914399ebd6c4 P 8bf6e26e98b14566899578a990b27d47: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20250109 08:58:11.095368 23077 log.cc:1198] T 0e06021cd6624e50af2c914399ebd6c4 P 8bf6e26e98b14566899578a990b27d47: Deleting WAL directory at /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736412987538469-20363-0/minicluster-data/ts-0-root/wals/0e06021cd6624e50af2c914399ebd6c4
I20250109 08:58:11.095489 23079 ts_tablet_manager.cc:1905] T 3b5049c00b0b4dd89c49fa5d13db169e P 787ddcc824c2490a808fc5a53452dc5e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250109 08:58:11.097106 21396 catalog_manager.cc:4872] TS 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): tablet 0e06021cd6624e50af2c914399ebd6c4 (table test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]) successfully deleted
I20250109 08:58:11.098126 23077 tablet_replica.cc:331] T d355d166de534fb4a50c9f48f2df7469 P 8bf6e26e98b14566899578a990b27d47: stopping tablet replica
I20250109 08:58:11.099315 23077 raft_consensus.cc:2238] T d355d166de534fb4a50c9f48f2df7469 P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:11.099957 23077 raft_consensus.cc:2267] T d355d166de534fb4a50c9f48f2df7469 P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:11.103655 23077 ts_tablet_manager.cc:1905] T d355d166de534fb4a50c9f48f2df7469 P 8bf6e26e98b14566899578a990b27d47: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250109 08:58:11.111796 23079 ts_tablet_manager.cc:1918] T 3b5049c00b0b4dd89c49fa5d13db169e P 787ddcc824c2490a808fc5a53452dc5e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20250109 08:58:11.112339 23079 log.cc:1198] T 3b5049c00b0b4dd89c49fa5d13db169e P 787ddcc824c2490a808fc5a53452dc5e: Deleting WAL directory at /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736412987538469-20363-0/minicluster-data/ts-2-root/wals/3b5049c00b0b4dd89c49fa5d13db169e
I20250109 08:58:11.114110 21396 catalog_manager.cc:4872] TS 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731): tablet 3b5049c00b0b4dd89c49fa5d13db169e (table test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]) successfully deleted
I20250109 08:58:11.125893 23077 ts_tablet_manager.cc:1918] T d355d166de534fb4a50c9f48f2df7469 P 8bf6e26e98b14566899578a990b27d47: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20250109 08:58:11.126461 23077 log.cc:1198] T d355d166de534fb4a50c9f48f2df7469 P 8bf6e26e98b14566899578a990b27d47: Deleting WAL directory at /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736412987538469-20363-0/minicluster-data/ts-0-root/wals/d355d166de534fb4a50c9f48f2df7469
I20250109 08:58:11.127648 21513 raft_consensus.cc:1270] T 1b582ab8d8f044d9b9038a28e958feeb P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Refusing update from remote peer 88935ab835544e9889d05a6587d8cf21: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250109 08:58:11.127911 21396 catalog_manager.cc:4872] TS 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): tablet d355d166de534fb4a50c9f48f2df7469 (table test-workload [id=0d19d05e9f294ed7a1fbb42318a51ec7]) successfully deleted
I20250109 08:58:11.129001 23031 consensus_queue.cc:1035] T 1b582ab8d8f044d9b9038a28e958feeb P 88935ab835544e9889d05a6587d8cf21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250109 08:58:11.143833 21663 raft_consensus.cc:1270] T 1b582ab8d8f044d9b9038a28e958feeb P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Refusing update from remote peer 88935ab835544e9889d05a6587d8cf21: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250109 08:58:11.147931 23031 consensus_queue.cc:1035] T 1b582ab8d8f044d9b9038a28e958feeb P 88935ab835544e9889d05a6587d8cf21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20250109 08:58:11.246454 21610 debug-util.cc:398] Leaking SignalData structure 0x7b0800171b00 after lost signal to thread 20366
W20250109 08:58:11.247498 21610 debug-util.cc:398] Leaking SignalData structure 0x7b080016d120 after lost signal to thread 21446
W20250109 08:58:11.248430 21610 debug-util.cc:398] Leaking SignalData structure 0x7b08000e2820 after lost signal to thread 21538
W20250109 08:58:11.249354 21610 debug-util.cc:398] Leaking SignalData structure 0x7b0800304b40 after lost signal to thread 21613
W20250109 08:58:11.250144 21610 debug-util.cc:398] Leaking SignalData structure 0x7b080035fba0 after lost signal to thread 21687
W20250109 08:58:11.250959 21610 debug-util.cc:398] Leaking SignalData structure 0x7b0800325360 after lost signal to thread 22202
I20250109 08:58:11.378707 22177 raft_consensus.cc:1270] T 206785f29c824cddbe9bbc00d365ffb3 P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250109 08:58:11.379892 22965 consensus_queue.cc:1035] T 206785f29c824cddbe9bbc00d365ffb3 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250109 08:58:11.388851 21509 raft_consensus.cc:1270] T 206785f29c824cddbe9bbc00d365ffb3 P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250109 08:58:11.390581 22965 consensus_queue.cc:1035] T 206785f29c824cddbe9bbc00d365ffb3 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250109 08:58:11.600298 20363 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250109 08:58:11.648098 21660 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "206785f29c824cddbe9bbc00d365ffb3"
dest_uuid: "787ddcc824c2490a808fc5a53452dc5e"
mode: GRACEFUL
new_leader_uuid: "88935ab835544e9889d05a6587d8cf21"
from {username='slave'} at 127.0.0.1:55054
I20250109 08:58:11.648651 21660 raft_consensus.cc:604] T 206785f29c824cddbe9bbc00d365ffb3 P 787ddcc824c2490a808fc5a53452dc5e [term 2 LEADER]: Received request to transfer leadership to TS 88935ab835544e9889d05a6587d8cf21
I20250109 08:58:11.649792 20363 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 1
I20250109 08:58:11.651331 20363 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250109 08:58:11.651918 20363 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250109 08:58:12.015640 23053 raft_consensus.cc:988] T 206785f29c824cddbe9bbc00d365ffb3 P 787ddcc824c2490a808fc5a53452dc5e: : Instructing follower 88935ab835544e9889d05a6587d8cf21 to start an election
I20250109 08:58:12.016887 23053 raft_consensus.cc:1076] T 206785f29c824cddbe9bbc00d365ffb3 P 787ddcc824c2490a808fc5a53452dc5e [term 2 LEADER]: Signalling peer 88935ab835544e9889d05a6587d8cf21 to start an election
I20250109 08:58:12.018369 22173 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "206785f29c824cddbe9bbc00d365ffb3"
dest_uuid: "88935ab835544e9889d05a6587d8cf21"
from {username='slave'} at 127.0.0.1:60458
I20250109 08:58:12.018931 22173 raft_consensus.cc:491] T 206785f29c824cddbe9bbc00d365ffb3 P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:58:12.019280 22173 raft_consensus.cc:3054] T 206785f29c824cddbe9bbc00d365ffb3 P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Advancing to term 3
I20250109 08:58:12.026523 22173 raft_consensus.cc:513] T 206785f29c824cddbe9bbc00d365ffb3 P 88935ab835544e9889d05a6587d8cf21 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:12.029325 21663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "206785f29c824cddbe9bbc00d365ffb3" candidate_uuid: "88935ab835544e9889d05a6587d8cf21" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "787ddcc824c2490a808fc5a53452dc5e"
I20250109 08:58:12.029899 21663 raft_consensus.cc:3049] T 206785f29c824cddbe9bbc00d365ffb3 P 787ddcc824c2490a808fc5a53452dc5e [term 2 LEADER]: Stepping down as leader of term 2
I20250109 08:58:12.030182 21663 raft_consensus.cc:738] T 206785f29c824cddbe9bbc00d365ffb3 P 787ddcc824c2490a808fc5a53452dc5e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Running, Role: LEADER
I20250109 08:58:12.030851 21663 consensus_queue.cc:260] T 206785f29c824cddbe9bbc00d365ffb3 P 787ddcc824c2490a808fc5a53452dc5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:12.031869 21663 raft_consensus.cc:3054] T 206785f29c824cddbe9bbc00d365ffb3 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Advancing to term 3
I20250109 08:58:12.037163 21663 raft_consensus.cc:2463] T 206785f29c824cddbe9bbc00d365ffb3 P 787ddcc824c2490a808fc5a53452dc5e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88935ab835544e9889d05a6587d8cf21 in term 3.
I20250109 08:58:12.038292 22141 leader_election.cc:304] T 206785f29c824cddbe9bbc00d365ffb3 P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, 88935ab835544e9889d05a6587d8cf21; no voters:
I20250109 08:58:12.039048 23031 raft_consensus.cc:2798] T 206785f29c824cddbe9bbc00d365ffb3 P 88935ab835544e9889d05a6587d8cf21 [term 3 FOLLOWER]: Leader election won for term 3
I20250109 08:58:12.039505 23031 raft_consensus.cc:695] T 206785f29c824cddbe9bbc00d365ffb3 P 88935ab835544e9889d05a6587d8cf21 [term 3 LEADER]: Becoming Leader. State: Replica: 88935ab835544e9889d05a6587d8cf21, State: Running, Role: LEADER
I20250109 08:58:12.040254 23031 consensus_queue.cc:237] T 206785f29c824cddbe9bbc00d365ffb3 P 88935ab835544e9889d05a6587d8cf21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:12.040760 22173 leader_election.cc:290] T 206785f29c824cddbe9bbc00d365ffb3 P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 3 election: Requested vote from peers 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731), 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725)
I20250109 08:58:12.043298 21513 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "206785f29c824cddbe9bbc00d365ffb3" candidate_uuid: "88935ab835544e9889d05a6587d8cf21" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "8bf6e26e98b14566899578a990b27d47"
I20250109 08:58:12.043884 21513 raft_consensus.cc:3054] T 206785f29c824cddbe9bbc00d365ffb3 P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Advancing to term 3
I20250109 08:58:12.055795 21412 catalog_manager.cc:5526] T 206785f29c824cddbe9bbc00d365ffb3 P 88935ab835544e9889d05a6587d8cf21 reported cstate change: term changed from 2 to 3, leader changed from 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195) to 88935ab835544e9889d05a6587d8cf21 (127.19.226.196). New cstate: current_term: 3 leader_uuid: "88935ab835544e9889d05a6587d8cf21" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250109 08:58:12.059923 21513 raft_consensus.cc:2463] T 206785f29c824cddbe9bbc00d365ffb3 P 8bf6e26e98b14566899578a990b27d47 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88935ab835544e9889d05a6587d8cf21 in term 3.
I20250109 08:58:12.101032 21581 consensus_queue.cc:237] T 8c0459d2eee64f0a83bd05ad5974f506 P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:58:12.109057 21513 raft_consensus.cc:1270] T 8c0459d2eee64f0a83bd05ad5974f506 P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Refusing update from remote peer a2b1a692435241ee988e0bd1661cd4c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20250109 08:58:12.109381 21663 raft_consensus.cc:1270] T 8c0459d2eee64f0a83bd05ad5974f506 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Refusing update from remote peer a2b1a692435241ee988e0bd1661cd4c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20250109 08:58:12.111485 23089 consensus_queue.cc:1035] T 8c0459d2eee64f0a83bd05ad5974f506 P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:58:12.112622 23044 consensus_queue.cc:1035] T 8c0459d2eee64f0a83bd05ad5974f506 P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20250109 08:58:12.118217 21553 consensus_peers.cc:487] T 8c0459d2eee64f0a83bd05ad5974f506 P a2b1a692435241ee988e0bd1661cd4c3 -> Peer 88935ab835544e9889d05a6587d8cf21 (127.19.226.196:44363): Couldn't send request to peer 88935ab835544e9889d05a6587d8cf21. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8c0459d2eee64f0a83bd05ad5974f506. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:12.131953 23036 raft_consensus.cc:2949] T 8c0459d2eee64f0a83bd05ad5974f506 P a2b1a692435241ee988e0bd1661cd4c3 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:58:12.138981 21663 raft_consensus.cc:2949] T 8c0459d2eee64f0a83bd05ad5974f506 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:58:12.141443 21509 raft_consensus.cc:2949] T 8c0459d2eee64f0a83bd05ad5974f506 P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:58:12.151494 21412 catalog_manager.cc:5526] T 8c0459d2eee64f0a83bd05ad5974f506 P a2b1a692435241ee988e0bd1661cd4c3 reported cstate change: config changed from index -1 to 3, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New cstate: current_term: 2 leader_uuid: "a2b1a692435241ee988e0bd1661cd4c3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:12.160705 21513 consensus_queue.cc:237] T 363fa693a5114f2bb22f4ab67722a1c6 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:58:12.180676 21663 raft_consensus.cc:1270] T 363fa693a5114f2bb22f4ab67722a1c6 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Refusing update from remote peer 8bf6e26e98b14566899578a990b27d47: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20250109 08:58:12.182576 21581 raft_consensus.cc:1270] T 363fa693a5114f2bb22f4ab67722a1c6 P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Refusing update from remote peer 8bf6e26e98b14566899578a990b27d47: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20250109 08:58:12.183538 23046 consensus_queue.cc:1035] T 363fa693a5114f2bb22f4ab67722a1c6 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:58:12.186831 23045 consensus_queue.cc:1035] T 363fa693a5114f2bb22f4ab67722a1c6 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20250109 08:58:12.192299 21478 consensus_peers.cc:487] T 363fa693a5114f2bb22f4ab67722a1c6 P 8bf6e26e98b14566899578a990b27d47 -> Peer 88935ab835544e9889d05a6587d8cf21 (127.19.226.196:44363): Couldn't send request to peer 88935ab835544e9889d05a6587d8cf21. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 363fa693a5114f2bb22f4ab67722a1c6. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:12.201828 21663 raft_consensus.cc:2949] T 363fa693a5114f2bb22f4ab67722a1c6 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:58:12.222559 21581 raft_consensus.cc:2949] T 363fa693a5114f2bb22f4ab67722a1c6 P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:58:12.201903 23045 raft_consensus.cc:2949] T 363fa693a5114f2bb22f4ab67722a1c6 P 8bf6e26e98b14566899578a990b27d47 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:58:12.228055 21409 catalog_manager.cc:5526] T 363fa693a5114f2bb22f4ab67722a1c6 P 787ddcc824c2490a808fc5a53452dc5e reported cstate change: config changed from index -1 to 3, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New cstate: current_term: 2 leader_uuid: "8bf6e26e98b14566899578a990b27d47" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:58:12.247259 21660 consensus_queue.cc:237] T af330dcd4767423ea4c179be7f88d0c4 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:58:12.264842 21581 raft_consensus.cc:1270] T af330dcd4767423ea4c179be7f88d0c4 P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
W20250109 08:58:12.266487 21627 consensus_peers.cc:487] T af330dcd4767423ea4c179be7f88d0c4 P 787ddcc824c2490a808fc5a53452dc5e -> Peer 88935ab835544e9889d05a6587d8cf21 (127.19.226.196:44363): Couldn't send request to peer 88935ab835544e9889d05a6587d8cf21. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: af330dcd4767423ea4c179be7f88d0c4. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:12.266858 21513 raft_consensus.cc:1270] T af330dcd4767423ea4c179be7f88d0c4 P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20250109 08:58:12.267138 23053 consensus_queue.cc:1035] T af330dcd4767423ea4c179be7f88d0c4 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:58:12.268267 23050 consensus_queue.cc:1035] T af330dcd4767423ea4c179be7f88d0c4 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:58:12.281879 22965 raft_consensus.cc:2949] T af330dcd4767423ea4c179be7f88d0c4 P 787ddcc824c2490a808fc5a53452dc5e [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:58:12.297363 21509 consensus_queue.cc:237] T e38ec0ef84af4b798073eafd4d210c35 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:58:12.298065 21514 raft_consensus.cc:2949] T af330dcd4767423ea4c179be7f88d0c4 P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:58:12.309566 21410 catalog_manager.cc:5526] T af330dcd4767423ea4c179be7f88d0c4 P 787ddcc824c2490a808fc5a53452dc5e reported cstate change: config changed from index -1 to 3, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New cstate: current_term: 2 leader_uuid: "787ddcc824c2490a808fc5a53452dc5e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:12.328876 21588 raft_consensus.cc:2949] T af330dcd4767423ea4c179be7f88d0c4 P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:58:12.365833 21660 raft_consensus.cc:1270] T e38ec0ef84af4b798073eafd4d210c35 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Refusing update from remote peer 8bf6e26e98b14566899578a990b27d47: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20250109 08:58:12.366158 21478 consensus_peers.cc:487] T e38ec0ef84af4b798073eafd4d210c35 P 8bf6e26e98b14566899578a990b27d47 -> Peer 88935ab835544e9889d05a6587d8cf21 (127.19.226.196:44363): Couldn't send request to peer 88935ab835544e9889d05a6587d8cf21. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e38ec0ef84af4b798073eafd4d210c35. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:12.367570 21581 raft_consensus.cc:1270] T e38ec0ef84af4b798073eafd4d210c35 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Refusing update from remote peer 8bf6e26e98b14566899578a990b27d47: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250109 08:58:12.369168 23097 consensus_queue.cc:1035] T e38ec0ef84af4b798073eafd4d210c35 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:58:12.373692 23097 consensus_queue.cc:1035] T e38ec0ef84af4b798073eafd4d210c35 P 8bf6e26e98b14566899578a990b27d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:58:12.388590 23097 raft_consensus.cc:2949] T e38ec0ef84af4b798073eafd4d210c35 P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:58:12.389977 21585 raft_consensus.cc:2949] T e38ec0ef84af4b798073eafd4d210c35 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:58:12.394019 21660 raft_consensus.cc:2949] T e38ec0ef84af4b798073eafd4d210c35 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:58:12.405568 21410 catalog_manager.cc:5526] T e38ec0ef84af4b798073eafd4d210c35 P a2b1a692435241ee988e0bd1661cd4c3 reported cstate change: config changed from index -1 to 2, NON_VOTER 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) added. New cstate: current_term: 1 leader_uuid: "8bf6e26e98b14566899578a990b27d47" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } }
I20250109 08:58:12.440300 21660 raft_consensus.cc:1270] T 206785f29c824cddbe9bbc00d365ffb3 P 787ddcc824c2490a808fc5a53452dc5e [term 3 FOLLOWER]: Refusing update from remote peer 88935ab835544e9889d05a6587d8cf21: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20250109 08:58:12.441943 23082 consensus_queue.cc:1035] T 206785f29c824cddbe9bbc00d365ffb3 P 88935ab835544e9889d05a6587d8cf21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20250109 08:58:12.473754 21514 raft_consensus.cc:1270] T 206785f29c824cddbe9bbc00d365ffb3 P 8bf6e26e98b14566899578a990b27d47 [term 3 FOLLOWER]: Refusing update from remote peer 88935ab835544e9889d05a6587d8cf21: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20250109 08:58:12.477159 23031 consensus_queue.cc:1035] T 206785f29c824cddbe9bbc00d365ffb3 P 88935ab835544e9889d05a6587d8cf21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20250109 08:58:12.570609 23113 ts_tablet_manager.cc:927] T 8c0459d2eee64f0a83bd05ad5974f506 P 88935ab835544e9889d05a6587d8cf21: Initiating tablet copy from peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
I20250109 08:58:12.588380 23113 tablet_copy_client.cc:323] T 8c0459d2eee64f0a83bd05ad5974f506 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Beginning tablet copy session from remote peer at address 127.19.226.194:45527
I20250109 08:58:12.590034 21599 tablet_copy_service.cc:140] P a2b1a692435241ee988e0bd1661cd4c3: Received BeginTabletCopySession request for tablet 8c0459d2eee64f0a83bd05ad5974f506 from peer 88935ab835544e9889d05a6587d8cf21 ({username='slave'} at 127.0.0.1:33652)
I20250109 08:58:12.590559 21599 tablet_copy_service.cc:161] P a2b1a692435241ee988e0bd1661cd4c3: Beginning new tablet copy session on tablet 8c0459d2eee64f0a83bd05ad5974f506 from peer 88935ab835544e9889d05a6587d8cf21 at {username='slave'} at 127.0.0.1:33652: session id = 88935ab835544e9889d05a6587d8cf21-8c0459d2eee64f0a83bd05ad5974f506
I20250109 08:58:12.602437 21599 tablet_copy_source_session.cc:215] T 8c0459d2eee64f0a83bd05ad5974f506 P a2b1a692435241ee988e0bd1661cd4c3: Tablet Copy: opened 0 blocks and 1 log segments
I20250109 08:58:12.606520 23113 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8c0459d2eee64f0a83bd05ad5974f506. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:12.632123 23113 tablet_copy_client.cc:806] T 8c0459d2eee64f0a83bd05ad5974f506 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Starting download of 0 data blocks...
I20250109 08:58:12.632705 23113 tablet_copy_client.cc:670] T 8c0459d2eee64f0a83bd05ad5974f506 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Starting download of 1 WAL segments...
I20250109 08:58:12.636965 23113 tablet_copy_client.cc:538] T 8c0459d2eee64f0a83bd05ad5974f506 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250109 08:58:12.645886 23113 tablet_bootstrap.cc:492] T 8c0459d2eee64f0a83bd05ad5974f506 P 88935ab835544e9889d05a6587d8cf21: Bootstrap starting.
I20250109 08:58:12.671640 23113 tablet_bootstrap.cc:492] T 8c0459d2eee64f0a83bd05ad5974f506 P 88935ab835544e9889d05a6587d8cf21: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250109 08:58:12.672686 23113 tablet_bootstrap.cc:492] T 8c0459d2eee64f0a83bd05ad5974f506 P 88935ab835544e9889d05a6587d8cf21: Bootstrap complete.
I20250109 08:58:12.673430 23113 ts_tablet_manager.cc:1397] T 8c0459d2eee64f0a83bd05ad5974f506 P 88935ab835544e9889d05a6587d8cf21: Time spent bootstrapping tablet: real 0.028s user 0.021s sys 0.004s
I20250109 08:58:12.676427 23113 raft_consensus.cc:357] T 8c0459d2eee64f0a83bd05ad5974f506 P 88935ab835544e9889d05a6587d8cf21 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:58:12.677201 23113 raft_consensus.cc:738] T 8c0459d2eee64f0a83bd05ad5974f506 P 88935ab835544e9889d05a6587d8cf21 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 88935ab835544e9889d05a6587d8cf21, State: Initialized, Role: LEARNER
I20250109 08:58:12.677822 23113 consensus_queue.cc:260] T 8c0459d2eee64f0a83bd05ad5974f506 P 88935ab835544e9889d05a6587d8cf21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:58:12.680318 23113 ts_tablet_manager.cc:1428] T 8c0459d2eee64f0a83bd05ad5974f506 P 88935ab835544e9889d05a6587d8cf21: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20250109 08:58:12.683226 21599 tablet_copy_service.cc:342] P a2b1a692435241ee988e0bd1661cd4c3: Request end of tablet copy session 88935ab835544e9889d05a6587d8cf21-8c0459d2eee64f0a83bd05ad5974f506 received from {username='slave'} at 127.0.0.1:33652
I20250109 08:58:12.683652 21599 tablet_copy_service.cc:434] P a2b1a692435241ee988e0bd1661cd4c3: ending tablet copy session 88935ab835544e9889d05a6587d8cf21-8c0459d2eee64f0a83bd05ad5974f506 on tablet 8c0459d2eee64f0a83bd05ad5974f506 with peer 88935ab835544e9889d05a6587d8cf21
I20250109 08:58:12.699124 20363 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250109 08:58:12.759876 20363 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250109 08:58:12.761581 20363 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250109 08:58:12.763396 20363 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250109 08:58:12.777158 23113 ts_tablet_manager.cc:927] T 363fa693a5114f2bb22f4ab67722a1c6 P 88935ab835544e9889d05a6587d8cf21: Initiating tablet copy from peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725)
I20250109 08:58:12.781275 23113 tablet_copy_client.cc:323] T 363fa693a5114f2bb22f4ab67722a1c6 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Beginning tablet copy session from remote peer at address 127.19.226.193:42725
I20250109 08:58:12.784821 21523 tablet_copy_service.cc:140] P 8bf6e26e98b14566899578a990b27d47: Received BeginTabletCopySession request for tablet 363fa693a5114f2bb22f4ab67722a1c6 from peer 88935ab835544e9889d05a6587d8cf21 ({username='slave'} at 127.0.0.1:41812)
I20250109 08:58:12.785367 21523 tablet_copy_service.cc:161] P 8bf6e26e98b14566899578a990b27d47: Beginning new tablet copy session on tablet 363fa693a5114f2bb22f4ab67722a1c6 from peer 88935ab835544e9889d05a6587d8cf21 at {username='slave'} at 127.0.0.1:41812: session id = 88935ab835544e9889d05a6587d8cf21-363fa693a5114f2bb22f4ab67722a1c6
I20250109 08:58:12.793351 21523 tablet_copy_source_session.cc:215] T 363fa693a5114f2bb22f4ab67722a1c6 P 8bf6e26e98b14566899578a990b27d47: Tablet Copy: opened 0 blocks and 1 log segments
I20250109 08:58:12.801820 23113 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 363fa693a5114f2bb22f4ab67722a1c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:12.813252 23117 ts_tablet_manager.cc:927] T af330dcd4767423ea4c179be7f88d0c4 P 88935ab835544e9889d05a6587d8cf21: Initiating tablet copy from peer 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:58:12.815951 23117 tablet_copy_client.cc:323] T af330dcd4767423ea4c179be7f88d0c4 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Beginning tablet copy session from remote peer at address 127.19.226.195:35731
I20250109 08:58:12.817636 21673 tablet_copy_service.cc:140] P 787ddcc824c2490a808fc5a53452dc5e: Received BeginTabletCopySession request for tablet af330dcd4767423ea4c179be7f88d0c4 from peer 88935ab835544e9889d05a6587d8cf21 ({username='slave'} at 127.0.0.1:52376)
I20250109 08:58:12.818117 21673 tablet_copy_service.cc:161] P 787ddcc824c2490a808fc5a53452dc5e: Beginning new tablet copy session on tablet af330dcd4767423ea4c179be7f88d0c4 from peer 88935ab835544e9889d05a6587d8cf21 at {username='slave'} at 127.0.0.1:52376: session id = 88935ab835544e9889d05a6587d8cf21-af330dcd4767423ea4c179be7f88d0c4
I20250109 08:58:12.825160 21673 tablet_copy_source_session.cc:215] T af330dcd4767423ea4c179be7f88d0c4 P 787ddcc824c2490a808fc5a53452dc5e: Tablet Copy: opened 0 blocks and 1 log segments
I20250109 08:58:12.828636 23117 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af330dcd4767423ea4c179be7f88d0c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:12.854938 20363 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250109 08:58:12.858829 23113 tablet_copy_client.cc:806] T 363fa693a5114f2bb22f4ab67722a1c6 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Starting download of 0 data blocks...
I20250109 08:58:12.859815 23119 ts_tablet_manager.cc:927] T e38ec0ef84af4b798073eafd4d210c35 P 88935ab835544e9889d05a6587d8cf21: Initiating tablet copy from peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725)
I20250109 08:58:12.859519 23113 tablet_copy_client.cc:670] T 363fa693a5114f2bb22f4ab67722a1c6 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Starting download of 1 WAL segments...
I20250109 08:58:12.860934 23117 tablet_copy_client.cc:806] T af330dcd4767423ea4c179be7f88d0c4 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Starting download of 0 data blocks...
I20250109 08:58:12.861495 23117 tablet_copy_client.cc:670] T af330dcd4767423ea4c179be7f88d0c4 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Starting download of 1 WAL segments...
I20250109 08:58:12.865962 23113 tablet_copy_client.cc:538] T 363fa693a5114f2bb22f4ab67722a1c6 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250109 08:58:12.870312 23117 tablet_copy_client.cc:538] T af330dcd4767423ea4c179be7f88d0c4 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250109 08:58:12.884424 23119 tablet_copy_client.cc:323] T e38ec0ef84af4b798073eafd4d210c35 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Beginning tablet copy session from remote peer at address 127.19.226.193:42725
I20250109 08:58:12.886101 21523 tablet_copy_service.cc:140] P 8bf6e26e98b14566899578a990b27d47: Received BeginTabletCopySession request for tablet e38ec0ef84af4b798073eafd4d210c35 from peer 88935ab835544e9889d05a6587d8cf21 ({username='slave'} at 127.0.0.1:41812)
I20250109 08:58:12.886649 21523 tablet_copy_service.cc:161] P 8bf6e26e98b14566899578a990b27d47: Beginning new tablet copy session on tablet e38ec0ef84af4b798073eafd4d210c35 from peer 88935ab835544e9889d05a6587d8cf21 at {username='slave'} at 127.0.0.1:41812: session id = 88935ab835544e9889d05a6587d8cf21-e38ec0ef84af4b798073eafd4d210c35
I20250109 08:58:12.894124 21523 tablet_copy_source_session.cc:215] T e38ec0ef84af4b798073eafd4d210c35 P 8bf6e26e98b14566899578a990b27d47: Tablet Copy: opened 0 blocks and 1 log segments
I20250109 08:58:12.896620 23117 tablet_bootstrap.cc:492] T af330dcd4767423ea4c179be7f88d0c4 P 88935ab835544e9889d05a6587d8cf21: Bootstrap starting.
I20250109 08:58:12.897329 23119 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e38ec0ef84af4b798073eafd4d210c35. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:12.942234 23113 tablet_bootstrap.cc:492] T 363fa693a5114f2bb22f4ab67722a1c6 P 88935ab835544e9889d05a6587d8cf21: Bootstrap starting.
I20250109 08:58:12.952425 20363 tablet_server.cc:178] TabletServer@127.19.226.193:0 shutting down...
I20250109 08:58:12.955402 23117 tablet_bootstrap.cc:492] T af330dcd4767423ea4c179be7f88d0c4 P 88935ab835544e9889d05a6587d8cf21: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20250109 08:58:12.959828 21396 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 3 similar messages]
I20250109 08:58:12.950871 23119 tablet_copy_client.cc:806] T e38ec0ef84af4b798073eafd4d210c35 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Starting download of 0 data blocks...
I20250109 08:58:12.960062 23117 tablet_bootstrap.cc:492] T af330dcd4767423ea4c179be7f88d0c4 P 88935ab835544e9889d05a6587d8cf21: Bootstrap complete.
I20250109 08:58:12.961872 23119 tablet_copy_client.cc:670] T e38ec0ef84af4b798073eafd4d210c35 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Starting download of 1 WAL segments...
W20250109 08:58:12.966830 21460 auto_rebalancer.cc:663] Could not move replica: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20250109 08:58:12.967244 21460 auto_rebalancer.cc:264] scheduled replica move failed to complete: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20250109 08:58:12.965551 23117 ts_tablet_manager.cc:1397] T af330dcd4767423ea4c179be7f88d0c4 P 88935ab835544e9889d05a6587d8cf21: Time spent bootstrapping tablet: real 0.069s user 0.016s sys 0.012s
I20250109 08:58:12.982635 20363 tablet_copy_service.cc:362] P 8bf6e26e98b14566899578a990b27d47: Destroying tablet copy session 88935ab835544e9889d05a6587d8cf21-e38ec0ef84af4b798073eafd4d210c35 due to service shutdown
W20250109 08:58:12.982981 21626 consensus_peers.cc:487] T 79c1ed8ab551478291f1f24cc940cb27 P 787ddcc824c2490a808fc5a53452dc5e -> Peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Couldn't send request to peer 8bf6e26e98b14566899578a990b27d47. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20250109 08:58:12.983618 21460 auto_rebalancer.cc:663] Could not move replica: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20250109 08:58:12.983942 21460 auto_rebalancer.cc:264] scheduled replica move failed to complete: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20250109 08:58:12.984934 23117 raft_consensus.cc:357] T af330dcd4767423ea4c179be7f88d0c4 P 88935ab835544e9889d05a6587d8cf21 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:58:12.986581 23117 raft_consensus.cc:738] T af330dcd4767423ea4c179be7f88d0c4 P 88935ab835544e9889d05a6587d8cf21 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 88935ab835544e9889d05a6587d8cf21, State: Initialized, Role: LEARNER
I20250109 08:58:12.987316 23117 consensus_queue.cc:260] T af330dcd4767423ea4c179be7f88d0c4 P 88935ab835544e9889d05a6587d8cf21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:58:12.983037 20363 tablet_copy_service.cc:434] P 8bf6e26e98b14566899578a990b27d47: ending tablet copy session 88935ab835544e9889d05a6587d8cf21-e38ec0ef84af4b798073eafd4d210c35 on tablet e38ec0ef84af4b798073eafd4d210c35 with peer 88935ab835544e9889d05a6587d8cf21
I20250109 08:58:12.988550 20363 tablet_copy_service.cc:362] P 8bf6e26e98b14566899578a990b27d47: Destroying tablet copy session 88935ab835544e9889d05a6587d8cf21-363fa693a5114f2bb22f4ab67722a1c6 due to service shutdown
I20250109 08:58:12.988901 20363 tablet_copy_service.cc:434] P 8bf6e26e98b14566899578a990b27d47: ending tablet copy session 88935ab835544e9889d05a6587d8cf21-363fa693a5114f2bb22f4ab67722a1c6 on tablet 363fa693a5114f2bb22f4ab67722a1c6 with peer 88935ab835544e9889d05a6587d8cf21
I20250109 08:58:12.998024 22173 raft_consensus.cc:1212] T 8c0459d2eee64f0a83bd05ad5974f506 P 88935ab835544e9889d05a6587d8cf21 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3] Dedup: 2.3->[]
I20250109 08:58:13.007800 23117 ts_tablet_manager.cc:1428] T af330dcd4767423ea4c179be7f88d0c4 P 88935ab835544e9889d05a6587d8cf21: Time spent starting tablet: real 0.039s user 0.005s sys 0.000s
I20250109 08:58:13.014072 21673 tablet_copy_service.cc:342] P 787ddcc824c2490a808fc5a53452dc5e: Request end of tablet copy session 88935ab835544e9889d05a6587d8cf21-af330dcd4767423ea4c179be7f88d0c4 received from {username='slave'} at 127.0.0.1:52376
I20250109 08:58:13.014565 21673 tablet_copy_service.cc:434] P 787ddcc824c2490a808fc5a53452dc5e: ending tablet copy session 88935ab835544e9889d05a6587d8cf21-af330dcd4767423ea4c179be7f88d0c4 on tablet af330dcd4767423ea4c179be7f88d0c4 with peer 88935ab835544e9889d05a6587d8cf21
I20250109 08:58:13.052173 23113 tablet_bootstrap.cc:492] T 363fa693a5114f2bb22f4ab67722a1c6 P 88935ab835544e9889d05a6587d8cf21: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250109 08:58:13.053385 23113 tablet_bootstrap.cc:492] T 363fa693a5114f2bb22f4ab67722a1c6 P 88935ab835544e9889d05a6587d8cf21: Bootstrap complete.
I20250109 08:58:13.054100 23113 ts_tablet_manager.cc:1397] T 363fa693a5114f2bb22f4ab67722a1c6 P 88935ab835544e9889d05a6587d8cf21: Time spent bootstrapping tablet: real 0.112s user 0.025s sys 0.000s
I20250109 08:58:13.060329 23113 raft_consensus.cc:357] T 363fa693a5114f2bb22f4ab67722a1c6 P 88935ab835544e9889d05a6587d8cf21 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:58:13.062131 23113 raft_consensus.cc:738] T 363fa693a5114f2bb22f4ab67722a1c6 P 88935ab835544e9889d05a6587d8cf21 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 88935ab835544e9889d05a6587d8cf21, State: Initialized, Role: LEARNER
I20250109 08:58:13.062853 23113 consensus_queue.cc:260] T 363fa693a5114f2bb22f4ab67722a1c6 P 88935ab835544e9889d05a6587d8cf21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
W20250109 08:58:13.055827 21626 consensus_peers.cc:487] T 7a7211b8d4374ad1afd82e8cf85bad81 P 787ddcc824c2490a808fc5a53452dc5e -> Peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Couldn't send request to peer 8bf6e26e98b14566899578a990b27d47. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20250109 08:58:13.072889 22141 consensus_peers.cc:487] T 5717a02ef53a4b22a96f431ca6c20d93 P 88935ab835544e9889d05a6587d8cf21 -> Peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Couldn't send request to peer 8bf6e26e98b14566899578a990b27d47. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:13.095111 23113 ts_tablet_manager.cc:1428] T 363fa693a5114f2bb22f4ab67722a1c6 P 88935ab835544e9889d05a6587d8cf21: Time spent starting tablet: real 0.041s user 0.007s sys 0.000s
W20250109 08:58:13.096580 22141 consensus_peers.cc:487] T 239ca74a4c0a455a9db2652d7debb989 P 88935ab835544e9889d05a6587d8cf21 -> Peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Couldn't send request to peer 8bf6e26e98b14566899578a990b27d47. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20250109 08:58:13.095656 21626 consensus_peers.cc:487] T af273bab41cc4a9da780147eebfd52ec P 787ddcc824c2490a808fc5a53452dc5e -> Peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Couldn't send request to peer 8bf6e26e98b14566899578a990b27d47. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20250109 08:58:13.112597 22141 consensus_peers.cc:487] T 206785f29c824cddbe9bbc00d365ffb3 P 88935ab835544e9889d05a6587d8cf21 -> Peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Couldn't send request to peer 8bf6e26e98b14566899578a990b27d47. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20250109 08:58:13.113539 22141 consensus_peers.cc:487] T 1b582ab8d8f044d9b9038a28e958feeb P 88935ab835544e9889d05a6587d8cf21 -> Peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Couldn't send request to peer 8bf6e26e98b14566899578a990b27d47. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20250109 08:58:13.114392 22141 consensus_peers.cc:487] T 20ea30dba4314290ae45d9b4c13bebdd P 88935ab835544e9889d05a6587d8cf21 -> Peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Couldn't send request to peer 8bf6e26e98b14566899578a990b27d47. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20250109 08:58:13.128906 21626 consensus_peers.cc:487] T 80123146836349938344efc61333695f P 787ddcc824c2490a808fc5a53452dc5e -> Peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Couldn't send request to peer 8bf6e26e98b14566899578a990b27d47. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20250109 08:58:13.133469 21552 consensus_peers.cc:487] T 90978f185a374feba3bc19b54f9d97f5 P a2b1a692435241ee988e0bd1661cd4c3 -> Peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Couldn't send request to peer 8bf6e26e98b14566899578a990b27d47. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20250109 08:58:13.134071 21552 consensus_peers.cc:487] T 56bbc306275e4a409e028a7ca1dd1602 P a2b1a692435241ee988e0bd1661cd4c3 -> Peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Couldn't send request to peer 8bf6e26e98b14566899578a990b27d47. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20250109 08:58:13.134534 21552 consensus_peers.cc:487] T f961d3f357944118a69f905621731552 P a2b1a692435241ee988e0bd1661cd4c3 -> Peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Couldn't send request to peer 8bf6e26e98b14566899578a990b27d47. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20250109 08:58:13.137235 22141 consensus_peers.cc:487] T 2935d69446cd458e92046d2b2f1b74ab P 88935ab835544e9889d05a6587d8cf21 -> Peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Couldn't send request to peer 8bf6e26e98b14566899578a990b27d47. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20250109 08:58:13.139621 21626 consensus_peers.cc:487] T 5ac5ffe3cd65463fb448e819e3978841 P 787ddcc824c2490a808fc5a53452dc5e -> Peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Couldn't send request to peer 8bf6e26e98b14566899578a990b27d47. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:13.139813 20363 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250109 08:58:13.143169 20363 tablet_replica.cc:331] T 2935d69446cd458e92046d2b2f1b74ab P 8bf6e26e98b14566899578a990b27d47: stopping tablet replica
I20250109 08:58:13.144062 20363 raft_consensus.cc:2238] T 2935d69446cd458e92046d2b2f1b74ab P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:13.144721 20363 raft_consensus.cc:2267] T 2935d69446cd458e92046d2b2f1b74ab P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:13.147573 20363 tablet_replica.cc:331] T edc9db96e0ff4aa5839a6922f3eb5f24 P 8bf6e26e98b14566899578a990b27d47: stopping tablet replica
I20250109 08:58:13.148387 20363 raft_consensus.cc:2238] T edc9db96e0ff4aa5839a6922f3eb5f24 P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Raft consensus shutting down.
I20250109 08:58:13.149398 20363 raft_consensus.cc:2267] T edc9db96e0ff4aa5839a6922f3eb5f24 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:13.151701 20363 tablet_replica.cc:331] T 7c19ed9fdde5463f84e43bea7b860d03 P 8bf6e26e98b14566899578a990b27d47: stopping tablet replica
I20250109 08:58:13.152577 20363 raft_consensus.cc:2238] T 7c19ed9fdde5463f84e43bea7b860d03 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:13.153291 20363 raft_consensus.cc:2267] T 7c19ed9fdde5463f84e43bea7b860d03 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:13.155671 20363 tablet_replica.cc:331] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 8bf6e26e98b14566899578a990b27d47: stopping tablet replica
I20250109 08:58:13.158792 20363 raft_consensus.cc:2238] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Raft consensus shutting down.
I20250109 08:58:13.159797 20363 raft_consensus.cc:2267] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:13.162310 20363 tablet_replica.cc:331] T 5717a02ef53a4b22a96f431ca6c20d93 P 8bf6e26e98b14566899578a990b27d47: stopping tablet replica
I20250109 08:58:13.163126 20363 raft_consensus.cc:2238] T 5717a02ef53a4b22a96f431ca6c20d93 P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:13.163638 20363 raft_consensus.cc:2267] T 5717a02ef53a4b22a96f431ca6c20d93 P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:13.166116 20363 tablet_replica.cc:331] T 1b582ab8d8f044d9b9038a28e958feeb P 8bf6e26e98b14566899578a990b27d47: stopping tablet replica
I20250109 08:58:13.166920 20363 raft_consensus.cc:2238] T 1b582ab8d8f044d9b9038a28e958feeb P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:13.167675 20363 raft_consensus.cc:2267] T 1b582ab8d8f044d9b9038a28e958feeb P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:13.173363 20363 tablet_replica.cc:331] T af330dcd4767423ea4c179be7f88d0c4 P 8bf6e26e98b14566899578a990b27d47: stopping tablet replica
I20250109 08:58:13.174211 20363 raft_consensus.cc:2238] T af330dcd4767423ea4c179be7f88d0c4 P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:13.174860 20363 raft_consensus.cc:2267] T af330dcd4767423ea4c179be7f88d0c4 P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Raft consensus is shut down!
W20250109 08:58:13.177834 22141 consensus_peers.cc:487] T 37280454805d47f98a415a8f0c16d869 P 88935ab835544e9889d05a6587d8cf21 -> Peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Couldn't send request to peer 8bf6e26e98b14566899578a990b27d47. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20250109 08:58:13.183070 21552 consensus_peers.cc:487] T 126a163a89c448be89ccb9490e9f0168 P a2b1a692435241ee988e0bd1661cd4c3 -> Peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Couldn't send request to peer 8bf6e26e98b14566899578a990b27d47. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20250109 08:58:13.184981 21552 consensus_peers.cc:487] T c18c1d46a6ca47598d7b161ed70a4e7a P a2b1a692435241ee988e0bd1661cd4c3 -> Peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Couldn't send request to peer 8bf6e26e98b14566899578a990b27d47. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:13.188146 20363 tablet_replica.cc:331] T 71ac0243d72845659291184835652dde P 8bf6e26e98b14566899578a990b27d47: stopping tablet replica
I20250109 08:58:13.189052 20363 raft_consensus.cc:2238] T 71ac0243d72845659291184835652dde P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Raft consensus shutting down.
I20250109 08:58:13.190106 20363 raft_consensus.cc:2267] T 71ac0243d72845659291184835652dde P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:13.194234 20363 tablet_replica.cc:331] T f961d3f357944118a69f905621731552 P 8bf6e26e98b14566899578a990b27d47: stopping tablet replica
W20250109 08:58:13.194768 21552 consensus_peers.cc:487] T 8c0459d2eee64f0a83bd05ad5974f506 P a2b1a692435241ee988e0bd1661cd4c3 -> Peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Couldn't send request to peer 8bf6e26e98b14566899578a990b27d47. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:13.194974 20363 raft_consensus.cc:2238] T f961d3f357944118a69f905621731552 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:13.195581 20363 raft_consensus.cc:2267] T f961d3f357944118a69f905621731552 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:13.198567 20363 tablet_replica.cc:331] T 56bbc306275e4a409e028a7ca1dd1602 P 8bf6e26e98b14566899578a990b27d47: stopping tablet replica
I20250109 08:58:13.199214 20363 raft_consensus.cc:2238] T 56bbc306275e4a409e028a7ca1dd1602 P 8bf6e26e98b14566899578a990b27d47 [term 3 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:13.199901 20363 raft_consensus.cc:2267] T 56bbc306275e4a409e028a7ca1dd1602 P 8bf6e26e98b14566899578a990b27d47 [term 3 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:13.202466 20363 tablet_replica.cc:331] T db34a00c02b84ee988edc7d6e344c1f5 P 8bf6e26e98b14566899578a990b27d47: stopping tablet replica
I20250109 08:58:13.203227 20363 raft_consensus.cc:2238] T db34a00c02b84ee988edc7d6e344c1f5 P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Raft consensus shutting down.
I20250109 08:58:13.204190 20363 raft_consensus.cc:2267] T db34a00c02b84ee988edc7d6e344c1f5 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:13.206663 20363 tablet_replica.cc:331] T 8c0459d2eee64f0a83bd05ad5974f506 P 8bf6e26e98b14566899578a990b27d47: stopping tablet replica
I20250109 08:58:13.207460 20363 raft_consensus.cc:2238] T 8c0459d2eee64f0a83bd05ad5974f506 P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:13.207974 20363 raft_consensus.cc:2267] T 8c0459d2eee64f0a83bd05ad5974f506 P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:13.210116 20363 tablet_replica.cc:331] T 37280454805d47f98a415a8f0c16d869 P 8bf6e26e98b14566899578a990b27d47: stopping tablet replica
I20250109 08:58:13.210772 20363 raft_consensus.cc:2238] T 37280454805d47f98a415a8f0c16d869 P 8bf6e26e98b14566899578a990b27d47 [term 3 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:13.211369 20363 raft_consensus.cc:2267] T 37280454805d47f98a415a8f0c16d869 P 8bf6e26e98b14566899578a990b27d47 [term 3 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:13.213882 20363 tablet_replica.cc:331] T 239ca74a4c0a455a9db2652d7debb989 P 8bf6e26e98b14566899578a990b27d47: stopping tablet replica
I20250109 08:58:13.214584 20363 raft_consensus.cc:2238] T 239ca74a4c0a455a9db2652d7debb989 P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:13.215289 20363 raft_consensus.cc:2267] T 239ca74a4c0a455a9db2652d7debb989 P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:13.217484 20363 tablet_replica.cc:331] T 363fa693a5114f2bb22f4ab67722a1c6 P 8bf6e26e98b14566899578a990b27d47: stopping tablet replica
I20250109 08:58:13.218216 20363 raft_consensus.cc:2238] T 363fa693a5114f2bb22f4ab67722a1c6 P 8bf6e26e98b14566899578a990b27d47 [term 2 LEADER]: Raft consensus shutting down.
W20250109 08:58:13.218652 21626 consensus_peers.cc:487] T 75edb5d04fa14201ad711c1d95c81f64 P 787ddcc824c2490a808fc5a53452dc5e -> Peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Couldn't send request to peer 8bf6e26e98b14566899578a990b27d47. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:13.219385 20363 raft_consensus.cc:2267] T 363fa693a5114f2bb22f4ab67722a1c6 P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:13.222553 20363 tablet_replica.cc:331] T af273bab41cc4a9da780147eebfd52ec P 8bf6e26e98b14566899578a990b27d47: stopping tablet replica
I20250109 08:58:13.223387 20363 raft_consensus.cc:2238] T af273bab41cc4a9da780147eebfd52ec P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:13.223872 20363 raft_consensus.cc:2267] T af273bab41cc4a9da780147eebfd52ec P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:13.226073 20363 tablet_replica.cc:331] T ab6c3fcdf226433c9e1303004f6c96b4 P 8bf6e26e98b14566899578a990b27d47: stopping tablet replica
I20250109 08:58:13.226905 20363 raft_consensus.cc:2238] T ab6c3fcdf226433c9e1303004f6c96b4 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:13.227427 20363 raft_consensus.cc:2267] T ab6c3fcdf226433c9e1303004f6c96b4 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:13.229146 20363 tablet_replica.cc:331] T 126a163a89c448be89ccb9490e9f0168 P 8bf6e26e98b14566899578a990b27d47: stopping tablet replica
I20250109 08:58:13.229645 20363 raft_consensus.cc:2238] T 126a163a89c448be89ccb9490e9f0168 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:13.230124 20363 raft_consensus.cc:2267] T 126a163a89c448be89ccb9490e9f0168 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:13.231972 20363 tablet_replica.cc:331] T d8493108cba04514bd2236192cef1079 P 8bf6e26e98b14566899578a990b27d47: stopping tablet replica
I20250109 08:58:13.232728 20363 raft_consensus.cc:2238] T d8493108cba04514bd2236192cef1079 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:13.233273 20363 raft_consensus.cc:2267] T d8493108cba04514bd2236192cef1079 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:13.235234 20363 tablet_replica.cc:331] T a3aa271f2f0144eea9261e0e07ab833b P 8bf6e26e98b14566899578a990b27d47: stopping tablet replica
I20250109 08:58:13.235862 20363 raft_consensus.cc:2238] T a3aa271f2f0144eea9261e0e07ab833b P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:13.236356 20363 raft_consensus.cc:2267] T a3aa271f2f0144eea9261e0e07ab833b P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:13.238404 20363 tablet_replica.cc:331] T df9715570a6844d9a2686307db16eff1 P 8bf6e26e98b14566899578a990b27d47: stopping tablet replica
I20250109 08:58:13.239091 20363 raft_consensus.cc:2238] T df9715570a6844d9a2686307db16eff1 P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:13.239611 20363 raft_consensus.cc:2267] T df9715570a6844d9a2686307db16eff1 P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:13.241922 20363 tablet_replica.cc:331] T c99aacce09d44f889be5480854a22760 P 8bf6e26e98b14566899578a990b27d47: stopping tablet replica
I20250109 08:58:13.242621 20363 raft_consensus.cc:2238] T c99aacce09d44f889be5480854a22760 P 8bf6e26e98b14566899578a990b27d47 [term 2 LEADER]: Raft consensus shutting down.
W20250109 08:58:13.243364 21552 consensus_peers.cc:487] T d6f988a231a74f6fa112da9a527d4a0b P a2b1a692435241ee988e0bd1661cd4c3 -> Peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Couldn't send request to peer 8bf6e26e98b14566899578a990b27d47. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:13.243532 20363 raft_consensus.cc:2267] T c99aacce09d44f889be5480854a22760 P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:13.246052 20363 tablet_replica.cc:331] T 0cc4fcda876242c396b804edebebb01f P 8bf6e26e98b14566899578a990b27d47: stopping tablet replica
I20250109 08:58:13.246820 20363 raft_consensus.cc:2238] T 0cc4fcda876242c396b804edebebb01f P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:13.247288 20363 raft_consensus.cc:2267] T 0cc4fcda876242c396b804edebebb01f P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:13.249706 20363 tablet_replica.cc:331] T 75edb5d04fa14201ad711c1d95c81f64 P 8bf6e26e98b14566899578a990b27d47: stopping tablet replica
I20250109 08:58:13.250799 20363 raft_consensus.cc:2238] T 75edb5d04fa14201ad711c1d95c81f64 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:13.251263 20363 raft_consensus.cc:2267] T 75edb5d04fa14201ad711c1d95c81f64 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:13.253551 20363 tablet_replica.cc:331] T 89e1eeb24fea483eb63bd98af66e0b6b P 8bf6e26e98b14566899578a990b27d47: stopping tablet replica
W20250109 08:58:13.253779 21552 consensus_peers.cc:487] T 3b5049c00b0b4dd89c49fa5d13db169e P a2b1a692435241ee988e0bd1661cd4c3 -> Peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Couldn't send request to peer 8bf6e26e98b14566899578a990b27d47. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:13.254719 20363 raft_consensus.cc:2238] T 89e1eeb24fea483eb63bd98af66e0b6b P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:13.255776 20363 raft_consensus.cc:2267] T 89e1eeb24fea483eb63bd98af66e0b6b P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Raft consensus is shut down!
W20250109 08:58:13.256561 21626 consensus_peers.cc:487] T a3aa271f2f0144eea9261e0e07ab833b P 787ddcc824c2490a808fc5a53452dc5e -> Peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Couldn't send request to peer 8bf6e26e98b14566899578a990b27d47. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:13.258252 20363 tablet_replica.cc:331] T e8ff570914d74df8b87296010e9b864f P 8bf6e26e98b14566899578a990b27d47: stopping tablet replica
I20250109 08:58:13.259116 20363 raft_consensus.cc:2238] T e8ff570914d74df8b87296010e9b864f P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Raft consensus shutting down.
I20250109 08:58:13.260068 20363 raft_consensus.cc:2267] T e8ff570914d74df8b87296010e9b864f P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:13.262344 20363 tablet_replica.cc:331] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P 8bf6e26e98b14566899578a990b27d47: stopping tablet replica
I20250109 08:58:13.263037 20363 raft_consensus.cc:2238] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:13.263653 20363 raft_consensus.cc:2267] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:13.266301 20363 tablet_replica.cc:331] T e03846e461974d0d8944c344e715b86a P 8bf6e26e98b14566899578a990b27d47: stopping tablet replica
I20250109 08:58:13.267010 20363 raft_consensus.cc:2238] T e03846e461974d0d8944c344e715b86a P 8bf6e26e98b14566899578a990b27d47 [term 2 LEADER]: Raft consensus shutting down.
I20250109 08:58:13.267936 20363 raft_consensus.cc:2267] T e03846e461974d0d8944c344e715b86a P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:13.270107 20363 tablet_replica.cc:331] T 79c1ed8ab551478291f1f24cc940cb27 P 8bf6e26e98b14566899578a990b27d47: stopping tablet replica
I20250109 08:58:13.270814 20363 raft_consensus.cc:2238] T 79c1ed8ab551478291f1f24cc940cb27 P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:13.271281 20363 raft_consensus.cc:2267] T 79c1ed8ab551478291f1f24cc940cb27 P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:13.273761 20363 tablet_replica.cc:331] T d6f988a231a74f6fa112da9a527d4a0b P 8bf6e26e98b14566899578a990b27d47: stopping tablet replica
I20250109 08:58:13.274412 20363 raft_consensus.cc:2238] T d6f988a231a74f6fa112da9a527d4a0b P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:13.274963 20363 raft_consensus.cc:2267] T d6f988a231a74f6fa112da9a527d4a0b P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:13.277454 20363 tablet_replica.cc:331] T 20ea30dba4314290ae45d9b4c13bebdd P 8bf6e26e98b14566899578a990b27d47: stopping tablet replica
I20250109 08:58:13.278275 20363 raft_consensus.cc:2238] T 20ea30dba4314290ae45d9b4c13bebdd P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:13.279033 20363 raft_consensus.cc:2267] T 20ea30dba4314290ae45d9b4c13bebdd P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:13.281644 20363 tablet_replica.cc:331] T 3b5049c00b0b4dd89c49fa5d13db169e P 8bf6e26e98b14566899578a990b27d47: stopping tablet replica
I20250109 08:58:13.282330 20363 raft_consensus.cc:2238] T 3b5049c00b0b4dd89c49fa5d13db169e P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:13.283038 20363 raft_consensus.cc:2267] T 3b5049c00b0b4dd89c49fa5d13db169e P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:13.285275 20363 tablet_replica.cc:331] T 206785f29c824cddbe9bbc00d365ffb3 P 8bf6e26e98b14566899578a990b27d47: stopping tablet replica
I20250109 08:58:13.286013 20363 raft_consensus.cc:2238] T 206785f29c824cddbe9bbc00d365ffb3 P 8bf6e26e98b14566899578a990b27d47 [term 3 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:13.286509 20363 raft_consensus.cc:2267] T 206785f29c824cddbe9bbc00d365ffb3 P 8bf6e26e98b14566899578a990b27d47 [term 3 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:13.289410 20363 tablet_replica.cc:331] T 5ac5ffe3cd65463fb448e819e3978841 P 8bf6e26e98b14566899578a990b27d47: stopping tablet replica
I20250109 08:58:13.290129 20363 raft_consensus.cc:2238] T 5ac5ffe3cd65463fb448e819e3978841 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:13.290763 20363 raft_consensus.cc:2267] T 5ac5ffe3cd65463fb448e819e3978841 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:13.293058 20363 tablet_replica.cc:331] T 90978f185a374feba3bc19b54f9d97f5 P 8bf6e26e98b14566899578a990b27d47: stopping tablet replica
I20250109 08:58:13.293777 20363 raft_consensus.cc:2238] T 90978f185a374feba3bc19b54f9d97f5 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:13.294503 20363 raft_consensus.cc:2267] T 90978f185a374feba3bc19b54f9d97f5 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:13.296818 20363 tablet_replica.cc:331] T 180b4d539a144a38905bf340ecb24ef4 P 8bf6e26e98b14566899578a990b27d47: stopping tablet replica
I20250109 08:58:13.297570 20363 raft_consensus.cc:2238] T 180b4d539a144a38905bf340ecb24ef4 P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Raft consensus shutting down.
I20250109 08:58:13.298496 20363 raft_consensus.cc:2267] T 180b4d539a144a38905bf340ecb24ef4 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:13.301609 20363 tablet_replica.cc:331] T c761cb691bd241ae825917a3fbc3ad24 P 8bf6e26e98b14566899578a990b27d47: stopping tablet replica
I20250109 08:58:13.302315 20363 raft_consensus.cc:2238] T c761cb691bd241ae825917a3fbc3ad24 P 8bf6e26e98b14566899578a990b27d47 [term 2 LEADER]: Raft consensus shutting down.
I20250109 08:58:13.303263 20363 raft_consensus.cc:2267] T c761cb691bd241ae825917a3fbc3ad24 P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:13.305588 20363 tablet_replica.cc:331] T 97304b06c7fa40ef8345a72c796c47f8 P 8bf6e26e98b14566899578a990b27d47: stopping tablet replica
I20250109 08:58:13.306301 20363 raft_consensus.cc:2238] T 97304b06c7fa40ef8345a72c796c47f8 P 8bf6e26e98b14566899578a990b27d47 [term 2 LEADER]: Raft consensus shutting down.
I20250109 08:58:13.307278 20363 raft_consensus.cc:2267] T 97304b06c7fa40ef8345a72c796c47f8 P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:13.309767 20363 tablet_replica.cc:331] T 7a7211b8d4374ad1afd82e8cf85bad81 P 8bf6e26e98b14566899578a990b27d47: stopping tablet replica
I20250109 08:58:13.310496 20363 raft_consensus.cc:2238] T 7a7211b8d4374ad1afd82e8cf85bad81 P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:13.311061 20363 raft_consensus.cc:2267] T 7a7211b8d4374ad1afd82e8cf85bad81 P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Raft consensus is shut down!
W20250109 08:58:13.312799 21626 consensus_peers.cc:487] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P 787ddcc824c2490a808fc5a53452dc5e -> Peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Couldn't send request to peer 8bf6e26e98b14566899578a990b27d47. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:13.313655 20363 tablet_replica.cc:331] T e601a034e10e418bac8683443d86a1c1 P 8bf6e26e98b14566899578a990b27d47: stopping tablet replica
I20250109 08:58:13.314473 20363 raft_consensus.cc:2238] T e601a034e10e418bac8683443d86a1c1 P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Raft consensus shutting down.
I20250109 08:58:13.315479 20363 raft_consensus.cc:2267] T e601a034e10e418bac8683443d86a1c1 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:13.317935 20363 tablet_replica.cc:331] T ddd14cdbefd64936960e9ffb5b38b5f2 P 8bf6e26e98b14566899578a990b27d47: stopping tablet replica
I20250109 08:58:13.318686 20363 raft_consensus.cc:2238] T ddd14cdbefd64936960e9ffb5b38b5f2 P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Raft consensus shutting down.
I20250109 08:58:13.319645 20363 raft_consensus.cc:2267] T ddd14cdbefd64936960e9ffb5b38b5f2 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:13.321952 20363 tablet_replica.cc:331] T 80123146836349938344efc61333695f P 8bf6e26e98b14566899578a990b27d47: stopping tablet replica
I20250109 08:58:13.322643 20363 raft_consensus.cc:2238] T 80123146836349938344efc61333695f P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:13.323154 20363 raft_consensus.cc:2267] T 80123146836349938344efc61333695f P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:13.325635 20363 tablet_replica.cc:331] T c18c1d46a6ca47598d7b161ed70a4e7a P 8bf6e26e98b14566899578a990b27d47: stopping tablet replica
I20250109 08:58:13.326274 20363 raft_consensus.cc:2238] T c18c1d46a6ca47598d7b161ed70a4e7a P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:13.326807 20363 raft_consensus.cc:2267] T c18c1d46a6ca47598d7b161ed70a4e7a P 8bf6e26e98b14566899578a990b27d47 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:13.328943 20363 tablet_replica.cc:331] T e38ec0ef84af4b798073eafd4d210c35 P 8bf6e26e98b14566899578a990b27d47: stopping tablet replica
I20250109 08:58:13.329594 20363 raft_consensus.cc:2238] T e38ec0ef84af4b798073eafd4d210c35 P 8bf6e26e98b14566899578a990b27d47 [term 1 LEADER]: Raft consensus shutting down.
W20250109 08:58:13.330529 21626 consensus_peers.cc:487] T d8493108cba04514bd2236192cef1079 P 787ddcc824c2490a808fc5a53452dc5e -> Peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Couldn't send request to peer 8bf6e26e98b14566899578a990b27d47. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:13.331010 20363 raft_consensus.cc:2267] T e38ec0ef84af4b798073eafd4d210c35 P 8bf6e26e98b14566899578a990b27d47 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:13.334220 20363 tablet_replica.cc:331] T 610441ef4b1d40c284eff3a714e4214e P 8bf6e26e98b14566899578a990b27d47: stopping tablet replica
I20250109 08:58:13.334872 20363 raft_consensus.cc:2238] T 610441ef4b1d40c284eff3a714e4214e P 8bf6e26e98b14566899578a990b27d47 [term 3 LEADER]: Raft consensus shutting down.
I20250109 08:58:13.335923 20363 raft_consensus.cc:2267] T 610441ef4b1d40c284eff3a714e4214e P 8bf6e26e98b14566899578a990b27d47 [term 3 FOLLOWER]: Raft consensus is shut down!
W20250109 08:58:13.349035 21552 consensus_peers.cc:487] T 0cc4fcda876242c396b804edebebb01f P a2b1a692435241ee988e0bd1661cd4c3 -> Peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Couldn't send request to peer 8bf6e26e98b14566899578a990b27d47. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20250109 08:58:13.350476 21626 consensus_peers.cc:487] T af330dcd4767423ea4c179be7f88d0c4 P 787ddcc824c2490a808fc5a53452dc5e -> Peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Couldn't send request to peer 8bf6e26e98b14566899578a990b27d47. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:13.363307 22177 raft_consensus.cc:1212] T af330dcd4767423ea4c179be7f88d0c4 P 88935ab835544e9889d05a6587d8cf21 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3] Dedup: 2.3->[]
W20250109 08:58:13.404897 21626 consensus_peers.cc:487] T 7c19ed9fdde5463f84e43bea7b860d03 P 787ddcc824c2490a808fc5a53452dc5e -> Peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Couldn't send request to peer 8bf6e26e98b14566899578a990b27d47. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20250109 08:58:13.415179 21626 consensus_peers.cc:487] T 89e1eeb24fea483eb63bd98af66e0b6b P 787ddcc824c2490a808fc5a53452dc5e -> Peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Couldn't send request to peer 8bf6e26e98b14566899578a990b27d47. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20250109 08:58:13.425951 22141 consensus_peers.cc:487] T df9715570a6844d9a2686307db16eff1 P 88935ab835544e9889d05a6587d8cf21 -> Peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Couldn't send request to peer 8bf6e26e98b14566899578a990b27d47. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:13.458274 20363 tablet_server.cc:195] TabletServer@127.19.226.193:0 shutdown complete.
W20250109 08:58:13.525700 21626 consensus_peers.cc:487] T ab6c3fcdf226433c9e1303004f6c96b4 P 787ddcc824c2490a808fc5a53452dc5e -> Peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Couldn't send request to peer 8bf6e26e98b14566899578a990b27d47. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.193:42725: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:13.568739 23130 raft_consensus.cc:1059] T 8c0459d2eee64f0a83bd05ad5974f506 P a2b1a692435241ee988e0bd1661cd4c3: attempting to promote NON_VOTER 88935ab835544e9889d05a6587d8cf21 to VOTER
I20250109 08:58:13.571594 23130 consensus_queue.cc:237] T 8c0459d2eee64f0a83bd05ad5974f506 P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:13.583319 22177 raft_consensus.cc:1270] T 8c0459d2eee64f0a83bd05ad5974f506 P 88935ab835544e9889d05a6587d8cf21 [term 2 LEARNER]: Refusing update from remote peer a2b1a692435241ee988e0bd1661cd4c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20250109 08:58:13.584704 21660 raft_consensus.cc:1270] T 8c0459d2eee64f0a83bd05ad5974f506 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Refusing update from remote peer a2b1a692435241ee988e0bd1661cd4c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20250109 08:58:13.585855 23131 consensus_queue.cc:1035] T 8c0459d2eee64f0a83bd05ad5974f506 P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20250109 08:58:13.588104 23113 tablet_copy_client.cc:1131] T 363fa693a5114f2bb22f4ab67722a1c6 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Unable to close tablet copy session: Network error: failure ending tablet copy session: Client connection negotiation failed: client connection to 127.19.226.193:42725: connect: Connection refused (error 111)
I20250109 08:58:13.592224 20363 tablet_server.cc:178] TabletServer@127.19.226.194:0 shutting down...
I20250109 08:58:13.592818 23126 consensus_queue.cc:1035] T 8c0459d2eee64f0a83bd05ad5974f506 P a2b1a692435241ee988e0bd1661cd4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20250109 08:58:13.642230 21627 consensus_peers.cc:487] T af330dcd4767423ea4c179be7f88d0c4 P 787ddcc824c2490a808fc5a53452dc5e -> Peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Couldn't send request to peer a2b1a692435241ee988e0bd1661cd4c3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20250109 08:58:13.642318 22142 consensus_peers.cc:487] T d52f79ecf037457cbcc09d5fa43456cf P 88935ab835544e9889d05a6587d8cf21 -> Peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Couldn't send request to peer a2b1a692435241ee988e0bd1661cd4c3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20250109 08:58:13.652797 21460 auto_rebalancer.cc:663] Could not move replica: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20250109 08:58:13.653158 21460 auto_rebalancer.cc:264] scheduled replica move failed to complete: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20250109 08:58:13.679730 21552 consensus_peers.cc:487] T 8c0459d2eee64f0a83bd05ad5974f506 P a2b1a692435241ee988e0bd1661cd4c3 -> Peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Couldn't send request to peer 8bf6e26e98b14566899578a990b27d47. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.193:42725: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20250109 08:58:13.680866 22142 consensus_peers.cc:487] T 2935d69446cd458e92046d2b2f1b74ab P 88935ab835544e9889d05a6587d8cf21 -> Peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Couldn't send request to peer a2b1a692435241ee988e0bd1661cd4c3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:13.682979 23036 raft_consensus.cc:2949] T 8c0459d2eee64f0a83bd05ad5974f506 P a2b1a692435241ee988e0bd1661cd4c3 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
W20250109 08:58:13.706501 21627 consensus_peers.cc:487] T 0f3bde1a1ccd4c359a2e80feca968453 P 787ddcc824c2490a808fc5a53452dc5e -> Peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Couldn't send request to peer a2b1a692435241ee988e0bd1661cd4c3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20250109 08:58:13.712601 21627 consensus_peers.cc:487] T 75edb5d04fa14201ad711c1d95c81f64 P 787ddcc824c2490a808fc5a53452dc5e -> Peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Couldn't send request to peer a2b1a692435241ee988e0bd1661cd4c3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20250109 08:58:13.712735 22142 consensus_peers.cc:487] T b00671c87aac405a81c6543f4defd722 P 88935ab835544e9889d05a6587d8cf21 -> Peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Couldn't send request to peer a2b1a692435241ee988e0bd1661cd4c3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:13.724182 21655 raft_consensus.cc:2949] T 8c0459d2eee64f0a83bd05ad5974f506 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
W20250109 08:58:13.728849 23119 ts_tablet_manager.cc:1005] T e38ec0ef84af4b798073eafd4d210c35 P 88935ab835544e9889d05a6587d8cf21: Tablet Copy: Unable to fetch data from remote peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Network error: Unable to download WAL segment with seq. number 1: unable to fetch data from remote: Client connection negotiation failed: client connection to 127.19.226.193:42725: connect: Connection refused (error 111)
I20250109 08:58:13.729390 23119 tablet_replica.cc:331] T e38ec0ef84af4b798073eafd4d210c35 P 88935ab835544e9889d05a6587d8cf21: stopping tablet replica
I20250109 08:58:13.729677 23119 raft_consensus.cc:2238] T e38ec0ef84af4b798073eafd4d210c35 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Raft consensus shutting down.
I20250109 08:58:13.730037 23119 raft_consensus.cc:2267] T e38ec0ef84af4b798073eafd4d210c35 P 88935ab835544e9889d05a6587d8cf21 [term 1 LEARNER]: Raft consensus is shut down!
W20250109 08:58:13.735499 23119 tablet_copy_client.cc:1131] T e38ec0ef84af4b798073eafd4d210c35 P 88935ab835544e9889d05a6587d8cf21: tablet copy: Unable to close tablet copy session: Network error: failure ending tablet copy session: Client connection negotiation failed: client connection to 127.19.226.193:42725: connect: Connection refused (error 111)
W20250109 08:58:13.736472 21627 consensus_peers.cc:487] T 80123146836349938344efc61333695f P 787ddcc824c2490a808fc5a53452dc5e -> Peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Couldn't send request to peer a2b1a692435241ee988e0bd1661cd4c3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:13.716786 22177 raft_consensus.cc:2949] T 8c0459d2eee64f0a83bd05ad5974f506 P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } }
W20250109 08:58:13.738005 23119 tablet_metadata.cc:476] failed to load DataDirGroup from superblock: Already present: tried to load directory group for tablet e38ec0ef84af4b798073eafd4d210c35 but one is already registered
I20250109 08:58:13.744323 23119 ts_tablet_manager.cc:1905] T e38ec0ef84af4b798073eafd4d210c35 P 88935ab835544e9889d05a6587d8cf21: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250109 08:58:13.749550 21409 catalog_manager.cc:5526] T 8c0459d2eee64f0a83bd05ad5974f506 P a2b1a692435241ee988e0bd1661cd4c3 reported cstate change: config changed from index 3 to 4, 88935ab835544e9889d05a6587d8cf21 (127.19.226.196) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "a2b1a692435241ee988e0bd1661cd4c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250109 08:58:13.780465 23119 ts_tablet_manager.cc:1918] T e38ec0ef84af4b798073eafd4d210c35 P 88935ab835544e9889d05a6587d8cf21: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.0
I20250109 08:58:13.781970 23119 log.cc:1198] T e38ec0ef84af4b798073eafd4d210c35 P 88935ab835544e9889d05a6587d8cf21: Deleting WAL directory at /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736412987538469-20363-0/minicluster-data/ts-3-root/wals/e38ec0ef84af4b798073eafd4d210c35
W20250109 08:58:13.786188 21627 consensus_peers.cc:487] T 79c1ed8ab551478291f1f24cc940cb27 P 787ddcc824c2490a808fc5a53452dc5e -> Peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Couldn't send request to peer a2b1a692435241ee988e0bd1661cd4c3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:13.809937 20363 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250109 08:58:13.812525 20363 tablet_replica.cc:331] T 56bbc306275e4a409e028a7ca1dd1602 P a2b1a692435241ee988e0bd1661cd4c3: stopping tablet replica
I20250109 08:58:13.813416 20363 raft_consensus.cc:2238] T 56bbc306275e4a409e028a7ca1dd1602 P a2b1a692435241ee988e0bd1661cd4c3 [term 3 LEADER]: Raft consensus shutting down.
I20250109 08:58:13.814494 20363 raft_consensus.cc:2267] T 56bbc306275e4a409e028a7ca1dd1602 P a2b1a692435241ee988e0bd1661cd4c3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:13.817152 20363 tablet_replica.cc:331] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P a2b1a692435241ee988e0bd1661cd4c3: stopping tablet replica
I20250109 08:58:13.818044 20363 raft_consensus.cc:2238] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:13.818605 20363 raft_consensus.cc:2267] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:13.822482 20363 tablet_replica.cc:331] T d015a10f95eb4f19b6d7c53cb201fa80 P a2b1a692435241ee988e0bd1661cd4c3: stopping tablet replica
I20250109 08:58:13.823328 20363 raft_consensus.cc:2238] T d015a10f95eb4f19b6d7c53cb201fa80 P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:13.823952 20363 raft_consensus.cc:2267] T d015a10f95eb4f19b6d7c53cb201fa80 P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:13.826290 20363 tablet_replica.cc:331] T d52f79ecf037457cbcc09d5fa43456cf P a2b1a692435241ee988e0bd1661cd4c3: stopping tablet replica
I20250109 08:58:13.827068 20363 raft_consensus.cc:2238] T d52f79ecf037457cbcc09d5fa43456cf P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:13.827575 20363 raft_consensus.cc:2267] T d52f79ecf037457cbcc09d5fa43456cf P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:13.829995 20363 tablet_replica.cc:331] T 2935d69446cd458e92046d2b2f1b74ab P a2b1a692435241ee988e0bd1661cd4c3: stopping tablet replica
I20250109 08:58:13.830752 20363 raft_consensus.cc:2238] T 2935d69446cd458e92046d2b2f1b74ab P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:13.831364 20363 raft_consensus.cc:2267] T 2935d69446cd458e92046d2b2f1b74ab P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:13.833479 23104 raft_consensus.cc:1059] T af330dcd4767423ea4c179be7f88d0c4 P 787ddcc824c2490a808fc5a53452dc5e: attempting to promote NON_VOTER 88935ab835544e9889d05a6587d8cf21 to VOTER
I20250109 08:58:13.833789 20363 tablet_replica.cc:331] T eb4516b9a16041c2adf34be520768f3f P a2b1a692435241ee988e0bd1661cd4c3: stopping tablet replica
I20250109 08:58:13.834633 20363 raft_consensus.cc:2238] T eb4516b9a16041c2adf34be520768f3f P a2b1a692435241ee988e0bd1661cd4c3 [term 2 LEADER]: Raft consensus shutting down.
I20250109 08:58:13.835583 20363 raft_consensus.cc:2267] T eb4516b9a16041c2adf34be520768f3f P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:13.835453 23104 consensus_queue.cc:237] T af330dcd4767423ea4c179be7f88d0c4 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:13.838173 20363 tablet_replica.cc:331] T 81d74e7d4472497aac5463a4b47e4fb9 P a2b1a692435241ee988e0bd1661cd4c3: stopping tablet replica
I20250109 08:58:13.838966 20363 raft_consensus.cc:2238] T 81d74e7d4472497aac5463a4b47e4fb9 P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:13.839696 20363 raft_consensus.cc:2267] T 81d74e7d4472497aac5463a4b47e4fb9 P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Raft consensus is shut down!
W20250109 08:58:13.841143 21627 consensus_peers.cc:487] T 89e1eeb24fea483eb63bd98af66e0b6b P 787ddcc824c2490a808fc5a53452dc5e -> Peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Couldn't send request to peer a2b1a692435241ee988e0bd1661cd4c3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:13.842232 20363 tablet_replica.cc:331] T ab6c3fcdf226433c9e1303004f6c96b4 P a2b1a692435241ee988e0bd1661cd4c3: stopping tablet replica
I20250109 08:58:13.843139 20363 raft_consensus.cc:2238] T ab6c3fcdf226433c9e1303004f6c96b4 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:13.843611 20363 raft_consensus.cc:2267] T ab6c3fcdf226433c9e1303004f6c96b4 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:13.846092 20363 tablet_replica.cc:331] T 126a163a89c448be89ccb9490e9f0168 P a2b1a692435241ee988e0bd1661cd4c3: stopping tablet replica
I20250109 08:58:13.846822 20363 raft_consensus.cc:2238] T 126a163a89c448be89ccb9490e9f0168 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 LEADER]: Raft consensus shutting down.
I20250109 08:58:13.849005 20363 raft_consensus.cc:2267] T 126a163a89c448be89ccb9490e9f0168 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:13.851229 20363 tablet_replica.cc:331] T 306c4892bbff4dbfbe5a0645294aa627 P a2b1a692435241ee988e0bd1661cd4c3: stopping tablet replica
I20250109 08:58:13.851971 20363 raft_consensus.cc:2238] T 306c4892bbff4dbfbe5a0645294aa627 P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:13.852710 20363 raft_consensus.cc:2267] T 306c4892bbff4dbfbe5a0645294aa627 P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:13.855329 20363 tablet_replica.cc:331] T 7781c00473024307987096b596da12ac P a2b1a692435241ee988e0bd1661cd4c3: stopping tablet replica
I20250109 08:58:13.856142 20363 raft_consensus.cc:2238] T 7781c00473024307987096b596da12ac P a2b1a692435241ee988e0bd1661cd4c3 [term 1 LEADER]: Raft consensus shutting down.
I20250109 08:58:13.857177 20363 raft_consensus.cc:2267] T 7781c00473024307987096b596da12ac P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:13.859798 20363 tablet_replica.cc:331] T 64c667f887ea41b4a484ee565b21fa16 P a2b1a692435241ee988e0bd1661cd4c3: stopping tablet replica
I20250109 08:58:13.860532 20363 raft_consensus.cc:2238] T 64c667f887ea41b4a484ee565b21fa16 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:13.861088 20363 raft_consensus.cc:2267] T 64c667f887ea41b4a484ee565b21fa16 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:13.863348 20363 tablet_replica.cc:331] T af330dcd4767423ea4c179be7f88d0c4 P a2b1a692435241ee988e0bd1661cd4c3: stopping tablet replica
I20250109 08:58:13.864087 20363 raft_consensus.cc:2238] T af330dcd4767423ea4c179be7f88d0c4 P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:13.864666 20363 raft_consensus.cc:2267] T af330dcd4767423ea4c179be7f88d0c4 P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:13.867094 20363 tablet_replica.cc:331] T 71ac0243d72845659291184835652dde P a2b1a692435241ee988e0bd1661cd4c3: stopping tablet replica
I20250109 08:58:13.867839 20363 raft_consensus.cc:2238] T 71ac0243d72845659291184835652dde P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:13.868453 20363 raft_consensus.cc:2267] T 71ac0243d72845659291184835652dde P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:13.870767 20363 tablet_replica.cc:331] T f961d3f357944118a69f905621731552 P a2b1a692435241ee988e0bd1661cd4c3: stopping tablet replica
W20250109 08:58:13.870952 21627 consensus_peers.cc:487] T af330dcd4767423ea4c179be7f88d0c4 P 787ddcc824c2490a808fc5a53452dc5e -> Peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Couldn't send request to peer a2b1a692435241ee988e0bd1661cd4c3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:13.871448 20363 raft_consensus.cc:2238] T f961d3f357944118a69f905621731552 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 LEADER]: Raft consensus shutting down.
I20250109 08:58:13.872336 20363 raft_consensus.cc:2267] T f961d3f357944118a69f905621731552 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:13.873234 22177 raft_consensus.cc:1270] T af330dcd4767423ea4c179be7f88d0c4 P 88935ab835544e9889d05a6587d8cf21 [term 2 LEARNER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20250109 08:58:13.874750 20363 tablet_replica.cc:331] T 0f3bde1a1ccd4c359a2e80feca968453 P a2b1a692435241ee988e0bd1661cd4c3: stopping tablet replica
I20250109 08:58:13.875188 23147 consensus_queue.cc:1035] T af330dcd4767423ea4c179be7f88d0c4 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20250109 08:58:13.875500 20363 raft_consensus.cc:2238] T 0f3bde1a1ccd4c359a2e80feca968453 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:13.881469 20363 raft_consensus.cc:2267] T 0f3bde1a1ccd4c359a2e80feca968453 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Raft consensus is shut down!
W20250109 08:58:13.883214 21626 consensus_peers.cc:487] T af330dcd4767423ea4c179be7f88d0c4 P 787ddcc824c2490a808fc5a53452dc5e -> Peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Couldn't send request to peer 8bf6e26e98b14566899578a990b27d47. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.193:42725: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:13.887045 20363 tablet_replica.cc:331] T c99aacce09d44f889be5480854a22760 P a2b1a692435241ee988e0bd1661cd4c3: stopping tablet replica
I20250109 08:58:13.887879 20363 raft_consensus.cc:2238] T c99aacce09d44f889be5480854a22760 P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:13.888464 20363 raft_consensus.cc:2267] T c99aacce09d44f889be5480854a22760 P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:13.890946 20363 tablet_replica.cc:331] T 0cc4fcda876242c396b804edebebb01f P a2b1a692435241ee988e0bd1661cd4c3: stopping tablet replica
I20250109 08:58:13.891604 20363 raft_consensus.cc:2238] T 0cc4fcda876242c396b804edebebb01f P a2b1a692435241ee988e0bd1661cd4c3 [term 2 LEADER]: Raft consensus shutting down.
I20250109 08:58:13.892418 20363 raft_consensus.cc:2267] T 0cc4fcda876242c396b804edebebb01f P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:13.894665 20363 tablet_replica.cc:331] T 363fa693a5114f2bb22f4ab67722a1c6 P a2b1a692435241ee988e0bd1661cd4c3: stopping tablet replica
I20250109 08:58:13.895349 20363 raft_consensus.cc:2238] T 363fa693a5114f2bb22f4ab67722a1c6 P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:13.895957 20363 raft_consensus.cc:2267] T 363fa693a5114f2bb22f4ab67722a1c6 P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:13.898105 20363 tablet_replica.cc:331] T af273bab41cc4a9da780147eebfd52ec P a2b1a692435241ee988e0bd1661cd4c3: stopping tablet replica
I20250109 08:58:13.898828 20363 raft_consensus.cc:2238] T af273bab41cc4a9da780147eebfd52ec P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:13.899257 20363 raft_consensus.cc:2267] T af273bab41cc4a9da780147eebfd52ec P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:13.901538 20363 tablet_replica.cc:331] T f5f85ad7c7c94acfaf9d550d55fce48c P a2b1a692435241ee988e0bd1661cd4c3: stopping tablet replica
I20250109 08:58:13.902223 20363 raft_consensus.cc:2238] T f5f85ad7c7c94acfaf9d550d55fce48c P a2b1a692435241ee988e0bd1661cd4c3 [term 1 LEADER]: Raft consensus shutting down.
I20250109 08:58:13.906904 20363 raft_consensus.cc:2267] T f5f85ad7c7c94acfaf9d550d55fce48c P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:13.908836 20363 tablet_replica.cc:331] T 90a4a2381cbe40f0b204be325ee9fea1 P a2b1a692435241ee988e0bd1661cd4c3: stopping tablet replica
I20250109 08:58:13.909559 20363 raft_consensus.cc:2238] T 90a4a2381cbe40f0b204be325ee9fea1 P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:13.910225 20363 raft_consensus.cc:2267] T 90a4a2381cbe40f0b204be325ee9fea1 P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:13.912566 20363 tablet_replica.cc:331] T 75edb5d04fa14201ad711c1d95c81f64 P a2b1a692435241ee988e0bd1661cd4c3: stopping tablet replica
I20250109 08:58:13.913239 20363 raft_consensus.cc:2238] T 75edb5d04fa14201ad711c1d95c81f64 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:13.913694 20363 raft_consensus.cc:2267] T 75edb5d04fa14201ad711c1d95c81f64 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:13.916571 20363 tablet_replica.cc:331] T a3aa271f2f0144eea9261e0e07ab833b P a2b1a692435241ee988e0bd1661cd4c3: stopping tablet replica
I20250109 08:58:13.917320 20363 raft_consensus.cc:2238] T a3aa271f2f0144eea9261e0e07ab833b P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:13.917780 20363 raft_consensus.cc:2267] T a3aa271f2f0144eea9261e0e07ab833b P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:13.920043 20363 tablet_replica.cc:331] T 3b5049c00b0b4dd89c49fa5d13db169e P a2b1a692435241ee988e0bd1661cd4c3: stopping tablet replica
I20250109 08:58:13.920738 20363 raft_consensus.cc:2238] T 3b5049c00b0b4dd89c49fa5d13db169e P a2b1a692435241ee988e0bd1661cd4c3 [term 1 LEADER]: Raft consensus shutting down.
I20250109 08:58:13.921818 20363 raft_consensus.cc:2267] T 3b5049c00b0b4dd89c49fa5d13db169e P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:13.924170 20363 tablet_replica.cc:331] T 89e1eeb24fea483eb63bd98af66e0b6b P a2b1a692435241ee988e0bd1661cd4c3: stopping tablet replica
I20250109 08:58:13.924882 20363 raft_consensus.cc:2238] T 89e1eeb24fea483eb63bd98af66e0b6b P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:13.925441 20363 raft_consensus.cc:2267] T 89e1eeb24fea483eb63bd98af66e0b6b P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:13.927716 20363 tablet_replica.cc:331] T e03846e461974d0d8944c344e715b86a P a2b1a692435241ee988e0bd1661cd4c3: stopping tablet replica
I20250109 08:58:13.928359 20363 raft_consensus.cc:2238] T e03846e461974d0d8944c344e715b86a P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:13.939874 20363 raft_consensus.cc:2267] T e03846e461974d0d8944c344e715b86a P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:13.942010 20363 tablet_replica.cc:331] T b00671c87aac405a81c6543f4defd722 P a2b1a692435241ee988e0bd1661cd4c3: stopping tablet replica
I20250109 08:58:13.942662 20363 raft_consensus.cc:2238] T b00671c87aac405a81c6543f4defd722 P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:13.943114 20363 raft_consensus.cc:2267] T b00671c87aac405a81c6543f4defd722 P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:13.945102 20363 tablet_replica.cc:331] T 79c1ed8ab551478291f1f24cc940cb27 P a2b1a692435241ee988e0bd1661cd4c3: stopping tablet replica
I20250109 08:58:13.945801 20363 raft_consensus.cc:2238] T 79c1ed8ab551478291f1f24cc940cb27 P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:13.946235 20363 raft_consensus.cc:2267] T 79c1ed8ab551478291f1f24cc940cb27 P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:13.948513 20363 tablet_replica.cc:331] T d6f988a231a74f6fa112da9a527d4a0b P a2b1a692435241ee988e0bd1661cd4c3: stopping tablet replica
I20250109 08:58:13.949210 20363 raft_consensus.cc:2238] T d6f988a231a74f6fa112da9a527d4a0b P a2b1a692435241ee988e0bd1661cd4c3 [term 2 LEADER]: Raft consensus shutting down.
I20250109 08:58:13.950254 20363 raft_consensus.cc:2267] T d6f988a231a74f6fa112da9a527d4a0b P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:13.952566 20363 tablet_replica.cc:331] T 1d3d5d2a2c88417a97c9d57262a356dd P a2b1a692435241ee988e0bd1661cd4c3: stopping tablet replica
I20250109 08:58:13.953240 20363 raft_consensus.cc:2238] T 1d3d5d2a2c88417a97c9d57262a356dd P a2b1a692435241ee988e0bd1661cd4c3 [term 2 LEADER]: Raft consensus shutting down.
W20250109 08:58:13.955958 22142 consensus_peers.cc:487] T d015a10f95eb4f19b6d7c53cb201fa80 P 88935ab835544e9889d05a6587d8cf21 -> Peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Couldn't send request to peer a2b1a692435241ee988e0bd1661cd4c3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:13.957096 20363 raft_consensus.cc:2267] T 1d3d5d2a2c88417a97c9d57262a356dd P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:13.959368 20363 tablet_replica.cc:331] T 97304b06c7fa40ef8345a72c796c47f8 P a2b1a692435241ee988e0bd1661cd4c3: stopping tablet replica
I20250109 08:58:13.960060 20363 raft_consensus.cc:2238] T 97304b06c7fa40ef8345a72c796c47f8 P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:13.960619 20363 raft_consensus.cc:2267] T 97304b06c7fa40ef8345a72c796c47f8 P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:13.962903 20363 tablet_replica.cc:331] T 7a7211b8d4374ad1afd82e8cf85bad81 P a2b1a692435241ee988e0bd1661cd4c3: stopping tablet replica
I20250109 08:58:13.963553 20363 raft_consensus.cc:2238] T 7a7211b8d4374ad1afd82e8cf85bad81 P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:13.964035 20363 raft_consensus.cc:2267] T 7a7211b8d4374ad1afd82e8cf85bad81 P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:13.967294 20363 tablet_replica.cc:331] T e601a034e10e418bac8683443d86a1c1 P a2b1a692435241ee988e0bd1661cd4c3: stopping tablet replica
I20250109 08:58:13.968050 20363 raft_consensus.cc:2238] T e601a034e10e418bac8683443d86a1c1 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:13.968763 20363 raft_consensus.cc:2267] T e601a034e10e418bac8683443d86a1c1 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:13.971318 20363 tablet_replica.cc:331] T 0e06021cd6624e50af2c914399ebd6c4 P a2b1a692435241ee988e0bd1661cd4c3: stopping tablet replica
I20250109 08:58:13.971961 20363 raft_consensus.cc:2238] T 0e06021cd6624e50af2c914399ebd6c4 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 LEADER]: Raft consensus shutting down.
I20250109 08:58:13.972831 20363 raft_consensus.cc:2267] T 0e06021cd6624e50af2c914399ebd6c4 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Raft consensus is shut down!
W20250109 08:58:13.974581 21627 consensus_peers.cc:487] T af273bab41cc4a9da780147eebfd52ec P 787ddcc824c2490a808fc5a53452dc5e -> Peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Couldn't send request to peer a2b1a692435241ee988e0bd1661cd4c3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:13.974987 20363 tablet_replica.cc:331] T d355d166de534fb4a50c9f48f2df7469 P a2b1a692435241ee988e0bd1661cd4c3: stopping tablet replica
I20250109 08:58:13.975729 20363 raft_consensus.cc:2238] T d355d166de534fb4a50c9f48f2df7469 P a2b1a692435241ee988e0bd1661cd4c3 [term 2 LEADER]: Raft consensus shutting down.
I20250109 08:58:13.976601 20363 raft_consensus.cc:2267] T d355d166de534fb4a50c9f48f2df7469 P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:13.979015 20363 tablet_replica.cc:331] T 8c0459d2eee64f0a83bd05ad5974f506 P a2b1a692435241ee988e0bd1661cd4c3: stopping tablet replica
I20250109 08:58:13.979669 20363 raft_consensus.cc:2238] T 8c0459d2eee64f0a83bd05ad5974f506 P a2b1a692435241ee988e0bd1661cd4c3 [term 2 LEADER]: Raft consensus shutting down.
I20250109 08:58:13.980840 20363 raft_consensus.cc:2267] T 8c0459d2eee64f0a83bd05ad5974f506 P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:13.983299 20363 tablet_replica.cc:331] T db34a00c02b84ee988edc7d6e344c1f5 P a2b1a692435241ee988e0bd1661cd4c3: stopping tablet replica
I20250109 08:58:13.984124 20363 raft_consensus.cc:2238] T db34a00c02b84ee988edc7d6e344c1f5 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:13.984810 20363 raft_consensus.cc:2267] T db34a00c02b84ee988edc7d6e344c1f5 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:13.987241 20363 tablet_replica.cc:331] T 90978f185a374feba3bc19b54f9d97f5 P a2b1a692435241ee988e0bd1661cd4c3: stopping tablet replica
I20250109 08:58:13.987881 20363 raft_consensus.cc:2238] T 90978f185a374feba3bc19b54f9d97f5 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 LEADER]: Raft consensus shutting down.
I20250109 08:58:13.988791 20363 raft_consensus.cc:2267] T 90978f185a374feba3bc19b54f9d97f5 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:13.990983 20363 tablet_replica.cc:331] T 180b4d539a144a38905bf340ecb24ef4 P a2b1a692435241ee988e0bd1661cd4c3: stopping tablet replica
I20250109 08:58:13.991609 20363 raft_consensus.cc:2238] T 180b4d539a144a38905bf340ecb24ef4 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:13.992252 20363 raft_consensus.cc:2267] T 180b4d539a144a38905bf340ecb24ef4 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:13.994627 20363 tablet_replica.cc:331] T c761cb691bd241ae825917a3fbc3ad24 P a2b1a692435241ee988e0bd1661cd4c3: stopping tablet replica
W20250109 08:58:13.995033 22142 consensus_peers.cc:487] T 90a4a2381cbe40f0b204be325ee9fea1 P 88935ab835544e9889d05a6587d8cf21 -> Peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Couldn't send request to peer a2b1a692435241ee988e0bd1661cd4c3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:13.995283 20363 raft_consensus.cc:2238] T c761cb691bd241ae825917a3fbc3ad24 P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:13.995927 20363 raft_consensus.cc:2267] T c761cb691bd241ae825917a3fbc3ad24 P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:13.998354 20363 tablet_replica.cc:331] T ddd14cdbefd64936960e9ffb5b38b5f2 P a2b1a692435241ee988e0bd1661cd4c3: stopping tablet replica
I20250109 08:58:13.998999 20363 raft_consensus.cc:2238] T ddd14cdbefd64936960e9ffb5b38b5f2 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:13.999466 20363 raft_consensus.cc:2267] T ddd14cdbefd64936960e9ffb5b38b5f2 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:14.001808 20363 tablet_replica.cc:331] T 80123146836349938344efc61333695f P a2b1a692435241ee988e0bd1661cd4c3: stopping tablet replica
I20250109 08:58:14.002463 20363 raft_consensus.cc:2238] T 80123146836349938344efc61333695f P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:14.002938 20363 raft_consensus.cc:2267] T 80123146836349938344efc61333695f P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:14.005141 20363 tablet_replica.cc:331] T e38ec0ef84af4b798073eafd4d210c35 P a2b1a692435241ee988e0bd1661cd4c3: stopping tablet replica
I20250109 08:58:14.005782 20363 raft_consensus.cc:2238] T e38ec0ef84af4b798073eafd4d210c35 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:14.006280 20363 raft_consensus.cc:2267] T e38ec0ef84af4b798073eafd4d210c35 P a2b1a692435241ee988e0bd1661cd4c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:14.008445 20363 tablet_replica.cc:331] T 610441ef4b1d40c284eff3a714e4214e P a2b1a692435241ee988e0bd1661cd4c3: stopping tablet replica
I20250109 08:58:14.009056 20363 raft_consensus.cc:2238] T 610441ef4b1d40c284eff3a714e4214e P a2b1a692435241ee988e0bd1661cd4c3 [term 3 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:14.009542 20363 raft_consensus.cc:2267] T 610441ef4b1d40c284eff3a714e4214e P a2b1a692435241ee988e0bd1661cd4c3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:14.011864 20363 tablet_replica.cc:331] T c18c1d46a6ca47598d7b161ed70a4e7a P a2b1a692435241ee988e0bd1661cd4c3: stopping tablet replica
I20250109 08:58:14.012459 20363 raft_consensus.cc:2238] T c18c1d46a6ca47598d7b161ed70a4e7a P a2b1a692435241ee988e0bd1661cd4c3 [term 2 LEADER]: Raft consensus shutting down.
I20250109 08:58:14.013291 20363 raft_consensus.cc:2267] T c18c1d46a6ca47598d7b161ed70a4e7a P a2b1a692435241ee988e0bd1661cd4c3 [term 2 FOLLOWER]: Raft consensus is shut down!
W20250109 08:58:14.018187 22142 consensus_peers.cc:487] T 306c4892bbff4dbfbe5a0645294aa627 P 88935ab835544e9889d05a6587d8cf21 -> Peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Couldn't send request to peer a2b1a692435241ee988e0bd1661cd4c3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20250109 08:58:14.030084 21627 consensus_peers.cc:487] T 64c667f887ea41b4a484ee565b21fa16 P 787ddcc824c2490a808fc5a53452dc5e -> Peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Couldn't send request to peer a2b1a692435241ee988e0bd1661cd4c3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20250109 08:58:14.036043 21627 consensus_peers.cc:487] T 7a7211b8d4374ad1afd82e8cf85bad81 P 787ddcc824c2490a808fc5a53452dc5e -> Peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Couldn't send request to peer a2b1a692435241ee988e0bd1661cd4c3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20250109 08:58:14.042052 21627 consensus_peers.cc:487] T ab6c3fcdf226433c9e1303004f6c96b4 P 787ddcc824c2490a808fc5a53452dc5e -> Peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Couldn't send request to peer a2b1a692435241ee988e0bd1661cd4c3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20250109 08:58:14.074330 22142 consensus_peers.cc:487] T 81d74e7d4472497aac5463a4b47e4fb9 P 88935ab835544e9889d05a6587d8cf21 -> Peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Couldn't send request to peer a2b1a692435241ee988e0bd1661cd4c3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20250109 08:58:14.075343 21627 consensus_peers.cc:487] T a3aa271f2f0144eea9261e0e07ab833b P 787ddcc824c2490a808fc5a53452dc5e -> Peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Couldn't send request to peer a2b1a692435241ee988e0bd1661cd4c3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:14.215785 23031 raft_consensus.cc:491] T edc9db96e0ff4aa5839a6922f3eb5f24 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8bf6e26e98b14566899578a990b27d47)
I20250109 08:58:14.216347 23031 raft_consensus.cc:513] T edc9db96e0ff4aa5839a6922f3eb5f24 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:14.220490 23031 leader_election.cc:290] T edc9db96e0ff4aa5839a6922f3eb5f24 P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:58:14.222627 21660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "edc9db96e0ff4aa5839a6922f3eb5f24" candidate_uuid: "88935ab835544e9889d05a6587d8cf21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e" is_pre_election: true
I20250109 08:58:14.223114 23128 raft_consensus.cc:491] T db34a00c02b84ee988edc7d6e344c1f5 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8bf6e26e98b14566899578a990b27d47)
I20250109 08:58:14.223600 23128 raft_consensus.cc:513] T db34a00c02b84ee988edc7d6e344c1f5 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:14.225580 23128 leader_election.cc:290] T db34a00c02b84ee988edc7d6e344c1f5 P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
W20250109 08:58:14.227186 22141 leader_election.cc:336] T edc9db96e0ff4aa5839a6922f3eb5f24 P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Network error: Client connection negotiation failed: client connection to 127.19.226.193:42725: connect: Connection refused (error 111)
W20250109 08:58:14.229555 22142 leader_election.cc:336] T db34a00c02b84ee988edc7d6e344c1f5 P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Network error: Client connection negotiation failed: client connection to 127.19.226.194:45527: connect: Connection refused (error 111)
I20250109 08:58:14.237316 22141 leader_election.cc:304] T edc9db96e0ff4aa5839a6922f3eb5f24 P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88935ab835544e9889d05a6587d8cf21; no voters: 787ddcc824c2490a808fc5a53452dc5e, 8bf6e26e98b14566899578a990b27d47
I20250109 08:58:14.238106 23031 raft_consensus.cc:2743] T edc9db96e0ff4aa5839a6922f3eb5f24 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20250109 08:58:14.238629 22141 leader_election.cc:336] T db34a00c02b84ee988edc7d6e344c1f5 P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Network error: Client connection negotiation failed: client connection to 127.19.226.193:42725: connect: Connection refused (error 111)
I20250109 08:58:14.239302 22141 leader_election.cc:304] T db34a00c02b84ee988edc7d6e344c1f5 P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88935ab835544e9889d05a6587d8cf21; no voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:58:14.240006 23031 raft_consensus.cc:2743] T db34a00c02b84ee988edc7d6e344c1f5 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250109 08:58:14.297636 23146 raft_consensus.cc:491] T e38ec0ef84af4b798073eafd4d210c35 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8bf6e26e98b14566899578a990b27d47)
I20250109 08:58:14.298134 23146 raft_consensus.cc:513] T e38ec0ef84af4b798073eafd4d210c35 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:58:14.300156 23146 leader_election.cc:290] T e38ec0ef84af4b798073eafd4d210c35 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
W20250109 08:58:14.303817 21626 leader_election.cc:336] T e38ec0ef84af4b798073eafd4d210c35 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Network error: Client connection negotiation failed: client connection to 127.19.226.193:42725: connect: Connection refused (error 111)
W20250109 08:58:14.309305 21627 leader_election.cc:336] T e38ec0ef84af4b798073eafd4d210c35 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Network error: Client connection negotiation failed: client connection to 127.19.226.194:45527: connect: Connection refused (error 111)
I20250109 08:58:14.309818 21627 leader_election.cc:304] T e38ec0ef84af4b798073eafd4d210c35 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e; no voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:58:14.311055 23121 raft_consensus.cc:2743] T e38ec0ef84af4b798073eafd4d210c35 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250109 08:58:14.326509 23146 raft_consensus.cc:491] T 363fa693a5114f2bb22f4ab67722a1c6 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8bf6e26e98b14566899578a990b27d47)
I20250109 08:58:14.327046 23146 raft_consensus.cc:513] T 363fa693a5114f2bb22f4ab67722a1c6 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:58:14.330582 23146 leader_election.cc:290] T 363fa693a5114f2bb22f4ab67722a1c6 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
W20250109 08:58:14.334940 21626 leader_election.cc:336] T 363fa693a5114f2bb22f4ab67722a1c6 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Network error: Client connection negotiation failed: client connection to 127.19.226.193:42725: connect: Connection refused (error 111)
W20250109 08:58:14.335812 21627 leader_election.cc:336] T 363fa693a5114f2bb22f4ab67722a1c6 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Network error: Client connection negotiation failed: client connection to 127.19.226.194:45527: connect: Connection refused (error 111)
I20250109 08:58:14.336299 21627 leader_election.cc:304] T 363fa693a5114f2bb22f4ab67722a1c6 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e; no voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:58:14.337517 23146 raft_consensus.cc:2743] T 363fa693a5114f2bb22f4ab67722a1c6 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20250109 08:58:14.347851 23031 raft_consensus.cc:491] T 97304b06c7fa40ef8345a72c796c47f8 P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8bf6e26e98b14566899578a990b27d47)
I20250109 08:58:14.348433 23031 raft_consensus.cc:513] T 97304b06c7fa40ef8345a72c796c47f8 P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:14.349064 23104 raft_consensus.cc:491] T e03846e461974d0d8944c344e715b86a P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8bf6e26e98b14566899578a990b27d47)
I20250109 08:58:14.349566 23104 raft_consensus.cc:513] T e03846e461974d0d8944c344e715b86a P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:58:14.350365 23031 leader_election.cc:290] T 97304b06c7fa40ef8345a72c796c47f8 P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
I20250109 08:58:14.351336 23104 leader_election.cc:290] T e03846e461974d0d8944c344e715b86a P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
W20250109 08:58:14.355720 22142 leader_election.cc:336] T 97304b06c7fa40ef8345a72c796c47f8 P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Network error: Client connection negotiation failed: client connection to 127.19.226.194:45527: connect: Connection refused (error 111)
W20250109 08:58:14.356093 21626 leader_election.cc:336] T e03846e461974d0d8944c344e715b86a P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Network error: Client connection negotiation failed: client connection to 127.19.226.193:42725: connect: Connection refused (error 111)
W20250109 08:58:14.361577 22141 leader_election.cc:336] T 97304b06c7fa40ef8345a72c796c47f8 P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Network error: Client connection negotiation failed: client connection to 127.19.226.193:42725: connect: Connection refused (error 111)
I20250109 08:58:14.362020 22141 leader_election.cc:304] T 97304b06c7fa40ef8345a72c796c47f8 P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88935ab835544e9889d05a6587d8cf21; no voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:58:14.362669 23127 raft_consensus.cc:2743] T 97304b06c7fa40ef8345a72c796c47f8 P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20250109 08:58:14.363111 21627 leader_election.cc:336] T e03846e461974d0d8944c344e715b86a P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Network error: Client connection negotiation failed: client connection to 127.19.226.194:45527: connect: Connection refused (error 111)
I20250109 08:58:14.363513 21627 leader_election.cc:304] T e03846e461974d0d8944c344e715b86a P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e; no voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:58:14.364079 23104 raft_consensus.cc:2743] T e03846e461974d0d8944c344e715b86a P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20250109 08:58:14.375124 23146 raft_consensus.cc:491] T c761cb691bd241ae825917a3fbc3ad24 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8bf6e26e98b14566899578a990b27d47)
I20250109 08:58:14.375717 23146 raft_consensus.cc:513] T c761cb691bd241ae825917a3fbc3ad24 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:58:14.377580 23146 leader_election.cc:290] T c761cb691bd241ae825917a3fbc3ad24 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
W20250109 08:58:14.382266 21626 leader_election.cc:336] T c761cb691bd241ae825917a3fbc3ad24 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Network error: Client connection negotiation failed: client connection to 127.19.226.193:42725: connect: Connection refused (error 111)
W20250109 08:58:14.383278 21627 leader_election.cc:336] T c761cb691bd241ae825917a3fbc3ad24 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Network error: Client connection negotiation failed: client connection to 127.19.226.194:45527: connect: Connection refused (error 111)
I20250109 08:58:14.383746 21627 leader_election.cc:304] T c761cb691bd241ae825917a3fbc3ad24 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e; no voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:58:14.384768 23146 raft_consensus.cc:2743] T c761cb691bd241ae825917a3fbc3ad24 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20250109 08:58:14.419983 23031 raft_consensus.cc:491] T e8ff570914d74df8b87296010e9b864f P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8bf6e26e98b14566899578a990b27d47)
I20250109 08:58:14.420539 23031 raft_consensus.cc:513] T e8ff570914d74df8b87296010e9b864f P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:14.422441 23031 leader_election.cc:290] T e8ff570914d74df8b87296010e9b864f P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:58:14.424257 21656 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8ff570914d74df8b87296010e9b864f" candidate_uuid: "88935ab835544e9889d05a6587d8cf21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e" is_pre_election: true
W20250109 08:58:14.427743 22141 leader_election.cc:336] T e8ff570914d74df8b87296010e9b864f P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Network error: Client connection negotiation failed: client connection to 127.19.226.193:42725: connect: Connection refused (error 111)
I20250109 08:58:14.428125 22141 leader_election.cc:304] T e8ff570914d74df8b87296010e9b864f P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88935ab835544e9889d05a6587d8cf21; no voters: 787ddcc824c2490a808fc5a53452dc5e, 8bf6e26e98b14566899578a990b27d47
I20250109 08:58:14.428653 23031 raft_consensus.cc:2743] T e8ff570914d74df8b87296010e9b864f P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250109 08:58:14.431682 23146 raft_consensus.cc:491] T edc9db96e0ff4aa5839a6922f3eb5f24 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8bf6e26e98b14566899578a990b27d47)
I20250109 08:58:14.432102 23146 raft_consensus.cc:513] T edc9db96e0ff4aa5839a6922f3eb5f24 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:14.433787 23146 leader_election.cc:290] T edc9db96e0ff4aa5839a6922f3eb5f24 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), 88935ab835544e9889d05a6587d8cf21 (127.19.226.196:44363)
I20250109 08:58:14.435887 22177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "edc9db96e0ff4aa5839a6922f3eb5f24" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "88935ab835544e9889d05a6587d8cf21" is_pre_election: true
I20250109 08:58:14.437111 22177 raft_consensus.cc:2463] T edc9db96e0ff4aa5839a6922f3eb5f24 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 1.
I20250109 08:58:14.438690 21627 leader_election.cc:304] T edc9db96e0ff4aa5839a6922f3eb5f24 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, 88935ab835544e9889d05a6587d8cf21; no voters:
I20250109 08:58:14.439404 23121 raft_consensus.cc:2798] T edc9db96e0ff4aa5839a6922f3eb5f24 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20250109 08:58:14.439745 23121 raft_consensus.cc:491] T edc9db96e0ff4aa5839a6922f3eb5f24 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8bf6e26e98b14566899578a990b27d47)
W20250109 08:58:14.440032 21626 leader_election.cc:336] T edc9db96e0ff4aa5839a6922f3eb5f24 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Network error: Client connection negotiation failed: client connection to 127.19.226.193:42725: connect: Connection refused (error 111)
I20250109 08:58:14.440070 23121 raft_consensus.cc:3054] T edc9db96e0ff4aa5839a6922f3eb5f24 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:14.447376 23121 raft_consensus.cc:513] T edc9db96e0ff4aa5839a6922f3eb5f24 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:14.449344 23121 leader_election.cc:290] T edc9db96e0ff4aa5839a6922f3eb5f24 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 2 election: Requested vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), 88935ab835544e9889d05a6587d8cf21 (127.19.226.196:44363)
I20250109 08:58:14.450181 22177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "edc9db96e0ff4aa5839a6922f3eb5f24" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "88935ab835544e9889d05a6587d8cf21"
I20250109 08:58:14.450809 22177 raft_consensus.cc:3054] T edc9db96e0ff4aa5839a6922f3eb5f24 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Advancing to term 2
W20250109 08:58:14.453174 21626 leader_election.cc:336] T edc9db96e0ff4aa5839a6922f3eb5f24 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Network error: Client connection negotiation failed: client connection to 127.19.226.193:42725: connect: Connection refused (error 111)
I20250109 08:58:14.457667 22177 raft_consensus.cc:2463] T edc9db96e0ff4aa5839a6922f3eb5f24 P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 2.
I20250109 08:58:14.460579 21627 leader_election.cc:304] T edc9db96e0ff4aa5839a6922f3eb5f24 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, 88935ab835544e9889d05a6587d8cf21; no voters: 8bf6e26e98b14566899578a990b27d47
I20250109 08:58:14.462056 23121 raft_consensus.cc:2798] T edc9db96e0ff4aa5839a6922f3eb5f24 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:58:14.462766 23121 raft_consensus.cc:695] T edc9db96e0ff4aa5839a6922f3eb5f24 P 787ddcc824c2490a808fc5a53452dc5e [term 2 LEADER]: Becoming Leader. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Running, Role: LEADER
I20250109 08:58:14.464421 23121 consensus_queue.cc:237] T edc9db96e0ff4aa5839a6922f3eb5f24 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:14.482882 23147 raft_consensus.cc:491] T e8ff570914d74df8b87296010e9b864f P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8bf6e26e98b14566899578a990b27d47)
I20250109 08:58:14.483382 23147 raft_consensus.cc:513] T e8ff570914d74df8b87296010e9b864f P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:14.485478 23147 leader_election.cc:290] T e8ff570914d74df8b87296010e9b864f P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), 88935ab835544e9889d05a6587d8cf21 (127.19.226.196:44363)
W20250109 08:58:14.490263 21626 leader_election.cc:336] T e8ff570914d74df8b87296010e9b864f P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Network error: Client connection negotiation failed: client connection to 127.19.226.193:42725: connect: Connection refused (error 111)
I20250109 08:58:14.492357 22177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8ff570914d74df8b87296010e9b864f" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "88935ab835544e9889d05a6587d8cf21" is_pre_election: true
I20250109 08:58:14.493047 22177 raft_consensus.cc:2463] T e8ff570914d74df8b87296010e9b864f P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 1.
I20250109 08:58:14.497102 21410 catalog_manager.cc:5526] T edc9db96e0ff4aa5839a6922f3eb5f24 P 787ddcc824c2490a808fc5a53452dc5e reported cstate change: term changed from 1 to 2, leader changed from 8bf6e26e98b14566899578a990b27d47 (127.19.226.193) to 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195). New cstate: current_term: 2 leader_uuid: "787ddcc824c2490a808fc5a53452dc5e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:14.497820 21627 leader_election.cc:304] T e8ff570914d74df8b87296010e9b864f P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, 88935ab835544e9889d05a6587d8cf21; no voters: 8bf6e26e98b14566899578a990b27d47
I20250109 08:58:14.498580 23147 raft_consensus.cc:2798] T e8ff570914d74df8b87296010e9b864f P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20250109 08:58:14.498997 23147 raft_consensus.cc:491] T e8ff570914d74df8b87296010e9b864f P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8bf6e26e98b14566899578a990b27d47)
I20250109 08:58:14.499382 23147 raft_consensus.cc:3054] T e8ff570914d74df8b87296010e9b864f P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:14.506572 23147 raft_consensus.cc:513] T e8ff570914d74df8b87296010e9b864f P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
W20250109 08:58:14.512667 21626 leader_election.cc:336] T e8ff570914d74df8b87296010e9b864f P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Network error: Client connection negotiation failed: client connection to 127.19.226.193:42725: connect: Connection refused (error 111)
I20250109 08:58:14.513540 23031 raft_consensus.cc:491] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8bf6e26e98b14566899578a990b27d47)
I20250109 08:58:14.514070 23031 raft_consensus.cc:513] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:14.513957 23147 leader_election.cc:290] T e8ff570914d74df8b87296010e9b864f P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 2 election: Requested vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), 88935ab835544e9889d05a6587d8cf21 (127.19.226.196:44363)
I20250109 08:58:14.516219 22177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8ff570914d74df8b87296010e9b864f" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "88935ab835544e9889d05a6587d8cf21"
I20250109 08:58:14.516824 22177 raft_consensus.cc:3054] T e8ff570914d74df8b87296010e9b864f P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Advancing to term 2
W20250109 08:58:14.521095 22142 leader_election.cc:336] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Network error: Client connection negotiation failed: client connection to 127.19.226.194:45527: connect: Connection refused (error 111)
I20250109 08:58:14.524418 22177 raft_consensus.cc:2463] T e8ff570914d74df8b87296010e9b864f P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 2.
I20250109 08:58:14.536688 21627 leader_election.cc:304] T e8ff570914d74df8b87296010e9b864f P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, 88935ab835544e9889d05a6587d8cf21; no voters: 8bf6e26e98b14566899578a990b27d47
I20250109 08:58:14.537747 23147 raft_consensus.cc:2798] T e8ff570914d74df8b87296010e9b864f P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:58:14.538195 23147 raft_consensus.cc:695] T e8ff570914d74df8b87296010e9b864f P 787ddcc824c2490a808fc5a53452dc5e [term 2 LEADER]: Becoming Leader. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Running, Role: LEADER
I20250109 08:58:14.516139 23031 leader_election.cc:290] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
I20250109 08:58:14.538959 23147 consensus_queue.cc:237] T e8ff570914d74df8b87296010e9b864f P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
W20250109 08:58:14.542699 22141 leader_election.cc:336] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Network error: Client connection negotiation failed: client connection to 127.19.226.193:42725: connect: Connection refused (error 111)
I20250109 08:58:14.543108 22141 leader_election.cc:304] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88935ab835544e9889d05a6587d8cf21; no voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:58:14.544232 23128 raft_consensus.cc:2743] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250109 08:58:14.552316 23031 raft_consensus.cc:491] T 71ac0243d72845659291184835652dde P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8bf6e26e98b14566899578a990b27d47)
I20250109 08:58:14.552807 23031 raft_consensus.cc:513] T 71ac0243d72845659291184835652dde P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:14.555352 23031 leader_election.cc:290] T 71ac0243d72845659291184835652dde P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
W20250109 08:58:14.561434 22141 leader_election.cc:336] T 71ac0243d72845659291184835652dde P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Network error: Client connection negotiation failed: client connection to 127.19.226.193:42725: connect: Connection refused (error 111)
W20250109 08:58:14.564096 22142 leader_election.cc:336] T 71ac0243d72845659291184835652dde P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Network error: Client connection negotiation failed: client connection to 127.19.226.194:45527: connect: Connection refused (error 111)
I20250109 08:58:14.564611 22142 leader_election.cc:304] T 71ac0243d72845659291184835652dde P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88935ab835544e9889d05a6587d8cf21; no voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:58:14.565531 23128 raft_consensus.cc:2743] T 71ac0243d72845659291184835652dde P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250109 08:58:14.567116 23147 raft_consensus.cc:491] T c99aacce09d44f889be5480854a22760 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8bf6e26e98b14566899578a990b27d47)
I20250109 08:58:14.567660 23147 raft_consensus.cc:513] T c99aacce09d44f889be5480854a22760 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:58:14.568626 21412 catalog_manager.cc:5526] T e8ff570914d74df8b87296010e9b864f P 787ddcc824c2490a808fc5a53452dc5e reported cstate change: term changed from 1 to 2, leader changed from 8bf6e26e98b14566899578a990b27d47 (127.19.226.193) to 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195). New cstate: current_term: 2 leader_uuid: "787ddcc824c2490a808fc5a53452dc5e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:14.570688 23147 leader_election.cc:290] T c99aacce09d44f889be5480854a22760 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
W20250109 08:58:14.575477 21627 leader_election.cc:336] T c99aacce09d44f889be5480854a22760 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Network error: Client connection negotiation failed: client connection to 127.19.226.194:45527: connect: Connection refused (error 111)
W20250109 08:58:14.575989 21626 leader_election.cc:336] T c99aacce09d44f889be5480854a22760 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Network error: Client connection negotiation failed: client connection to 127.19.226.193:42725: connect: Connection refused (error 111)
I20250109 08:58:14.576431 21626 leader_election.cc:304] T c99aacce09d44f889be5480854a22760 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e; no voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:58:14.576494 23147 raft_consensus.cc:491] T ddd14cdbefd64936960e9ffb5b38b5f2 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8bf6e26e98b14566899578a990b27d47)
I20250109 08:58:14.576974 23147 raft_consensus.cc:513] T ddd14cdbefd64936960e9ffb5b38b5f2 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:58:14.577457 23121 raft_consensus.cc:2743] T c99aacce09d44f889be5480854a22760 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20250109 08:58:14.578910 23147 leader_election.cc:290] T ddd14cdbefd64936960e9ffb5b38b5f2 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
W20250109 08:58:14.583186 21627 leader_election.cc:336] T ddd14cdbefd64936960e9ffb5b38b5f2 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Network error: Client connection negotiation failed: client connection to 127.19.226.194:45527: connect: Connection refused (error 111)
W20250109 08:58:14.583979 21626 leader_election.cc:336] T ddd14cdbefd64936960e9ffb5b38b5f2 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Network error: Client connection negotiation failed: client connection to 127.19.226.193:42725: connect: Connection refused (error 111)
I20250109 08:58:14.584398 21626 leader_election.cc:304] T ddd14cdbefd64936960e9ffb5b38b5f2 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e; no voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:58:14.585044 23147 raft_consensus.cc:2743] T ddd14cdbefd64936960e9ffb5b38b5f2 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250109 08:58:14.706753 23031 raft_consensus.cc:491] T e601a034e10e418bac8683443d86a1c1 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8bf6e26e98b14566899578a990b27d47)
I20250109 08:58:14.707219 23031 raft_consensus.cc:513] T e601a034e10e418bac8683443d86a1c1 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:14.708935 23031 leader_election.cc:290] T e601a034e10e418bac8683443d86a1c1 P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
W20250109 08:58:14.713282 22142 leader_election.cc:336] T e601a034e10e418bac8683443d86a1c1 P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Network error: Client connection negotiation failed: client connection to 127.19.226.194:45527: connect: Connection refused (error 111)
W20250109 08:58:14.713991 22141 leader_election.cc:336] T e601a034e10e418bac8683443d86a1c1 P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Network error: Client connection negotiation failed: client connection to 127.19.226.193:42725: connect: Connection refused (error 111)
I20250109 08:58:14.714416 22141 leader_election.cc:304] T e601a034e10e418bac8683443d86a1c1 P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88935ab835544e9889d05a6587d8cf21; no voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:58:14.715080 23031 raft_consensus.cc:2743] T e601a034e10e418bac8683443d86a1c1 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250109 08:58:14.776096 23147 raft_consensus.cc:491] T 610441ef4b1d40c284eff3a714e4214e P 787ddcc824c2490a808fc5a53452dc5e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8bf6e26e98b14566899578a990b27d47)
I20250109 08:58:14.776592 23147 raft_consensus.cc:513] T 610441ef4b1d40c284eff3a714e4214e P 787ddcc824c2490a808fc5a53452dc5e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:58:14.778429 23147 leader_election.cc:290] T 610441ef4b1d40c284eff3a714e4214e P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
W20250109 08:58:14.784917 21627 leader_election.cc:336] T 610441ef4b1d40c284eff3a714e4214e P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Network error: Client connection negotiation failed: client connection to 127.19.226.194:45527: connect: Connection refused (error 111)
W20250109 08:58:14.785648 21626 leader_election.cc:336] T 610441ef4b1d40c284eff3a714e4214e P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Network error: Client connection negotiation failed: client connection to 127.19.226.193:42725: connect: Connection refused (error 111)
I20250109 08:58:14.786049 21626 leader_election.cc:304] T 610441ef4b1d40c284eff3a714e4214e P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e; no voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:58:14.786698 23147 raft_consensus.cc:2743] T 610441ef4b1d40c284eff3a714e4214e P 787ddcc824c2490a808fc5a53452dc5e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20250109 08:58:14.797093 23031 raft_consensus.cc:491] T 180b4d539a144a38905bf340ecb24ef4 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8bf6e26e98b14566899578a990b27d47)
I20250109 08:58:14.797533 23031 raft_consensus.cc:513] T 180b4d539a144a38905bf340ecb24ef4 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:14.798985 23031 leader_election.cc:290] T 180b4d539a144a38905bf340ecb24ef4 P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
W20250109 08:58:14.802928 22141 leader_election.cc:336] T 180b4d539a144a38905bf340ecb24ef4 P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Network error: Client connection negotiation failed: client connection to 127.19.226.193:42725: connect: Connection refused (error 111)
W20250109 08:58:14.803359 22142 leader_election.cc:336] T 180b4d539a144a38905bf340ecb24ef4 P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Network error: Client connection negotiation failed: client connection to 127.19.226.194:45527: connect: Connection refused (error 111)
I20250109 08:58:14.803764 22142 leader_election.cc:304] T 180b4d539a144a38905bf340ecb24ef4 P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88935ab835544e9889d05a6587d8cf21; no voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:58:14.804841 23031 raft_consensus.cc:2743] T 180b4d539a144a38905bf340ecb24ef4 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250109 08:58:14.883832 23128 raft_consensus.cc:491] T 3b5049c00b0b4dd89c49fa5d13db169e P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2b1a692435241ee988e0bd1661cd4c3)
I20250109 08:58:14.884256 23128 raft_consensus.cc:513] T 3b5049c00b0b4dd89c49fa5d13db169e P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
W20250109 08:58:14.889772 22141 leader_election.cc:336] T 3b5049c00b0b4dd89c49fa5d13db169e P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Network error: Client connection negotiation failed: client connection to 127.19.226.193:42725: connect: Connection refused (error 111)
I20250109 08:58:14.886029 23128 leader_election.cc:290] T 3b5049c00b0b4dd89c49fa5d13db169e P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
W20250109 08:58:14.890538 22142 leader_election.cc:336] T 3b5049c00b0b4dd89c49fa5d13db169e P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Network error: Client connection negotiation failed: client connection to 127.19.226.194:45527: connect: Connection refused (error 111)
I20250109 08:58:14.890941 22142 leader_election.cc:304] T 3b5049c00b0b4dd89c49fa5d13db169e P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88935ab835544e9889d05a6587d8cf21; no voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:58:14.891598 23128 raft_consensus.cc:2743] T 3b5049c00b0b4dd89c49fa5d13db169e P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250109 08:58:14.920665 23127 raft_consensus.cc:491] T d355d166de534fb4a50c9f48f2df7469 P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a2b1a692435241ee988e0bd1661cd4c3)
I20250109 08:58:14.921136 23127 raft_consensus.cc:513] T d355d166de534fb4a50c9f48f2df7469 P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:14.923007 23127 leader_election.cc:290] T d355d166de534fb4a50c9f48f2df7469 P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:58:14.924234 21660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d355d166de534fb4a50c9f48f2df7469" candidate_uuid: "88935ab835544e9889d05a6587d8cf21" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e" is_pre_election: true
W20250109 08:58:14.927325 22142 leader_election.cc:336] T d355d166de534fb4a50c9f48f2df7469 P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Network error: Client connection negotiation failed: client connection to 127.19.226.194:45527: connect: Connection refused (error 111)
I20250109 08:58:14.927430 23127 raft_consensus.cc:491] T f5f85ad7c7c94acfaf9d550d55fce48c P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2b1a692435241ee988e0bd1661cd4c3)
I20250109 08:58:14.927743 22142 leader_election.cc:304] T d355d166de534fb4a50c9f48f2df7469 P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88935ab835544e9889d05a6587d8cf21; no voters: 787ddcc824c2490a808fc5a53452dc5e, a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:58:14.927933 23127 raft_consensus.cc:513] T f5f85ad7c7c94acfaf9d550d55fce48c P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:14.928723 23128 raft_consensus.cc:2743] T d355d166de534fb4a50c9f48f2df7469 P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20250109 08:58:14.931340 21660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5f85ad7c7c94acfaf9d550d55fce48c" candidate_uuid: "88935ab835544e9889d05a6587d8cf21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e" is_pre_election: true
W20250109 08:58:14.934623 22142 leader_election.cc:336] T f5f85ad7c7c94acfaf9d550d55fce48c P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Network error: Client connection negotiation failed: client connection to 127.19.226.194:45527: connect: Connection refused (error 111)
I20250109 08:58:14.934975 22142 leader_election.cc:304] T f5f85ad7c7c94acfaf9d550d55fce48c P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88935ab835544e9889d05a6587d8cf21; no voters: 787ddcc824c2490a808fc5a53452dc5e, a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:58:14.935652 23128 raft_consensus.cc:2743] T f5f85ad7c7c94acfaf9d550d55fce48c P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250109 08:58:14.935722 23127 leader_election.cc:290] T f5f85ad7c7c94acfaf9d550d55fce48c P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:58:14.947698 23147 raft_consensus.cc:491] T d355d166de534fb4a50c9f48f2df7469 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a2b1a692435241ee988e0bd1661cd4c3)
I20250109 08:58:14.948132 23147 raft_consensus.cc:513] T d355d166de534fb4a50c9f48f2df7469 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:14.949867 23147 leader_election.cc:290] T d355d166de534fb4a50c9f48f2df7469 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 88935ab835544e9889d05a6587d8cf21 (127.19.226.196:44363)
I20250109 08:58:14.953281 22177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d355d166de534fb4a50c9f48f2df7469" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "88935ab835544e9889d05a6587d8cf21" is_pre_election: true
I20250109 08:58:14.953905 22177 raft_consensus.cc:2463] T d355d166de534fb4a50c9f48f2df7469 P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 2.
I20250109 08:58:14.962049 21627 leader_election.cc:304] T d355d166de534fb4a50c9f48f2df7469 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, 88935ab835544e9889d05a6587d8cf21; no voters:
W20250109 08:58:14.962798 21627 leader_election.cc:336] T d355d166de534fb4a50c9f48f2df7469 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Network error: Client connection negotiation failed: client connection to 127.19.226.194:45527: connect: Connection refused (error 111)
I20250109 08:58:14.963968 23147 raft_consensus.cc:2798] T d355d166de534fb4a50c9f48f2df7469 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20250109 08:58:14.965039 22177 raft_consensus.cc:1270] T e8ff570914d74df8b87296010e9b864f P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250109 08:58:14.964394 23147 raft_consensus.cc:491] T d355d166de534fb4a50c9f48f2df7469 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Starting leader election (detected failure of leader a2b1a692435241ee988e0bd1661cd4c3)
I20250109 08:58:14.967072 23147 raft_consensus.cc:3054] T d355d166de534fb4a50c9f48f2df7469 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Advancing to term 3
I20250109 08:58:14.972966 23147 raft_consensus.cc:513] T d355d166de534fb4a50c9f48f2df7469 P 787ddcc824c2490a808fc5a53452dc5e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:14.974434 23147 leader_election.cc:290] T d355d166de534fb4a50c9f48f2df7469 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 3 election: Requested vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 88935ab835544e9889d05a6587d8cf21 (127.19.226.196:44363)
I20250109 08:58:14.976766 22177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d355d166de534fb4a50c9f48f2df7469" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "88935ab835544e9889d05a6587d8cf21"
I20250109 08:58:14.977231 22177 raft_consensus.cc:3054] T d355d166de534fb4a50c9f48f2df7469 P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Advancing to term 3
I20250109 08:58:14.982129 22177 raft_consensus.cc:2463] T d355d166de534fb4a50c9f48f2df7469 P 88935ab835544e9889d05a6587d8cf21 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 3.
I20250109 08:58:14.984460 23146 consensus_queue.cc:1035] T e8ff570914d74df8b87296010e9b864f P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250109 08:58:14.987596 21627 leader_election.cc:304] T d355d166de534fb4a50c9f48f2df7469 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, 88935ab835544e9889d05a6587d8cf21; no voters:
I20250109 08:58:14.989015 23146 raft_consensus.cc:2798] T d355d166de534fb4a50c9f48f2df7469 P 787ddcc824c2490a808fc5a53452dc5e [term 3 FOLLOWER]: Leader election won for term 3
I20250109 08:58:14.989459 23146 raft_consensus.cc:695] T d355d166de534fb4a50c9f48f2df7469 P 787ddcc824c2490a808fc5a53452dc5e [term 3 LEADER]: Becoming Leader. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Running, Role: LEADER
I20250109 08:58:14.990226 23146 consensus_queue.cc:237] T d355d166de534fb4a50c9f48f2df7469 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
W20250109 08:58:14.995319 21627 leader_election.cc:336] T d355d166de534fb4a50c9f48f2df7469 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Network error: Client connection negotiation failed: client connection to 127.19.226.194:45527: connect: Connection refused (error 111)
W20250109 08:58:15.002017 21626 consensus_peers.cc:487] T e8ff570914d74df8b87296010e9b864f P 787ddcc824c2490a808fc5a53452dc5e -> Peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Couldn't send request to peer 8bf6e26e98b14566899578a990b27d47. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.193:42725: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:15.013881 21412 catalog_manager.cc:5526] T d355d166de534fb4a50c9f48f2df7469 P 787ddcc824c2490a808fc5a53452dc5e reported cstate change: term changed from 2 to 3, leader changed from a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194) to 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195). New cstate: current_term: 3 leader_uuid: "787ddcc824c2490a808fc5a53452dc5e" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:15.031042 23128 raft_consensus.cc:491] T 7781c00473024307987096b596da12ac P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2b1a692435241ee988e0bd1661cd4c3)
I20250109 08:58:15.031597 23128 raft_consensus.cc:513] T 7781c00473024307987096b596da12ac P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:15.033540 23128 leader_election.cc:290] T 7781c00473024307987096b596da12ac P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:58:15.034847 21660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7781c00473024307987096b596da12ac" candidate_uuid: "88935ab835544e9889d05a6587d8cf21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e" is_pre_election: true
W20250109 08:58:15.038538 22142 leader_election.cc:336] T 7781c00473024307987096b596da12ac P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Network error: Client connection negotiation failed: client connection to 127.19.226.194:45527: connect: Connection refused (error 111)
I20250109 08:58:15.039043 22142 leader_election.cc:304] T 7781c00473024307987096b596da12ac P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88935ab835544e9889d05a6587d8cf21; no voters: 787ddcc824c2490a808fc5a53452dc5e, a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:58:15.040858 23128 raft_consensus.cc:2743] T 7781c00473024307987096b596da12ac P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250109 08:58:15.044034 22173 raft_consensus.cc:1270] T edc9db96e0ff4aa5839a6922f3eb5f24 P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250109 08:58:15.046458 23146 consensus_queue.cc:1035] T edc9db96e0ff4aa5839a6922f3eb5f24 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20250109 08:58:15.057814 21626 consensus_peers.cc:487] T edc9db96e0ff4aa5839a6922f3eb5f24 P 787ddcc824c2490a808fc5a53452dc5e -> Peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Couldn't send request to peer 8bf6e26e98b14566899578a990b27d47. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.193:42725: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:15.058547 23128 raft_consensus.cc:491] T 56bbc306275e4a409e028a7ca1dd1602 P 88935ab835544e9889d05a6587d8cf21 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a2b1a692435241ee988e0bd1661cd4c3)
I20250109 08:58:15.062029 23128 raft_consensus.cc:513] T 56bbc306275e4a409e028a7ca1dd1602 P 88935ab835544e9889d05a6587d8cf21 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:15.064245 23128 leader_election.cc:290] T 56bbc306275e4a409e028a7ca1dd1602 P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
W20250109 08:58:15.069157 22142 leader_election.cc:336] T 56bbc306275e4a409e028a7ca1dd1602 P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Network error: Client connection negotiation failed: client connection to 127.19.226.194:45527: connect: Connection refused (error 111)
W20250109 08:58:15.070974 22141 leader_election.cc:336] T 56bbc306275e4a409e028a7ca1dd1602 P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Network error: Client connection negotiation failed: client connection to 127.19.226.193:42725: connect: Connection refused (error 111)
I20250109 08:58:15.075520 22141 leader_election.cc:304] T 56bbc306275e4a409e028a7ca1dd1602 P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88935ab835544e9889d05a6587d8cf21; no voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:58:15.078115 23128 raft_consensus.cc:2743] T 56bbc306275e4a409e028a7ca1dd1602 P 88935ab835544e9889d05a6587d8cf21 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20250109 08:58:15.110981 20363 thread.cc:535] Waited for 1000ms trying to join with diag-logger (tid 21610)
W20250109 08:58:15.118541 22141 consensus_peers.cc:487] T 206785f29c824cddbe9bbc00d365ffb3 P 88935ab835544e9889d05a6587d8cf21 -> Peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Couldn't send request to peer 8bf6e26e98b14566899578a990b27d47. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.193:42725: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20250109 08:58:15.119673 23121 raft_consensus.cc:491] T eb4516b9a16041c2adf34be520768f3f P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a2b1a692435241ee988e0bd1661cd4c3)
I20250109 08:58:15.120195 23121 raft_consensus.cc:513] T eb4516b9a16041c2adf34be520768f3f P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:15.121109 23128 raft_consensus.cc:491] T 0cc4fcda876242c396b804edebebb01f P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a2b1a692435241ee988e0bd1661cd4c3)
I20250109 08:58:15.121673 23128 raft_consensus.cc:513] T 0cc4fcda876242c396b804edebebb01f P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:15.122668 23121 leader_election.cc:290] T eb4516b9a16041c2adf34be520768f3f P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 88935ab835544e9889d05a6587d8cf21 (127.19.226.196:44363)
I20250109 08:58:15.123606 23128 leader_election.cc:290] T 0cc4fcda876242c396b804edebebb01f P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
I20250109 08:58:15.126395 22173 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb4516b9a16041c2adf34be520768f3f" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "88935ab835544e9889d05a6587d8cf21" is_pre_election: true
W20250109 08:58:15.126664 21627 leader_election.cc:336] T eb4516b9a16041c2adf34be520768f3f P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Network error: Client connection negotiation failed: client connection to 127.19.226.194:45527: connect: Connection refused (error 111)
I20250109 08:58:15.127803 21627 leader_election.cc:304] T eb4516b9a16041c2adf34be520768f3f P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e; no voters: 88935ab835544e9889d05a6587d8cf21, a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:58:15.128593 23146 raft_consensus.cc:2743] T eb4516b9a16041c2adf34be520768f3f P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20250109 08:58:15.130977 22141 leader_election.cc:336] T 0cc4fcda876242c396b804edebebb01f P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Network error: Client connection negotiation failed: client connection to 127.19.226.193:42725: connect: Connection refused (error 111)
W20250109 08:58:15.134968 22142 leader_election.cc:336] T 0cc4fcda876242c396b804edebebb01f P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Network error: Client connection negotiation failed: client connection to 127.19.226.194:45527: connect: Connection refused (error 111)
I20250109 08:58:15.135298 22142 leader_election.cc:304] T 0cc4fcda876242c396b804edebebb01f P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88935ab835544e9889d05a6587d8cf21; no voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:58:15.135926 23128 raft_consensus.cc:2743] T 0cc4fcda876242c396b804edebebb01f P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20250109 08:58:15.137872 23121 raft_consensus.cc:491] T 1d3d5d2a2c88417a97c9d57262a356dd P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a2b1a692435241ee988e0bd1661cd4c3)
I20250109 08:58:15.138379 23121 raft_consensus.cc:513] T 1d3d5d2a2c88417a97c9d57262a356dd P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:15.140064 23121 leader_election.cc:290] T 1d3d5d2a2c88417a97c9d57262a356dd P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 88935ab835544e9889d05a6587d8cf21 (127.19.226.196:44363)
I20250109 08:58:15.141158 22173 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d3d5d2a2c88417a97c9d57262a356dd" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "88935ab835544e9889d05a6587d8cf21" is_pre_election: true
W20250109 08:58:15.144152 21627 leader_election.cc:336] T 1d3d5d2a2c88417a97c9d57262a356dd P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Network error: Client connection negotiation failed: client connection to 127.19.226.194:45527: connect: Connection refused (error 111)
I20250109 08:58:15.144598 21627 leader_election.cc:304] T 1d3d5d2a2c88417a97c9d57262a356dd P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e; no voters: 88935ab835544e9889d05a6587d8cf21, a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:58:15.146054 23121 raft_consensus.cc:2743] T 1d3d5d2a2c88417a97c9d57262a356dd P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20250109 08:58:15.147409 23128 raft_consensus.cc:491] T 1d3d5d2a2c88417a97c9d57262a356dd P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a2b1a692435241ee988e0bd1661cd4c3)
I20250109 08:58:15.147828 23128 raft_consensus.cc:513] T 1d3d5d2a2c88417a97c9d57262a356dd P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:15.149441 23128 leader_election.cc:290] T 1d3d5d2a2c88417a97c9d57262a356dd P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:58:15.152330 21660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d3d5d2a2c88417a97c9d57262a356dd" candidate_uuid: "88935ab835544e9889d05a6587d8cf21" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e" is_pre_election: true
I20250109 08:58:15.153055 21660 raft_consensus.cc:2463] T 1d3d5d2a2c88417a97c9d57262a356dd P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88935ab835544e9889d05a6587d8cf21 in term 2.
I20250109 08:58:15.154413 22141 leader_election.cc:304] T 1d3d5d2a2c88417a97c9d57262a356dd P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, 88935ab835544e9889d05a6587d8cf21; no voters:
I20250109 08:58:15.154959 23128 raft_consensus.cc:2798] T 1d3d5d2a2c88417a97c9d57262a356dd P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20250109 08:58:15.155030 22142 leader_election.cc:336] T 1d3d5d2a2c88417a97c9d57262a356dd P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Network error: Client connection negotiation failed: client connection to 127.19.226.194:45527: connect: Connection refused (error 111)
I20250109 08:58:15.155409 23128 raft_consensus.cc:491] T 1d3d5d2a2c88417a97c9d57262a356dd P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Starting leader election (detected failure of leader a2b1a692435241ee988e0bd1661cd4c3)
I20250109 08:58:15.155802 23128 raft_consensus.cc:3054] T 1d3d5d2a2c88417a97c9d57262a356dd P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Advancing to term 3
I20250109 08:58:15.160342 23128 raft_consensus.cc:513] T 1d3d5d2a2c88417a97c9d57262a356dd P 88935ab835544e9889d05a6587d8cf21 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:15.162081 23128 leader_election.cc:290] T 1d3d5d2a2c88417a97c9d57262a356dd P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 3 election: Requested vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:58:15.163041 21660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d3d5d2a2c88417a97c9d57262a356dd" candidate_uuid: "88935ab835544e9889d05a6587d8cf21" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e"
I20250109 08:58:15.165354 21660 raft_consensus.cc:3054] T 1d3d5d2a2c88417a97c9d57262a356dd P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Advancing to term 3
W20250109 08:58:15.166740 22142 leader_election.cc:336] T 1d3d5d2a2c88417a97c9d57262a356dd P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Network error: Client connection negotiation failed: client connection to 127.19.226.194:45527: connect: Connection refused (error 111)
I20250109 08:58:15.173161 21660 raft_consensus.cc:2463] T 1d3d5d2a2c88417a97c9d57262a356dd P 787ddcc824c2490a808fc5a53452dc5e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88935ab835544e9889d05a6587d8cf21 in term 3.
I20250109 08:58:15.174284 22141 leader_election.cc:304] T 1d3d5d2a2c88417a97c9d57262a356dd P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, 88935ab835544e9889d05a6587d8cf21; no voters: a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:58:15.175714 23127 raft_consensus.cc:491] T eb4516b9a16041c2adf34be520768f3f P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a2b1a692435241ee988e0bd1661cd4c3)
I20250109 08:58:15.175268 23128 raft_consensus.cc:2798] T 1d3d5d2a2c88417a97c9d57262a356dd P 88935ab835544e9889d05a6587d8cf21 [term 3 FOLLOWER]: Leader election won for term 3
I20250109 08:58:15.176188 23127 raft_consensus.cc:513] T eb4516b9a16041c2adf34be520768f3f P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:15.177007 23128 raft_consensus.cc:695] T 1d3d5d2a2c88417a97c9d57262a356dd P 88935ab835544e9889d05a6587d8cf21 [term 3 LEADER]: Becoming Leader. State: Replica: 88935ab835544e9889d05a6587d8cf21, State: Running, Role: LEADER
I20250109 08:58:15.178014 23127 leader_election.cc:290] T eb4516b9a16041c2adf34be520768f3f P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:58:15.177883 23128 consensus_queue.cc:237] T 1d3d5d2a2c88417a97c9d57262a356dd P 88935ab835544e9889d05a6587d8cf21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:15.181349 21663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb4516b9a16041c2adf34be520768f3f" candidate_uuid: "88935ab835544e9889d05a6587d8cf21" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e" is_pre_election: true
I20250109 08:58:15.181967 21663 raft_consensus.cc:2463] T eb4516b9a16041c2adf34be520768f3f P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88935ab835544e9889d05a6587d8cf21 in term 2.
W20250109 08:58:15.182307 22142 leader_election.cc:336] T eb4516b9a16041c2adf34be520768f3f P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Network error: Client connection negotiation failed: client connection to 127.19.226.194:45527: connect: Connection refused (error 111)
I20250109 08:58:15.182868 22141 leader_election.cc:304] T eb4516b9a16041c2adf34be520768f3f P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, 88935ab835544e9889d05a6587d8cf21; no voters: a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:58:15.183557 23127 raft_consensus.cc:2798] T eb4516b9a16041c2adf34be520768f3f P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20250109 08:58:15.183890 23127 raft_consensus.cc:491] T eb4516b9a16041c2adf34be520768f3f P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Starting leader election (detected failure of leader a2b1a692435241ee988e0bd1661cd4c3)
I20250109 08:58:15.184211 23127 raft_consensus.cc:3054] T eb4516b9a16041c2adf34be520768f3f P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Advancing to term 3
I20250109 08:58:15.191035 23127 raft_consensus.cc:513] T eb4516b9a16041c2adf34be520768f3f P 88935ab835544e9889d05a6587d8cf21 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:15.192955 23031 raft_consensus.cc:491] T 126a163a89c448be89ccb9490e9f0168 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2b1a692435241ee988e0bd1661cd4c3)
I20250109 08:58:15.193312 23031 raft_consensus.cc:513] T 126a163a89c448be89ccb9490e9f0168 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:15.195034 23031 leader_election.cc:290] T 126a163a89c448be89ccb9490e9f0168 P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
I20250109 08:58:15.195801 23121 raft_consensus.cc:491] T f5f85ad7c7c94acfaf9d550d55fce48c P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2b1a692435241ee988e0bd1661cd4c3)
I20250109 08:58:15.196076 23127 leader_election.cc:290] T eb4516b9a16041c2adf34be520768f3f P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 3 election: Requested vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:58:15.196295 23121 raft_consensus.cc:513] T f5f85ad7c7c94acfaf9d550d55fce48c P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:15.199014 23121 leader_election.cc:290] T f5f85ad7c7c94acfaf9d550d55fce48c P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 88935ab835544e9889d05a6587d8cf21 (127.19.226.196:44363)
W20250109 08:58:15.199941 22142 leader_election.cc:336] T eb4516b9a16041c2adf34be520768f3f P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Network error: Client connection negotiation failed: client connection to 127.19.226.194:45527: connect: Connection refused (error 111)
W20250109 08:58:15.200477 22142 leader_election.cc:336] T 126a163a89c448be89ccb9490e9f0168 P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Network error: Client connection negotiation failed: client connection to 127.19.226.194:45527: connect: Connection refused (error 111)
I20250109 08:58:15.201576 22173 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5f85ad7c7c94acfaf9d550d55fce48c" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "88935ab835544e9889d05a6587d8cf21" is_pre_election: true
I20250109 08:58:15.201607 21663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb4516b9a16041c2adf34be520768f3f" candidate_uuid: "88935ab835544e9889d05a6587d8cf21" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e"
I20250109 08:58:15.202227 22173 raft_consensus.cc:2463] T f5f85ad7c7c94acfaf9d550d55fce48c P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 1.
I20250109 08:58:15.202240 21663 raft_consensus.cc:3054] T eb4516b9a16041c2adf34be520768f3f P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Advancing to term 3
I20250109 08:58:15.203351 21627 leader_election.cc:304] T f5f85ad7c7c94acfaf9d550d55fce48c P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, 88935ab835544e9889d05a6587d8cf21; no voters:
W20250109 08:58:15.204078 21627 leader_election.cc:336] T f5f85ad7c7c94acfaf9d550d55fce48c P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Network error: Client connection negotiation failed: client connection to 127.19.226.194:45527: connect: Connection refused (error 111)
I20250109 08:58:15.204945 23121 raft_consensus.cc:2798] T f5f85ad7c7c94acfaf9d550d55fce48c P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20250109 08:58:15.205327 23121 raft_consensus.cc:491] T f5f85ad7c7c94acfaf9d550d55fce48c P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Starting leader election (detected failure of leader a2b1a692435241ee988e0bd1661cd4c3)
I20250109 08:58:15.205683 23121 raft_consensus.cc:3054] T f5f85ad7c7c94acfaf9d550d55fce48c P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:15.211665 21412 catalog_manager.cc:5526] T 1d3d5d2a2c88417a97c9d57262a356dd P 88935ab835544e9889d05a6587d8cf21 reported cstate change: term changed from 2 to 3, leader changed from a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194) to 88935ab835544e9889d05a6587d8cf21 (127.19.226.196). New cstate: current_term: 3 leader_uuid: "88935ab835544e9889d05a6587d8cf21" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250109 08:58:15.212419 23121 raft_consensus.cc:513] T f5f85ad7c7c94acfaf9d550d55fce48c P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
W20250109 08:58:15.215812 22141 leader_election.cc:336] T 126a163a89c448be89ccb9490e9f0168 P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Network error: Client connection negotiation failed: client connection to 127.19.226.193:42725: connect: Connection refused (error 111)
I20250109 08:58:15.216243 22141 leader_election.cc:304] T 126a163a89c448be89ccb9490e9f0168 P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88935ab835544e9889d05a6587d8cf21; no voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3
W20250109 08:58:15.217846 21627 leader_election.cc:336] T f5f85ad7c7c94acfaf9d550d55fce48c P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Network error: Client connection negotiation failed: client connection to 127.19.226.194:45527: connect: Connection refused (error 111)
I20250109 08:58:15.219442 23127 raft_consensus.cc:2743] T 126a163a89c448be89ccb9490e9f0168 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250109 08:58:15.220052 23127 raft_consensus.cc:491] T d6f988a231a74f6fa112da9a527d4a0b P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a2b1a692435241ee988e0bd1661cd4c3)
I20250109 08:58:15.220220 23031 raft_consensus.cc:491] T 8c0459d2eee64f0a83bd05ad5974f506 P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a2b1a692435241ee988e0bd1661cd4c3)
I20250109 08:58:15.220716 23031 raft_consensus.cc:513] T 8c0459d2eee64f0a83bd05ad5974f506 P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:15.222484 23031 leader_election.cc:290] T 8c0459d2eee64f0a83bd05ad5974f506 P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:58:15.225199 23146 raft_consensus.cc:491] T 8c0459d2eee64f0a83bd05ad5974f506 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a2b1a692435241ee988e0bd1661cd4c3)
I20250109 08:58:15.226012 21660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c0459d2eee64f0a83bd05ad5974f506" candidate_uuid: "88935ab835544e9889d05a6587d8cf21" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e" is_pre_election: true
I20250109 08:58:15.225731 23146 raft_consensus.cc:513] T 8c0459d2eee64f0a83bd05ad5974f506 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:15.228200 23146 leader_election.cc:290] T 8c0459d2eee64f0a83bd05ad5974f506 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 88935ab835544e9889d05a6587d8cf21 (127.19.226.196:44363)
W20250109 08:58:15.230535 22142 leader_election.cc:336] T 8c0459d2eee64f0a83bd05ad5974f506 P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Network error: Client connection negotiation failed: client connection to 127.19.226.194:45527: connect: Connection refused (error 111)
I20250109 08:58:15.231456 21660 raft_consensus.cc:2463] T 8c0459d2eee64f0a83bd05ad5974f506 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88935ab835544e9889d05a6587d8cf21 in term 2.
I20250109 08:58:15.231737 22173 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c0459d2eee64f0a83bd05ad5974f506" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "88935ab835544e9889d05a6587d8cf21" is_pre_election: true
I20250109 08:58:15.232287 23127 raft_consensus.cc:513] T d6f988a231a74f6fa112da9a527d4a0b P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:15.232908 22173 raft_consensus.cc:2463] T 8c0459d2eee64f0a83bd05ad5974f506 P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 2.
W20250109 08:58:15.233592 21626 leader_election.cc:336] T 8c0459d2eee64f0a83bd05ad5974f506 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Network error: Client connection negotiation failed: client connection to 127.19.226.193:42725: connect: Connection refused (error 111)
W20250109 08:58:15.234287 21627 leader_election.cc:336] T 8c0459d2eee64f0a83bd05ad5974f506 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Network error: Client connection negotiation failed: client connection to 127.19.226.194:45527: connect: Connection refused (error 111)
I20250109 08:58:15.234705 21627 leader_election.cc:304] T 8c0459d2eee64f0a83bd05ad5974f506 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e; no voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:58:15.234550 23127 leader_election.cc:290] T d6f988a231a74f6fa112da9a527d4a0b P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
I20250109 08:58:15.231884 23121 leader_election.cc:290] T f5f85ad7c7c94acfaf9d550d55fce48c P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 2 election: Requested vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 88935ab835544e9889d05a6587d8cf21 (127.19.226.196:44363)
I20250109 08:58:15.231998 22177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5f85ad7c7c94acfaf9d550d55fce48c" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "88935ab835544e9889d05a6587d8cf21"
I20250109 08:58:15.236088 22177 raft_consensus.cc:3054] T f5f85ad7c7c94acfaf9d550d55fce48c P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Advancing to term 2
W20250109 08:58:15.236616 22141 leader_election.cc:336] T 8c0459d2eee64f0a83bd05ad5974f506 P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Network error: Client connection negotiation failed: client connection to 127.19.226.193:42725: connect: Connection refused (error 111)
I20250109 08:58:15.237018 22141 leader_election.cc:304] T 8c0459d2eee64f0a83bd05ad5974f506 P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 88935ab835544e9889d05a6587d8cf21; no voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:58:15.237988 23127 raft_consensus.cc:2743] T 8c0459d2eee64f0a83bd05ad5974f506 P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20250109 08:58:15.236093 23121 raft_consensus.cc:2743] T 8c0459d2eee64f0a83bd05ad5974f506 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20250109 08:58:15.242293 22177 raft_consensus.cc:2463] T f5f85ad7c7c94acfaf9d550d55fce48c P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 2.
I20250109 08:58:15.244246 23121 raft_consensus.cc:491] T 7781c00473024307987096b596da12ac P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2b1a692435241ee988e0bd1661cd4c3)
I20250109 08:58:15.244724 23121 raft_consensus.cc:513] T 7781c00473024307987096b596da12ac P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:15.246176 21627 leader_election.cc:304] T f5f85ad7c7c94acfaf9d550d55fce48c P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, 88935ab835544e9889d05a6587d8cf21; no voters: a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:58:15.246838 23146 raft_consensus.cc:2798] T f5f85ad7c7c94acfaf9d550d55fce48c P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:58:15.250535 23146 raft_consensus.cc:695] T f5f85ad7c7c94acfaf9d550d55fce48c P 787ddcc824c2490a808fc5a53452dc5e [term 2 LEADER]: Becoming Leader. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Running, Role: LEADER
I20250109 08:58:15.251396 23146 consensus_queue.cc:237] T f5f85ad7c7c94acfaf9d550d55fce48c P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
W20250109 08:58:15.251880 22142 leader_election.cc:336] T d6f988a231a74f6fa112da9a527d4a0b P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Network error: Client connection negotiation failed: client connection to 127.19.226.194:45527: connect: Connection refused (error 111)
I20250109 08:58:15.242293 21663 raft_consensus.cc:2463] T eb4516b9a16041c2adf34be520768f3f P 787ddcc824c2490a808fc5a53452dc5e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88935ab835544e9889d05a6587d8cf21 in term 3.
I20250109 08:58:15.248843 23121 leader_election.cc:290] T 7781c00473024307987096b596da12ac P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 88935ab835544e9889d05a6587d8cf21 (127.19.226.196:44363)
I20250109 08:58:15.257054 22141 leader_election.cc:304] T eb4516b9a16041c2adf34be520768f3f P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, 88935ab835544e9889d05a6587d8cf21; no voters: a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:58:15.257787 23128 raft_consensus.cc:2798] T eb4516b9a16041c2adf34be520768f3f P 88935ab835544e9889d05a6587d8cf21 [term 3 FOLLOWER]: Leader election won for term 3
W20250109 08:58:15.258030 22141 leader_election.cc:336] T d6f988a231a74f6fa112da9a527d4a0b P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Network error: Client connection negotiation failed: client connection to 127.19.226.193:42725: connect: Connection refused (error 111)
I20250109 08:58:15.258426 23128 raft_consensus.cc:695] T eb4516b9a16041c2adf34be520768f3f P 88935ab835544e9889d05a6587d8cf21 [term 3 LEADER]: Becoming Leader. State: Replica: 88935ab835544e9889d05a6587d8cf21, State: Running, Role: LEADER
I20250109 08:58:15.259276 23128 consensus_queue.cc:237] T eb4516b9a16041c2adf34be520768f3f P 88935ab835544e9889d05a6587d8cf21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:15.258447 22141 leader_election.cc:304] T d6f988a231a74f6fa112da9a527d4a0b P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88935ab835544e9889d05a6587d8cf21; no voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:58:15.260843 23031 raft_consensus.cc:2743] T d6f988a231a74f6fa112da9a527d4a0b P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20250109 08:58:15.262293 22177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7781c00473024307987096b596da12ac" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "88935ab835544e9889d05a6587d8cf21" is_pre_election: true
I20250109 08:58:15.262912 22177 raft_consensus.cc:2463] T 7781c00473024307987096b596da12ac P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 1.
W20250109 08:58:15.268481 21627 leader_election.cc:336] T 7781c00473024307987096b596da12ac P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Network error: Client connection negotiation failed: client connection to 127.19.226.194:45527: connect: Connection refused (error 111)
I20250109 08:58:15.269407 21627 leader_election.cc:304] T 7781c00473024307987096b596da12ac P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, 88935ab835544e9889d05a6587d8cf21; no voters: a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:58:15.276068 21412 catalog_manager.cc:5526] T f5f85ad7c7c94acfaf9d550d55fce48c P 787ddcc824c2490a808fc5a53452dc5e reported cstate change: term changed from 1 to 2, leader changed from a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194) to 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195). New cstate: current_term: 2 leader_uuid: "787ddcc824c2490a808fc5a53452dc5e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:15.277521 23127 raft_consensus.cc:491] T 0e06021cd6624e50af2c914399ebd6c4 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2b1a692435241ee988e0bd1661cd4c3)
I20250109 08:58:15.278080 23127 raft_consensus.cc:513] T 0e06021cd6624e50af2c914399ebd6c4 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:15.280005 23127 leader_election.cc:290] T 0e06021cd6624e50af2c914399ebd6c4 P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731)
I20250109 08:58:15.281277 23146 raft_consensus.cc:2798] T 7781c00473024307987096b596da12ac P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20250109 08:58:15.281764 23146 raft_consensus.cc:491] T 7781c00473024307987096b596da12ac P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Starting leader election (detected failure of leader a2b1a692435241ee988e0bd1661cd4c3)
I20250109 08:58:15.282147 23146 raft_consensus.cc:3054] T 7781c00473024307987096b596da12ac P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:15.284526 23121 raft_consensus.cc:491] T f961d3f357944118a69f905621731552 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2b1a692435241ee988e0bd1661cd4c3)
I20250109 08:58:15.285072 23121 raft_consensus.cc:513] T f961d3f357944118a69f905621731552 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:58:15.287117 23121 leader_election.cc:290] T f961d3f357944118a69f905621731552 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
W20250109 08:58:15.291604 21626 leader_election.cc:336] T f961d3f357944118a69f905621731552 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Network error: Client connection negotiation failed: client connection to 127.19.226.193:42725: connect: Connection refused (error 111)
I20250109 08:58:15.292330 23146 raft_consensus.cc:513] T 7781c00473024307987096b596da12ac P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:15.304386 23146 leader_election.cc:290] T 7781c00473024307987096b596da12ac P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 2 election: Requested vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 88935ab835544e9889d05a6587d8cf21 (127.19.226.196:44363)
W20250109 08:58:15.307183 22141 consensus_peers.cc:487] T 20ea30dba4314290ae45d9b4c13bebdd P 88935ab835544e9889d05a6587d8cf21 -> Peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Couldn't send request to peer 8bf6e26e98b14566899578a990b27d47. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.193:42725: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20250109 08:58:15.307859 21663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e06021cd6624e50af2c914399ebd6c4" candidate_uuid: "88935ab835544e9889d05a6587d8cf21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "787ddcc824c2490a808fc5a53452dc5e" is_pre_election: true
I20250109 08:58:15.315505 21412 catalog_manager.cc:5526] T eb4516b9a16041c2adf34be520768f3f P 88935ab835544e9889d05a6587d8cf21 reported cstate change: term changed from 2 to 3, leader changed from a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194) to 88935ab835544e9889d05a6587d8cf21 (127.19.226.196). New cstate: current_term: 3 leader_uuid: "88935ab835544e9889d05a6587d8cf21" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20250109 08:58:15.317083 22141 consensus_peers.cc:487] T 1b582ab8d8f044d9b9038a28e958feeb P 88935ab835544e9889d05a6587d8cf21 -> Peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Couldn't send request to peer 8bf6e26e98b14566899578a990b27d47. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.193:42725: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20250109 08:58:15.318382 22142 leader_election.cc:336] T 0e06021cd6624e50af2c914399ebd6c4 P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Network error: Client connection negotiation failed: client connection to 127.19.226.194:45527: connect: Connection refused (error 111)
I20250109 08:58:15.323262 22142 leader_election.cc:304] T 0e06021cd6624e50af2c914399ebd6c4 P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88935ab835544e9889d05a6587d8cf21; no voters: 787ddcc824c2490a808fc5a53452dc5e, a2b1a692435241ee988e0bd1661cd4c3
W20250109 08:58:15.323527 21626 consensus_peers.cc:487] T 7a7211b8d4374ad1afd82e8cf85bad81 P 787ddcc824c2490a808fc5a53452dc5e -> Peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Couldn't send request to peer 8bf6e26e98b14566899578a990b27d47. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.193:42725: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20250109 08:58:15.326875 23121 raft_consensus.cc:491] T 0e06021cd6624e50af2c914399ebd6c4 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2b1a692435241ee988e0bd1661cd4c3)
I20250109 08:58:15.327342 23121 raft_consensus.cc:513] T 0e06021cd6624e50af2c914399ebd6c4 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:15.328176 23031 raft_consensus.cc:2743] T 0e06021cd6624e50af2c914399ebd6c4 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250109 08:58:15.329217 23121 leader_election.cc:290] T 0e06021cd6624e50af2c914399ebd6c4 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 88935ab835544e9889d05a6587d8cf21 (127.19.226.196:44363)
W20250109 08:58:15.329782 21627 leader_election.cc:336] T f961d3f357944118a69f905621731552 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Network error: Client connection negotiation failed: client connection to 127.19.226.194:45527: connect: Connection refused (error 111)
I20250109 08:58:15.333267 22177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7781c00473024307987096b596da12ac" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "88935ab835544e9889d05a6587d8cf21"
I20250109 08:58:15.334257 22177 raft_consensus.cc:3054] T 7781c00473024307987096b596da12ac P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:15.340782 22177 raft_consensus.cc:2463] T 7781c00473024307987096b596da12ac P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 2.
I20250109 08:58:15.330153 21627 leader_election.cc:304] T f961d3f357944118a69f905621731552 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e; no voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:58:15.343622 23146 raft_consensus.cc:2743] T f961d3f357944118a69f905621731552 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20250109 08:58:15.344527 21627 leader_election.cc:336] T 7781c00473024307987096b596da12ac P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Network error: Client connection negotiation failed: client connection to 127.19.226.194:45527: connect: Connection refused (error 111)
W20250109 08:58:15.347039 22141 consensus_peers.cc:487] T 5717a02ef53a4b22a96f431ca6c20d93 P 88935ab835544e9889d05a6587d8cf21 -> Peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Couldn't send request to peer 8bf6e26e98b14566899578a990b27d47. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.193:42725: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20250109 08:58:15.349511 22177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e06021cd6624e50af2c914399ebd6c4" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "88935ab835544e9889d05a6587d8cf21" is_pre_election: true
I20250109 08:58:15.350031 22177 raft_consensus.cc:2463] T 0e06021cd6624e50af2c914399ebd6c4 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 1.
I20250109 08:58:15.351809 21627 leader_election.cc:304] T 7781c00473024307987096b596da12ac P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, 88935ab835544e9889d05a6587d8cf21; no voters: a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:58:15.352509 23121 raft_consensus.cc:491] T c18c1d46a6ca47598d7b161ed70a4e7a P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a2b1a692435241ee988e0bd1661cd4c3)
I20250109 08:58:15.352953 23121 raft_consensus.cc:513] T c18c1d46a6ca47598d7b161ed70a4e7a P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:58:15.353711 23146 raft_consensus.cc:2798] T 7781c00473024307987096b596da12ac P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:58:15.355195 21627 leader_election.cc:304] T 0e06021cd6624e50af2c914399ebd6c4 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, 88935ab835544e9889d05a6587d8cf21; no voters:
I20250109 08:58:15.356127 23147 raft_consensus.cc:2798] T 0e06021cd6624e50af2c914399ebd6c4 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20250109 08:58:15.356511 23147 raft_consensus.cc:491] T 0e06021cd6624e50af2c914399ebd6c4 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Starting leader election (detected failure of leader a2b1a692435241ee988e0bd1661cd4c3)
I20250109 08:58:15.356310 23121 leader_election.cc:290] T c18c1d46a6ca47598d7b161ed70a4e7a P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
W20250109 08:58:15.358996 21627 leader_election.cc:336] T 0e06021cd6624e50af2c914399ebd6c4 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Network error: Client connection negotiation failed: client connection to 127.19.226.194:45527: connect: Connection refused (error 111)
I20250109 08:58:15.360258 23146 raft_consensus.cc:695] T 7781c00473024307987096b596da12ac P 787ddcc824c2490a808fc5a53452dc5e [term 2 LEADER]: Becoming Leader. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Running, Role: LEADER
I20250109 08:58:15.356865 23147 raft_consensus.cc:3054] T 0e06021cd6624e50af2c914399ebd6c4 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:15.361302 23146 consensus_queue.cc:237] T 7781c00473024307987096b596da12ac P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
W20250109 08:58:15.364427 21626 leader_election.cc:336] T c18c1d46a6ca47598d7b161ed70a4e7a P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Network error: Client connection negotiation failed: client connection to 127.19.226.193:42725: connect: Connection refused (error 111)
W20250109 08:58:15.368062 21627 leader_election.cc:336] T c18c1d46a6ca47598d7b161ed70a4e7a P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Network error: Client connection negotiation failed: client connection to 127.19.226.194:45527: connect: Connection refused (error 111)
I20250109 08:58:15.367833 23147 raft_consensus.cc:513] T 0e06021cd6624e50af2c914399ebd6c4 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:15.368562 21627 leader_election.cc:304] T c18c1d46a6ca47598d7b161ed70a4e7a P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e; no voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:58:15.369262 23121 raft_consensus.cc:2743] T c18c1d46a6ca47598d7b161ed70a4e7a P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20250109 08:58:15.369872 23147 leader_election.cc:290] T 0e06021cd6624e50af2c914399ebd6c4 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 2 election: Requested vote from peers a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 88935ab835544e9889d05a6587d8cf21 (127.19.226.196:44363)
I20250109 08:58:15.376058 22177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e06021cd6624e50af2c914399ebd6c4" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "88935ab835544e9889d05a6587d8cf21"
I20250109 08:58:15.376734 22177 raft_consensus.cc:3054] T 0e06021cd6624e50af2c914399ebd6c4 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:15.378937 21412 catalog_manager.cc:5526] T 7781c00473024307987096b596da12ac P 787ddcc824c2490a808fc5a53452dc5e reported cstate change: term changed from 1 to 2, leader changed from a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194) to 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195). New cstate: current_term: 2 leader_uuid: "787ddcc824c2490a808fc5a53452dc5e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20250109 08:58:15.382019 21627 leader_election.cc:336] T 0e06021cd6624e50af2c914399ebd6c4 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Network error: Client connection negotiation failed: client connection to 127.19.226.194:45527: connect: Connection refused (error 111)
I20250109 08:58:15.383047 22177 raft_consensus.cc:2463] T 0e06021cd6624e50af2c914399ebd6c4 P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 2.
I20250109 08:58:15.384234 21627 leader_election.cc:304] T 0e06021cd6624e50af2c914399ebd6c4 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, 88935ab835544e9889d05a6587d8cf21; no voters: a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:58:15.385267 23146 raft_consensus.cc:2798] T 0e06021cd6624e50af2c914399ebd6c4 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:58:15.385927 23146 raft_consensus.cc:695] T 0e06021cd6624e50af2c914399ebd6c4 P 787ddcc824c2490a808fc5a53452dc5e [term 2 LEADER]: Becoming Leader. State: Replica: 787ddcc824c2490a808fc5a53452dc5e, State: Running, Role: LEADER
I20250109 08:58:15.386762 23146 consensus_queue.cc:237] T 0e06021cd6624e50af2c914399ebd6c4 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:15.403553 21412 catalog_manager.cc:5526] T 0e06021cd6624e50af2c914399ebd6c4 P 787ddcc824c2490a808fc5a53452dc5e reported cstate change: term changed from 1 to 2, leader changed from a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194) to 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195). New cstate: current_term: 2 leader_uuid: "787ddcc824c2490a808fc5a53452dc5e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:15.405616 23127 raft_consensus.cc:491] T 90978f185a374feba3bc19b54f9d97f5 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2b1a692435241ee988e0bd1661cd4c3)
I20250109 08:58:15.406133 23127 raft_consensus.cc:513] T 90978f185a374feba3bc19b54f9d97f5 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:15.407768 23127 leader_election.cc:290] T 90978f185a374feba3bc19b54f9d97f5 P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
W20250109 08:58:15.415514 22142 leader_election.cc:336] T 90978f185a374feba3bc19b54f9d97f5 P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Network error: Client connection negotiation failed: client connection to 127.19.226.194:45527: connect: Connection refused (error 111)
W20250109 08:58:15.416901 22141 leader_election.cc:336] T 90978f185a374feba3bc19b54f9d97f5 P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Network error: Client connection negotiation failed: client connection to 127.19.226.193:42725: connect: Connection refused (error 111)
I20250109 08:58:15.417356 22141 leader_election.cc:304] T 90978f185a374feba3bc19b54f9d97f5 P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88935ab835544e9889d05a6587d8cf21; no voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:58:15.417990 23127 raft_consensus.cc:2743] T 90978f185a374feba3bc19b54f9d97f5 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20250109 08:58:15.419085 21626 consensus_peers.cc:487] T 79c1ed8ab551478291f1f24cc940cb27 P 787ddcc824c2490a808fc5a53452dc5e -> Peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Couldn't send request to peer 8bf6e26e98b14566899578a990b27d47. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.193:42725: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20250109 08:58:15.428897 21627 consensus_peers.cc:487] T d355d166de534fb4a50c9f48f2df7469 P 787ddcc824c2490a808fc5a53452dc5e -> Peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Couldn't send request to peer a2b1a692435241ee988e0bd1661cd4c3. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.194:45527: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20250109 08:58:15.429955 21626 consensus_peers.cc:487] T af273bab41cc4a9da780147eebfd52ec P 787ddcc824c2490a808fc5a53452dc5e -> Peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Couldn't send request to peer 8bf6e26e98b14566899578a990b27d47. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.193:42725: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20250109 08:58:15.492441 22141 consensus_peers.cc:487] T 239ca74a4c0a455a9db2652d7debb989 P 88935ab835544e9889d05a6587d8cf21 -> Peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Couldn't send request to peer 8bf6e26e98b14566899578a990b27d47. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.193:42725: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20250109 08:58:15.513285 21626 consensus_peers.cc:487] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P 787ddcc824c2490a808fc5a53452dc5e -> Peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Couldn't send request to peer 8bf6e26e98b14566899578a990b27d47. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.193:42725: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20250109 08:58:15.563966 22141 consensus_peers.cc:487] T 2935d69446cd458e92046d2b2f1b74ab P 88935ab835544e9889d05a6587d8cf21 -> Peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Couldn't send request to peer 8bf6e26e98b14566899578a990b27d47. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.193:42725: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20250109 08:58:15.566589 22141 consensus_peers.cc:487] T 37280454805d47f98a415a8f0c16d869 P 88935ab835544e9889d05a6587d8cf21 -> Peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Couldn't send request to peer 8bf6e26e98b14566899578a990b27d47. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.193:42725: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20250109 08:58:15.599475 22177 raft_consensus.cc:1270] T d355d166de534fb4a50c9f48f2df7469 P 88935ab835544e9889d05a6587d8cf21 [term 3 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20250109 08:58:15.600937 23147 consensus_queue.cc:1035] T d355d166de534fb4a50c9f48f2df7469 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20250109 08:58:15.604725 21626 consensus_peers.cc:487] T 75edb5d04fa14201ad711c1d95c81f64 P 787ddcc824c2490a808fc5a53452dc5e -> Peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Couldn't send request to peer 8bf6e26e98b14566899578a990b27d47. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.193:42725: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20250109 08:58:15.631034 21663 raft_consensus.cc:1270] T 1d3d5d2a2c88417a97c9d57262a356dd P 787ddcc824c2490a808fc5a53452dc5e [term 3 FOLLOWER]: Refusing update from remote peer 88935ab835544e9889d05a6587d8cf21: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20250109 08:58:15.632458 23127 consensus_queue.cc:1035] T 1d3d5d2a2c88417a97c9d57262a356dd P 88935ab835544e9889d05a6587d8cf21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20250109 08:58:15.647182 22142 consensus_peers.cc:487] T 1d3d5d2a2c88417a97c9d57262a356dd P 88935ab835544e9889d05a6587d8cf21 -> Peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Couldn't send request to peer a2b1a692435241ee988e0bd1661cd4c3. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.194:45527: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20250109 08:58:15.657716 21626 consensus_peers.cc:487] T 80123146836349938344efc61333695f P 787ddcc824c2490a808fc5a53452dc5e -> Peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Couldn't send request to peer 8bf6e26e98b14566899578a990b27d47. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.193:42725: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20250109 08:58:15.675755 21660 raft_consensus.cc:1270] T eb4516b9a16041c2adf34be520768f3f P 787ddcc824c2490a808fc5a53452dc5e [term 3 FOLLOWER]: Refusing update from remote peer 88935ab835544e9889d05a6587d8cf21: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20250109 08:58:15.677383 23128 consensus_queue.cc:1035] T eb4516b9a16041c2adf34be520768f3f P 88935ab835544e9889d05a6587d8cf21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20250109 08:58:15.692524 22142 consensus_peers.cc:487] T eb4516b9a16041c2adf34be520768f3f P 88935ab835544e9889d05a6587d8cf21 -> Peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Couldn't send request to peer a2b1a692435241ee988e0bd1661cd4c3. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.194:45527: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20250109 08:58:15.781045 21627 consensus_peers.cc:487] T f5f85ad7c7c94acfaf9d550d55fce48c P 787ddcc824c2490a808fc5a53452dc5e -> Peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Couldn't send request to peer a2b1a692435241ee988e0bd1661cd4c3. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.194:45527: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:15.808866 22177 raft_consensus.cc:1270] T 7781c00473024307987096b596da12ac P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250109 08:58:15.811143 23146 consensus_queue.cc:1035] T 7781c00473024307987096b596da12ac P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250109 08:58:15.812700 22177 raft_consensus.cc:1270] T f5f85ad7c7c94acfaf9d550d55fce48c P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250109 08:58:15.814424 23146 consensus_queue.cc:1035] T f5f85ad7c7c94acfaf9d550d55fce48c P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20250109 08:58:15.844748 21626 consensus_peers.cc:487] T 5ac5ffe3cd65463fb448e819e3978841 P 787ddcc824c2490a808fc5a53452dc5e -> Peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Couldn't send request to peer 8bf6e26e98b14566899578a990b27d47. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.193:42725: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20250109 08:58:15.855253 21627 consensus_peers.cc:487] T 7781c00473024307987096b596da12ac P 787ddcc824c2490a808fc5a53452dc5e -> Peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Couldn't send request to peer a2b1a692435241ee988e0bd1661cd4c3. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.194:45527: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20250109 08:58:15.858693 21626 consensus_peers.cc:487] T d8493108cba04514bd2236192cef1079 P 787ddcc824c2490a808fc5a53452dc5e -> Peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Couldn't send request to peer 8bf6e26e98b14566899578a990b27d47. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.193:42725: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20250109 08:58:15.884713 21627 consensus_peers.cc:487] T 0e06021cd6624e50af2c914399ebd6c4 P 787ddcc824c2490a808fc5a53452dc5e -> Peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Couldn't send request to peer a2b1a692435241ee988e0bd1661cd4c3. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.194:45527: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20250109 08:58:15.894503 21626 consensus_peers.cc:487] T ab6c3fcdf226433c9e1303004f6c96b4 P 787ddcc824c2490a808fc5a53452dc5e -> Peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Couldn't send request to peer 8bf6e26e98b14566899578a990b27d47. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.193:42725: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20250109 08:58:15.944576 22177 raft_consensus.cc:1270] T 0e06021cd6624e50af2c914399ebd6c4 P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Refusing update from remote peer 787ddcc824c2490a808fc5a53452dc5e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250109 08:58:15.945830 23146 consensus_queue.cc:1035] T 0e06021cd6624e50af2c914399ebd6c4 P 787ddcc824c2490a808fc5a53452dc5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20250109 08:58:15.950734 21626 consensus_peers.cc:487] T a3aa271f2f0144eea9261e0e07ab833b P 787ddcc824c2490a808fc5a53452dc5e -> Peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Couldn't send request to peer 8bf6e26e98b14566899578a990b27d47. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.193:42725: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20250109 08:58:15.977810 22141 consensus_peers.cc:487] T df9715570a6844d9a2686307db16eff1 P 88935ab835544e9889d05a6587d8cf21 -> Peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Couldn't send request to peer 8bf6e26e98b14566899578a990b27d47. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.193:42725: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20250109 08:58:15.983634 23172 raft_consensus.cc:491] T 363fa693a5114f2bb22f4ab67722a1c6 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8bf6e26e98b14566899578a990b27d47)
I20250109 08:58:15.984220 23172 raft_consensus.cc:513] T 363fa693a5114f2bb22f4ab67722a1c6 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } attrs { replace: true } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:58:15.986373 23172 leader_election.cc:290] T 363fa693a5114f2bb22f4ab67722a1c6 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
W20250109 08:58:15.991065 21626 leader_election.cc:336] T 363fa693a5114f2bb22f4ab67722a1c6 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Network error: Client connection negotiation failed: client connection to 127.19.226.193:42725: connect: Connection refused (error 111)
W20250109 08:58:15.994239 21627 leader_election.cc:336] T 363fa693a5114f2bb22f4ab67722a1c6 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Network error: Client connection negotiation failed: client connection to 127.19.226.194:45527: connect: Connection refused (error 111)
I20250109 08:58:15.994668 21627 leader_election.cc:304] T 363fa693a5114f2bb22f4ab67722a1c6 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e; no voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:58:15.995337 23172 raft_consensus.cc:2743] T 363fa693a5114f2bb22f4ab67722a1c6 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20250109 08:58:16.012946 22142 consensus_peers.cc:487] T d52f79ecf037457cbcc09d5fa43456cf P 88935ab835544e9889d05a6587d8cf21 -> Peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Couldn't send request to peer a2b1a692435241ee988e0bd1661cd4c3. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.194:45527: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20250109 08:58:16.077179 23172 raft_consensus.cc:491] T e03846e461974d0d8944c344e715b86a P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8bf6e26e98b14566899578a990b27d47)
I20250109 08:58:16.077705 23172 raft_consensus.cc:513] T e03846e461974d0d8944c344e715b86a P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
W20250109 08:58:16.081245 21626 consensus_peers.cc:487] T 89e1eeb24fea483eb63bd98af66e0b6b P 787ddcc824c2490a808fc5a53452dc5e -> Peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Couldn't send request to peer 8bf6e26e98b14566899578a990b27d47. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.193:42725: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20250109 08:58:16.081440 23172 leader_election.cc:290] T e03846e461974d0d8944c344e715b86a P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
W20250109 08:58:16.083572 21627 leader_election.cc:336] T e03846e461974d0d8944c344e715b86a P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Network error: Client connection negotiation failed: client connection to 127.19.226.194:45527: connect: Connection refused (error 111)
W20250109 08:58:16.084070 21626 leader_election.cc:336] T e03846e461974d0d8944c344e715b86a P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Network error: Client connection negotiation failed: client connection to 127.19.226.193:42725: connect: Connection refused (error 111)
I20250109 08:58:16.084501 21626 leader_election.cc:304] T e03846e461974d0d8944c344e715b86a P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e; no voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:58:16.085282 23172 raft_consensus.cc:2743] T e03846e461974d0d8944c344e715b86a P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20250109 08:58:16.087769 23031 raft_consensus.cc:491] T 97304b06c7fa40ef8345a72c796c47f8 P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8bf6e26e98b14566899578a990b27d47)
I20250109 08:58:16.088270 23031 raft_consensus.cc:513] T 97304b06c7fa40ef8345a72c796c47f8 P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:16.090159 23031 leader_election.cc:290] T 97304b06c7fa40ef8345a72c796c47f8 P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
W20250109 08:58:16.093734 22141 leader_election.cc:336] T 97304b06c7fa40ef8345a72c796c47f8 P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Network error: Client connection negotiation failed: client connection to 127.19.226.193:42725: connect: Connection refused (error 111)
W20250109 08:58:16.100533 22142 leader_election.cc:336] T 97304b06c7fa40ef8345a72c796c47f8 P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Network error: Client connection negotiation failed: client connection to 127.19.226.194:45527: connect: Connection refused (error 111)
I20250109 08:58:16.100968 22142 leader_election.cc:304] T 97304b06c7fa40ef8345a72c796c47f8 P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88935ab835544e9889d05a6587d8cf21; no voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:58:16.101634 23031 raft_consensus.cc:2743] T 97304b06c7fa40ef8345a72c796c47f8 P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20250109 08:58:16.111402 20363 thread.cc:535] Waited for 2000ms trying to join with diag-logger (tid 21610)
W20250109 08:58:16.115711 21626 consensus_peers.cc:487] T 7c19ed9fdde5463f84e43bea7b860d03 P 787ddcc824c2490a808fc5a53452dc5e -> Peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Couldn't send request to peer 8bf6e26e98b14566899578a990b27d47. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.193:42725: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20250109 08:58:16.145342 21626 consensus_peers.cc:487] T af330dcd4767423ea4c179be7f88d0c4 P 787ddcc824c2490a808fc5a53452dc5e -> Peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Couldn't send request to peer 8bf6e26e98b14566899578a990b27d47. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.193:42725: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20250109 08:58:16.162400 21627 consensus_peers.cc:487] T 89e1eeb24fea483eb63bd98af66e0b6b P 787ddcc824c2490a808fc5a53452dc5e -> Peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Couldn't send request to peer a2b1a692435241ee988e0bd1661cd4c3. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.194:45527: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20250109 08:58:16.180019 21627 consensus_peers.cc:487] T 75edb5d04fa14201ad711c1d95c81f64 P 787ddcc824c2490a808fc5a53452dc5e -> Peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Couldn't send request to peer a2b1a692435241ee988e0bd1661cd4c3. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.194:45527: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20250109 08:58:16.206187 23121 raft_consensus.cc:491] T c99aacce09d44f889be5480854a22760 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8bf6e26e98b14566899578a990b27d47)
I20250109 08:58:16.206734 23121 raft_consensus.cc:513] T c99aacce09d44f889be5480854a22760 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:58:16.208300 23121 leader_election.cc:290] T c99aacce09d44f889be5480854a22760 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
I20250109 08:58:16.210320 23172 raft_consensus.cc:491] T c761cb691bd241ae825917a3fbc3ad24 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8bf6e26e98b14566899578a990b27d47)
I20250109 08:58:16.210810 23172 raft_consensus.cc:513] T c761cb691bd241ae825917a3fbc3ad24 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:58:16.212682 23172 leader_election.cc:290] T c761cb691bd241ae825917a3fbc3ad24 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
W20250109 08:58:16.213559 21627 leader_election.cc:336] T c99aacce09d44f889be5480854a22760 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Network error: Client connection negotiation failed: client connection to 127.19.226.194:45527: connect: Connection refused (error 111)
W20250109 08:58:16.214005 21626 leader_election.cc:336] T c99aacce09d44f889be5480854a22760 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Network error: Client connection negotiation failed: client connection to 127.19.226.193:42725: connect: Connection refused (error 111)
I20250109 08:58:16.214465 21626 leader_election.cc:304] T c99aacce09d44f889be5480854a22760 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e; no voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:58:16.216125 23172 raft_consensus.cc:2743] T c99aacce09d44f889be5480854a22760 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20250109 08:58:16.217006 21626 leader_election.cc:336] T c761cb691bd241ae825917a3fbc3ad24 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Network error: Client connection negotiation failed: client connection to 127.19.226.193:42725: connect: Connection refused (error 111)
W20250109 08:58:16.219861 21627 leader_election.cc:336] T c761cb691bd241ae825917a3fbc3ad24 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Network error: Client connection negotiation failed: client connection to 127.19.226.194:45527: connect: Connection refused (error 111)
I20250109 08:58:16.220248 21627 leader_election.cc:304] T c761cb691bd241ae825917a3fbc3ad24 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e; no voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:58:16.221261 23172 raft_consensus.cc:2743] T c761cb691bd241ae825917a3fbc3ad24 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20250109 08:58:16.261411 23031 raft_consensus.cc:491] T db34a00c02b84ee988edc7d6e344c1f5 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8bf6e26e98b14566899578a990b27d47)
I20250109 08:58:16.261886 23031 raft_consensus.cc:513] T db34a00c02b84ee988edc7d6e344c1f5 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:16.263525 23031 leader_election.cc:290] T db34a00c02b84ee988edc7d6e344c1f5 P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
W20250109 08:58:16.270733 22142 leader_election.cc:336] T db34a00c02b84ee988edc7d6e344c1f5 P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Network error: Client connection negotiation failed: client connection to 127.19.226.194:45527: connect: Connection refused (error 111)
W20250109 08:58:16.272740 22141 leader_election.cc:336] T db34a00c02b84ee988edc7d6e344c1f5 P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Network error: Client connection negotiation failed: client connection to 127.19.226.193:42725: connect: Connection refused (error 111)
I20250109 08:58:16.273170 22141 leader_election.cc:304] T db34a00c02b84ee988edc7d6e344c1f5 P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88935ab835544e9889d05a6587d8cf21; no voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:58:16.275719 23031 raft_consensus.cc:2743] T db34a00c02b84ee988edc7d6e344c1f5 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20250109 08:58:16.291009 21627 consensus_peers.cc:487] T 80123146836349938344efc61333695f P 787ddcc824c2490a808fc5a53452dc5e -> Peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Couldn't send request to peer a2b1a692435241ee988e0bd1661cd4c3. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.194:45527: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20250109 08:58:16.298190 23031 raft_consensus.cc:491] T 71ac0243d72845659291184835652dde P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8bf6e26e98b14566899578a990b27d47)
I20250109 08:58:16.298710 23031 raft_consensus.cc:513] T 71ac0243d72845659291184835652dde P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:16.300146 23031 leader_election.cc:290] T 71ac0243d72845659291184835652dde P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
W20250109 08:58:16.306926 22141 leader_election.cc:336] T 71ac0243d72845659291184835652dde P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Network error: Client connection negotiation failed: client connection to 127.19.226.193:42725: connect: Connection refused (error 111)
W20250109 08:58:16.308549 22142 leader_election.cc:336] T 71ac0243d72845659291184835652dde P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Network error: Client connection negotiation failed: client connection to 127.19.226.194:45527: connect: Connection refused (error 111)
I20250109 08:58:16.309000 22142 leader_election.cc:304] T 71ac0243d72845659291184835652dde P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88935ab835544e9889d05a6587d8cf21; no voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:58:16.309839 23031 raft_consensus.cc:2743] T 71ac0243d72845659291184835652dde P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20250109 08:58:16.316824 22142 consensus_peers.cc:487] T b00671c87aac405a81c6543f4defd722 P 88935ab835544e9889d05a6587d8cf21 -> Peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Couldn't send request to peer a2b1a692435241ee988e0bd1661cd4c3. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.194:45527: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20250109 08:58:16.384124 21627 consensus_peers.cc:487] T 7a7211b8d4374ad1afd82e8cf85bad81 P 787ddcc824c2490a808fc5a53452dc5e -> Peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Couldn't send request to peer a2b1a692435241ee988e0bd1661cd4c3. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.194:45527: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20250109 08:58:16.387668 21627 consensus_peers.cc:487] T af273bab41cc4a9da780147eebfd52ec P 787ddcc824c2490a808fc5a53452dc5e -> Peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Couldn't send request to peer a2b1a692435241ee988e0bd1661cd4c3. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.194:45527: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20250109 08:58:16.436270 21627 consensus_peers.cc:487] T ab6c3fcdf226433c9e1303004f6c96b4 P 787ddcc824c2490a808fc5a53452dc5e -> Peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Couldn't send request to peer a2b1a692435241ee988e0bd1661cd4c3. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.194:45527: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20250109 08:58:16.437922 21627 consensus_peers.cc:487] T a3aa271f2f0144eea9261e0e07ab833b P 787ddcc824c2490a808fc5a53452dc5e -> Peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Couldn't send request to peer a2b1a692435241ee988e0bd1661cd4c3. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.194:45527: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20250109 08:58:16.464541 21627 consensus_peers.cc:487] T 0f3bde1a1ccd4c359a2e80feca968453 P 787ddcc824c2490a808fc5a53452dc5e -> Peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Couldn't send request to peer a2b1a692435241ee988e0bd1661cd4c3. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.194:45527: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20250109 08:58:16.477082 23121 raft_consensus.cc:491] T e38ec0ef84af4b798073eafd4d210c35 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8bf6e26e98b14566899578a990b27d47)
I20250109 08:58:16.477600 23121 raft_consensus.cc:513] T e38ec0ef84af4b798073eafd4d210c35 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } }
I20250109 08:58:16.479136 23121 leader_election.cc:290] T e38ec0ef84af4b798073eafd4d210c35 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
W20250109 08:58:16.484102 21627 consensus_peers.cc:487] T 79c1ed8ab551478291f1f24cc940cb27 P 787ddcc824c2490a808fc5a53452dc5e -> Peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Couldn't send request to peer a2b1a692435241ee988e0bd1661cd4c3. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.194:45527: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20250109 08:58:16.484448 21626 leader_election.cc:336] T e38ec0ef84af4b798073eafd4d210c35 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Network error: Client connection negotiation failed: client connection to 127.19.226.193:42725: connect: Connection refused (error 111)
W20250109 08:58:16.485555 22142 consensus_peers.cc:487] T 2935d69446cd458e92046d2b2f1b74ab P 88935ab835544e9889d05a6587d8cf21 -> Peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Couldn't send request to peer a2b1a692435241ee988e0bd1661cd4c3. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.194:45527: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20250109 08:58:16.487126 21627 leader_election.cc:336] T e38ec0ef84af4b798073eafd4d210c35 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Network error: Client connection negotiation failed: client connection to 127.19.226.194:45527: connect: Connection refused (error 111)
I20250109 08:58:16.487548 21627 leader_election.cc:304] T e38ec0ef84af4b798073eafd4d210c35 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e; no voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:58:16.488114 23121 raft_consensus.cc:2743] T e38ec0ef84af4b798073eafd4d210c35 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20250109 08:58:16.558604 22142 consensus_peers.cc:487] T d015a10f95eb4f19b6d7c53cb201fa80 P 88935ab835544e9889d05a6587d8cf21 -> Peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Couldn't send request to peer a2b1a692435241ee988e0bd1661cd4c3. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.194:45527: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20250109 08:58:16.571836 22142 consensus_peers.cc:487] T 81d74e7d4472497aac5463a4b47e4fb9 P 88935ab835544e9889d05a6587d8cf21 -> Peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Couldn't send request to peer a2b1a692435241ee988e0bd1661cd4c3. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.194:45527: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20250109 08:58:16.582746 22142 consensus_peers.cc:487] T 90a4a2381cbe40f0b204be325ee9fea1 P 88935ab835544e9889d05a6587d8cf21 -> Peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Couldn't send request to peer a2b1a692435241ee988e0bd1661cd4c3. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.194:45527: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20250109 08:58:16.592672 23128 raft_consensus.cc:491] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8bf6e26e98b14566899578a990b27d47)
I20250109 08:58:16.593514 23128 raft_consensus.cc:513] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:16.595407 23128 leader_election.cc:290] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
W20250109 08:58:16.601904 22141 leader_election.cc:336] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Network error: Client connection negotiation failed: client connection to 127.19.226.193:42725: connect: Connection refused (error 111)
W20250109 08:58:16.603358 22142 leader_election.cc:336] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Network error: Client connection negotiation failed: client connection to 127.19.226.194:45527: connect: Connection refused (error 111)
I20250109 08:58:16.603777 22142 leader_election.cc:304] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88935ab835544e9889d05a6587d8cf21; no voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:58:16.604733 23128 raft_consensus.cc:2743] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20250109 08:58:16.620743 21627 consensus_peers.cc:487] T af330dcd4767423ea4c179be7f88d0c4 P 787ddcc824c2490a808fc5a53452dc5e -> Peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Couldn't send request to peer a2b1a692435241ee988e0bd1661cd4c3. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.194:45527: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20250109 08:58:16.637286 22142 consensus_peers.cc:487] T 306c4892bbff4dbfbe5a0645294aa627 P 88935ab835544e9889d05a6587d8cf21 -> Peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Couldn't send request to peer a2b1a692435241ee988e0bd1661cd4c3. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.194:45527: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20250109 08:58:16.657135 23172 raft_consensus.cc:491] T ddd14cdbefd64936960e9ffb5b38b5f2 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8bf6e26e98b14566899578a990b27d47)
I20250109 08:58:16.657672 23172 raft_consensus.cc:513] T ddd14cdbefd64936960e9ffb5b38b5f2 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:58:16.660441 23172 leader_election.cc:290] T ddd14cdbefd64936960e9ffb5b38b5f2 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
W20250109 08:58:16.666153 21627 leader_election.cc:336] T ddd14cdbefd64936960e9ffb5b38b5f2 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Network error: Client connection negotiation failed: client connection to 127.19.226.194:45527: connect: Connection refused (error 111)
W20250109 08:58:16.668282 21626 leader_election.cc:336] T ddd14cdbefd64936960e9ffb5b38b5f2 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Network error: Client connection negotiation failed: client connection to 127.19.226.193:42725: connect: Connection refused (error 111)
I20250109 08:58:16.668725 21626 leader_election.cc:304] T ddd14cdbefd64936960e9ffb5b38b5f2 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e; no voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:58:16.671828 23121 raft_consensus.cc:2743] T ddd14cdbefd64936960e9ffb5b38b5f2 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20250109 08:58:16.695358 21627 consensus_peers.cc:487] T 64c667f887ea41b4a484ee565b21fa16 P 787ddcc824c2490a808fc5a53452dc5e -> Peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Couldn't send request to peer a2b1a692435241ee988e0bd1661cd4c3. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.194:45527: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20250109 08:58:16.698256 23128 raft_consensus.cc:491] T e601a034e10e418bac8683443d86a1c1 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8bf6e26e98b14566899578a990b27d47)
I20250109 08:58:16.698822 23128 raft_consensus.cc:513] T e601a034e10e418bac8683443d86a1c1 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:16.701021 23128 leader_election.cc:290] T e601a034e10e418bac8683443d86a1c1 P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
W20250109 08:58:16.704540 22141 leader_election.cc:336] T e601a034e10e418bac8683443d86a1c1 P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Network error: Client connection negotiation failed: client connection to 127.19.226.193:42725: connect: Connection refused (error 111)
W20250109 08:58:16.708772 22142 leader_election.cc:336] T e601a034e10e418bac8683443d86a1c1 P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Network error: Client connection negotiation failed: client connection to 127.19.226.194:45527: connect: Connection refused (error 111)
I20250109 08:58:16.709218 22142 leader_election.cc:304] T e601a034e10e418bac8683443d86a1c1 P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88935ab835544e9889d05a6587d8cf21; no voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:58:16.709890 23128 raft_consensus.cc:2743] T e601a034e10e418bac8683443d86a1c1 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250109 08:58:16.713902 23121 raft_consensus.cc:491] T 610441ef4b1d40c284eff3a714e4214e P 787ddcc824c2490a808fc5a53452dc5e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8bf6e26e98b14566899578a990b27d47)
I20250109 08:58:16.714287 23121 raft_consensus.cc:513] T 610441ef4b1d40c284eff3a714e4214e P 787ddcc824c2490a808fc5a53452dc5e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:58:16.716004 23121 leader_election.cc:290] T 610441ef4b1d40c284eff3a714e4214e P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
W20250109 08:58:16.720026 21626 leader_election.cc:336] T 610441ef4b1d40c284eff3a714e4214e P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Network error: Client connection negotiation failed: client connection to 127.19.226.193:42725: connect: Connection refused (error 111)
W20250109 08:58:16.720872 21627 leader_election.cc:336] T 610441ef4b1d40c284eff3a714e4214e P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Network error: Client connection negotiation failed: client connection to 127.19.226.194:45527: connect: Connection refused (error 111)
I20250109 08:58:16.721289 21627 leader_election.cc:304] T 610441ef4b1d40c284eff3a714e4214e P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e; no voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:58:16.721979 23172 raft_consensus.cc:2743] T 610441ef4b1d40c284eff3a714e4214e P 787ddcc824c2490a808fc5a53452dc5e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20250109 08:58:16.773335 23172 raft_consensus.cc:491] T 8c0459d2eee64f0a83bd05ad5974f506 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a2b1a692435241ee988e0bd1661cd4c3)
I20250109 08:58:16.773849 23172 raft_consensus.cc:513] T 8c0459d2eee64f0a83bd05ad5974f506 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } attrs { replace: true } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:16.776211 23172 leader_election.cc:290] T 8c0459d2eee64f0a83bd05ad5974f506 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527), 88935ab835544e9889d05a6587d8cf21 (127.19.226.196:44363)
I20250109 08:58:16.777396 22173 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c0459d2eee64f0a83bd05ad5974f506" candidate_uuid: "787ddcc824c2490a808fc5a53452dc5e" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "88935ab835544e9889d05a6587d8cf21" is_pre_election: true
I20250109 08:58:16.778045 22173 raft_consensus.cc:2463] T 8c0459d2eee64f0a83bd05ad5974f506 P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 787ddcc824c2490a808fc5a53452dc5e in term 2.
W20250109 08:58:16.780927 21626 leader_election.cc:336] T 8c0459d2eee64f0a83bd05ad5974f506 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Network error: Client connection negotiation failed: client connection to 127.19.226.193:42725: connect: Connection refused (error 111)
W20250109 08:58:16.781504 21627 leader_election.cc:336] T 8c0459d2eee64f0a83bd05ad5974f506 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Network error: Client connection negotiation failed: client connection to 127.19.226.194:45527: connect: Connection refused (error 111)
I20250109 08:58:16.781898 21627 leader_election.cc:304] T 8c0459d2eee64f0a83bd05ad5974f506 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e, 88935ab835544e9889d05a6587d8cf21; no voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:58:16.782560 23172 raft_consensus.cc:2743] T 8c0459d2eee64f0a83bd05ad5974f506 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20250109 08:58:16.798627 23031 raft_consensus.cc:491] T 0cc4fcda876242c396b804edebebb01f P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a2b1a692435241ee988e0bd1661cd4c3)
I20250109 08:58:16.799094 23031 raft_consensus.cc:513] T 0cc4fcda876242c396b804edebebb01f P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:16.800709 23031 leader_election.cc:290] T 0cc4fcda876242c396b804edebebb01f P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
W20250109 08:58:16.805337 22141 leader_election.cc:336] T 0cc4fcda876242c396b804edebebb01f P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Network error: Client connection negotiation failed: client connection to 127.19.226.193:42725: connect: Connection refused (error 111)
W20250109 08:58:16.806434 22142 leader_election.cc:336] T 0cc4fcda876242c396b804edebebb01f P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Network error: Client connection negotiation failed: client connection to 127.19.226.194:45527: connect: Connection refused (error 111)
I20250109 08:58:16.806852 22142 leader_election.cc:304] T 0cc4fcda876242c396b804edebebb01f P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88935ab835544e9889d05a6587d8cf21; no voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:58:16.807374 23031 raft_consensus.cc:2743] T 0cc4fcda876242c396b804edebebb01f P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20250109 08:58:16.828503 23031 raft_consensus.cc:491] T 3b5049c00b0b4dd89c49fa5d13db169e P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2b1a692435241ee988e0bd1661cd4c3)
I20250109 08:58:16.828985 23031 raft_consensus.cc:513] T 3b5049c00b0b4dd89c49fa5d13db169e P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:16.830727 23031 leader_election.cc:290] T 3b5049c00b0b4dd89c49fa5d13db169e P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
W20250109 08:58:16.834010 22141 leader_election.cc:336] T 3b5049c00b0b4dd89c49fa5d13db169e P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Network error: Client connection negotiation failed: client connection to 127.19.226.193:42725: connect: Connection refused (error 111)
W20250109 08:58:16.835268 22142 leader_election.cc:336] T 3b5049c00b0b4dd89c49fa5d13db169e P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Network error: Client connection negotiation failed: client connection to 127.19.226.194:45527: connect: Connection refused (error 111)
I20250109 08:58:16.835677 22142 leader_election.cc:304] T 3b5049c00b0b4dd89c49fa5d13db169e P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88935ab835544e9889d05a6587d8cf21; no voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:58:16.836328 23031 raft_consensus.cc:2743] T 3b5049c00b0b4dd89c49fa5d13db169e P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250109 08:58:16.934913 23031 raft_consensus.cc:491] T 56bbc306275e4a409e028a7ca1dd1602 P 88935ab835544e9889d05a6587d8cf21 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a2b1a692435241ee988e0bd1661cd4c3)
I20250109 08:58:16.935411 23031 raft_consensus.cc:513] T 56bbc306275e4a409e028a7ca1dd1602 P 88935ab835544e9889d05a6587d8cf21 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:16.947081 23031 leader_election.cc:290] T 56bbc306275e4a409e028a7ca1dd1602 P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
I20250109 08:58:16.949555 23128 raft_consensus.cc:491] T 180b4d539a144a38905bf340ecb24ef4 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8bf6e26e98b14566899578a990b27d47)
W20250109 08:58:16.957844 22142 leader_election.cc:336] T 56bbc306275e4a409e028a7ca1dd1602 P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Network error: Client connection negotiation failed: client connection to 127.19.226.194:45527: connect: Connection refused (error 111)
W20250109 08:58:16.958397 22141 leader_election.cc:336] T 56bbc306275e4a409e028a7ca1dd1602 P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Network error: Client connection negotiation failed: client connection to 127.19.226.193:42725: connect: Connection refused (error 111)
I20250109 08:58:16.957906 23128 raft_consensus.cc:513] T 180b4d539a144a38905bf340ecb24ef4 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:16.960065 23128 leader_election.cc:290] T 180b4d539a144a38905bf340ecb24ef4 P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
I20250109 08:58:16.958866 22141 leader_election.cc:304] T 56bbc306275e4a409e028a7ca1dd1602 P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88935ab835544e9889d05a6587d8cf21; no voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:58:16.963829 23128 raft_consensus.cc:2743] T 56bbc306275e4a409e028a7ca1dd1602 P 88935ab835544e9889d05a6587d8cf21 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20250109 08:58:16.967125 22142 leader_election.cc:336] T 180b4d539a144a38905bf340ecb24ef4 P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Network error: Client connection negotiation failed: client connection to 127.19.226.194:45527: connect: Connection refused (error 111)
W20250109 08:58:16.971871 22141 leader_election.cc:336] T 180b4d539a144a38905bf340ecb24ef4 P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Network error: Client connection negotiation failed: client connection to 127.19.226.193:42725: connect: Connection refused (error 111)
I20250109 08:58:16.972640 22141 leader_election.cc:304] T 180b4d539a144a38905bf340ecb24ef4 P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88935ab835544e9889d05a6587d8cf21; no voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:58:16.973306 23128 raft_consensus.cc:2743] T 180b4d539a144a38905bf340ecb24ef4 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250109 08:58:17.023412 23172 raft_consensus.cc:491] T f961d3f357944118a69f905621731552 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2b1a692435241ee988e0bd1661cd4c3)
I20250109 08:58:17.023933 23172 raft_consensus.cc:513] T f961d3f357944118a69f905621731552 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:58:17.027309 23172 leader_election.cc:290] T f961d3f357944118a69f905621731552 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
W20250109 08:58:17.031104 21626 leader_election.cc:336] T f961d3f357944118a69f905621731552 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Network error: Client connection negotiation failed: client connection to 127.19.226.193:42725: connect: Connection refused (error 111)
W20250109 08:58:17.034013 21627 leader_election.cc:336] T f961d3f357944118a69f905621731552 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Network error: Client connection negotiation failed: client connection to 127.19.226.194:45527: connect: Connection refused (error 111)
I20250109 08:58:17.034389 21627 leader_election.cc:304] T f961d3f357944118a69f905621731552 P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e; no voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:58:17.035072 23172 raft_consensus.cc:2743] T f961d3f357944118a69f905621731552 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250109 08:58:17.052037 20363 tablet_server.cc:195] TabletServer@127.19.226.194:0 shutdown complete.
I20250109 08:58:17.159798 23172 raft_consensus.cc:491] T c18c1d46a6ca47598d7b161ed70a4e7a P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a2b1a692435241ee988e0bd1661cd4c3)
I20250109 08:58:17.160429 23172 raft_consensus.cc:513] T c18c1d46a6ca47598d7b161ed70a4e7a P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } }
I20250109 08:58:17.162655 23172 leader_election.cc:290] T c18c1d46a6ca47598d7b161ed70a4e7a P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
I20250109 08:58:17.164885 20363 tablet_server.cc:178] TabletServer@127.19.226.195:0 shutting down...
W20250109 08:58:17.167096 21626 leader_election.cc:336] T c18c1d46a6ca47598d7b161ed70a4e7a P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Network error: Client connection negotiation failed: client connection to 127.19.226.193:42725: connect: Connection refused (error 111)
W20250109 08:58:17.167929 21627 leader_election.cc:336] T c18c1d46a6ca47598d7b161ed70a4e7a P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Network error: Client connection negotiation failed: client connection to 127.19.226.194:45527: connect: Connection refused (error 111)
I20250109 08:58:17.168352 21627 leader_election.cc:304] T c18c1d46a6ca47598d7b161ed70a4e7a P 787ddcc824c2490a808fc5a53452dc5e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 787ddcc824c2490a808fc5a53452dc5e; no voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:58:17.169934 23121 raft_consensus.cc:2743] T c18c1d46a6ca47598d7b161ed70a4e7a P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20250109 08:58:17.178822 22141 consensus_peers.cc:487] T 90a4a2381cbe40f0b204be325ee9fea1 P 88935ab835544e9889d05a6587d8cf21 -> Peer 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731): Couldn't send request to peer 787ddcc824c2490a808fc5a53452dc5e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20250109 08:58:17.179657 21460 auto_rebalancer.cc:663] Could not move replica: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20250109 08:58:17.179970 21460 auto_rebalancer.cc:264] scheduled replica move failed to complete: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20250109 08:58:17.205348 20363 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250109 08:58:17.205828 23031 raft_consensus.cc:491] T 126a163a89c448be89ccb9490e9f0168 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2b1a692435241ee988e0bd1661cd4c3)
I20250109 08:58:17.206291 23031 raft_consensus.cc:513] T 126a163a89c448be89ccb9490e9f0168 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:17.208070 23031 leader_election.cc:290] T 126a163a89c448be89ccb9490e9f0168 P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
I20250109 08:58:17.208323 20363 tablet_replica.cc:331] T 1d3d5d2a2c88417a97c9d57262a356dd P 787ddcc824c2490a808fc5a53452dc5e: stopping tablet replica
I20250109 08:58:17.209187 20363 raft_consensus.cc:2238] T 1d3d5d2a2c88417a97c9d57262a356dd P 787ddcc824c2490a808fc5a53452dc5e [term 3 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:17.209889 20363 raft_consensus.cc:2267] T 1d3d5d2a2c88417a97c9d57262a356dd P 787ddcc824c2490a808fc5a53452dc5e [term 3 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:17.212260 20363 tablet_replica.cc:331] T 7c19ed9fdde5463f84e43bea7b860d03 P 787ddcc824c2490a808fc5a53452dc5e: stopping tablet replica
I20250109 08:58:17.213064 20363 raft_consensus.cc:2238] T 7c19ed9fdde5463f84e43bea7b860d03 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Raft consensus shutting down.
W20250109 08:58:17.213114 22141 leader_election.cc:336] T 126a163a89c448be89ccb9490e9f0168 P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Network error: Client connection negotiation failed: client connection to 127.19.226.193:42725: connect: Connection refused (error 111)
W20250109 08:58:17.213871 22141 consensus_peers.cc:487] T 1b582ab8d8f044d9b9038a28e958feeb P 88935ab835544e9889d05a6587d8cf21 -> Peer 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731): Couldn't send request to peer 787ddcc824c2490a808fc5a53452dc5e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:17.214041 20363 raft_consensus.cc:2267] T 7c19ed9fdde5463f84e43bea7b860d03 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Raft consensus is shut down!
W20250109 08:58:17.215749 22142 leader_election.cc:336] T 126a163a89c448be89ccb9490e9f0168 P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Network error: Client connection negotiation failed: client connection to 127.19.226.194:45527: connect: Connection refused (error 111)
I20250109 08:58:17.216145 20363 tablet_replica.cc:331] T edc9db96e0ff4aa5839a6922f3eb5f24 P 787ddcc824c2490a808fc5a53452dc5e: stopping tablet replica
I20250109 08:58:17.216181 22142 leader_election.cc:304] T 126a163a89c448be89ccb9490e9f0168 P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88935ab835544e9889d05a6587d8cf21; no voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:58:17.216840 20363 raft_consensus.cc:2238] T edc9db96e0ff4aa5839a6922f3eb5f24 P 787ddcc824c2490a808fc5a53452dc5e [term 2 LEADER]: Raft consensus shutting down.
I20250109 08:58:17.217016 23128 raft_consensus.cc:2743] T 126a163a89c448be89ccb9490e9f0168 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250109 08:58:17.217823 20363 raft_consensus.cc:2267] T edc9db96e0ff4aa5839a6922f3eb5f24 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:17.219393 20363 tablet_replica.cc:331] T 5717a02ef53a4b22a96f431ca6c20d93 P 787ddcc824c2490a808fc5a53452dc5e: stopping tablet replica
I20250109 08:58:17.219882 20363 raft_consensus.cc:2238] T 5717a02ef53a4b22a96f431ca6c20d93 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:17.220234 20363 raft_consensus.cc:2267] T 5717a02ef53a4b22a96f431ca6c20d93 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:17.221836 20363 tablet_replica.cc:331] T 64c667f887ea41b4a484ee565b21fa16 P 787ddcc824c2490a808fc5a53452dc5e: stopping tablet replica
I20250109 08:58:17.222319 20363 raft_consensus.cc:2238] T 64c667f887ea41b4a484ee565b21fa16 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Raft consensus shutting down.
I20250109 08:58:17.223018 20363 raft_consensus.cc:2267] T 64c667f887ea41b4a484ee565b21fa16 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:17.224646 20363 tablet_replica.cc:331] T d015a10f95eb4f19b6d7c53cb201fa80 P 787ddcc824c2490a808fc5a53452dc5e: stopping tablet replica
I20250109 08:58:17.225152 20363 raft_consensus.cc:2238] T d015a10f95eb4f19b6d7c53cb201fa80 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:17.225507 20363 raft_consensus.cc:2267] T d015a10f95eb4f19b6d7c53cb201fa80 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:17.227147 20363 tablet_replica.cc:331] T d52f79ecf037457cbcc09d5fa43456cf P 787ddcc824c2490a808fc5a53452dc5e: stopping tablet replica
I20250109 08:58:17.227706 20363 raft_consensus.cc:2238] T d52f79ecf037457cbcc09d5fa43456cf P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:17.228060 20363 raft_consensus.cc:2267] T d52f79ecf037457cbcc09d5fa43456cf P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:17.229722 20363 tablet_replica.cc:331] T 8c0459d2eee64f0a83bd05ad5974f506 P 787ddcc824c2490a808fc5a53452dc5e: stopping tablet replica
I20250109 08:58:17.230216 20363 raft_consensus.cc:2238] T 8c0459d2eee64f0a83bd05ad5974f506 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:17.230674 20363 raft_consensus.cc:2267] T 8c0459d2eee64f0a83bd05ad5974f506 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:17.232262 20363 tablet_replica.cc:331] T 81d74e7d4472497aac5463a4b47e4fb9 P 787ddcc824c2490a808fc5a53452dc5e: stopping tablet replica
I20250109 08:58:17.232770 20363 raft_consensus.cc:2238] T 81d74e7d4472497aac5463a4b47e4fb9 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:17.233137 20363 raft_consensus.cc:2267] T 81d74e7d4472497aac5463a4b47e4fb9 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:17.234731 20363 tablet_replica.cc:331] T ab6c3fcdf226433c9e1303004f6c96b4 P 787ddcc824c2490a808fc5a53452dc5e: stopping tablet replica
I20250109 08:58:17.235212 20363 raft_consensus.cc:2238] T ab6c3fcdf226433c9e1303004f6c96b4 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Raft consensus shutting down.
I20250109 08:58:17.235849 20363 raft_consensus.cc:2267] T ab6c3fcdf226433c9e1303004f6c96b4 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:17.237453 20363 tablet_replica.cc:331] T 306c4892bbff4dbfbe5a0645294aa627 P 787ddcc824c2490a808fc5a53452dc5e: stopping tablet replica
I20250109 08:58:17.237937 20363 raft_consensus.cc:2238] T 306c4892bbff4dbfbe5a0645294aa627 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:17.238271 20363 raft_consensus.cc:2267] T 306c4892bbff4dbfbe5a0645294aa627 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:17.239801 20363 tablet_replica.cc:331] T 7781c00473024307987096b596da12ac P 787ddcc824c2490a808fc5a53452dc5e: stopping tablet replica
I20250109 08:58:17.240253 20363 raft_consensus.cc:2238] T 7781c00473024307987096b596da12ac P 787ddcc824c2490a808fc5a53452dc5e [term 2 LEADER]: Raft consensus shutting down.
I20250109 08:58:17.241078 20363 raft_consensus.cc:2267] T 7781c00473024307987096b596da12ac P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:17.242660 20363 tablet_replica.cc:331] T eb4516b9a16041c2adf34be520768f3f P 787ddcc824c2490a808fc5a53452dc5e: stopping tablet replica
I20250109 08:58:17.243182 20363 raft_consensus.cc:2238] T eb4516b9a16041c2adf34be520768f3f P 787ddcc824c2490a808fc5a53452dc5e [term 3 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:17.243661 20363 raft_consensus.cc:2267] T eb4516b9a16041c2adf34be520768f3f P 787ddcc824c2490a808fc5a53452dc5e [term 3 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:17.245671 20363 tablet_replica.cc:331] T d8493108cba04514bd2236192cef1079 P 787ddcc824c2490a808fc5a53452dc5e: stopping tablet replica
I20250109 08:58:17.246415 20363 raft_consensus.cc:2238] T d8493108cba04514bd2236192cef1079 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Raft consensus shutting down.
W20250109 08:58:17.246755 22141 consensus_peers.cc:487] T 206785f29c824cddbe9bbc00d365ffb3 P 88935ab835544e9889d05a6587d8cf21 -> Peer 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731): Couldn't send request to peer 787ddcc824c2490a808fc5a53452dc5e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:17.247303 20363 raft_consensus.cc:2267] T d8493108cba04514bd2236192cef1079 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:17.249383 20363 tablet_replica.cc:331] T a3aa271f2f0144eea9261e0e07ab833b P 787ddcc824c2490a808fc5a53452dc5e: stopping tablet replica
I20250109 08:58:17.250051 20363 raft_consensus.cc:2238] T a3aa271f2f0144eea9261e0e07ab833b P 787ddcc824c2490a808fc5a53452dc5e [term 2 LEADER]: Raft consensus shutting down.
I20250109 08:58:17.250803 20363 raft_consensus.cc:2267] T a3aa271f2f0144eea9261e0e07ab833b P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:17.252259 20363 tablet_replica.cc:331] T df9715570a6844d9a2686307db16eff1 P 787ddcc824c2490a808fc5a53452dc5e: stopping tablet replica
I20250109 08:58:17.252764 20363 raft_consensus.cc:2238] T df9715570a6844d9a2686307db16eff1 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:17.253115 20363 raft_consensus.cc:2267] T df9715570a6844d9a2686307db16eff1 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:17.254689 20363 tablet_replica.cc:331] T 37280454805d47f98a415a8f0c16d869 P 787ddcc824c2490a808fc5a53452dc5e: stopping tablet replica
I20250109 08:58:17.255168 20363 raft_consensus.cc:2238] T 37280454805d47f98a415a8f0c16d869 P 787ddcc824c2490a808fc5a53452dc5e [term 3 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:17.255556 20363 raft_consensus.cc:2267] T 37280454805d47f98a415a8f0c16d869 P 787ddcc824c2490a808fc5a53452dc5e [term 3 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:17.257128 20363 tablet_replica.cc:331] T 239ca74a4c0a455a9db2652d7debb989 P 787ddcc824c2490a808fc5a53452dc5e: stopping tablet replica
I20250109 08:58:17.257601 20363 raft_consensus.cc:2238] T 239ca74a4c0a455a9db2652d7debb989 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:17.257949 20363 raft_consensus.cc:2267] T 239ca74a4c0a455a9db2652d7debb989 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:17.259373 20363 tablet_replica.cc:331] T 363fa693a5114f2bb22f4ab67722a1c6 P 787ddcc824c2490a808fc5a53452dc5e: stopping tablet replica
I20250109 08:58:17.259814 20363 raft_consensus.cc:2238] T 363fa693a5114f2bb22f4ab67722a1c6 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:17.260206 20363 raft_consensus.cc:2267] T 363fa693a5114f2bb22f4ab67722a1c6 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:17.261941 20363 tablet_replica.cc:331] T af273bab41cc4a9da780147eebfd52ec P 787ddcc824c2490a808fc5a53452dc5e: stopping tablet replica
I20250109 08:58:17.262447 20363 raft_consensus.cc:2238] T af273bab41cc4a9da780147eebfd52ec P 787ddcc824c2490a808fc5a53452dc5e [term 2 LEADER]: Raft consensus shutting down.
I20250109 08:58:17.263200 20363 raft_consensus.cc:2267] T af273bab41cc4a9da780147eebfd52ec P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:17.264993 20363 tablet_replica.cc:331] T f5f85ad7c7c94acfaf9d550d55fce48c P 787ddcc824c2490a808fc5a53452dc5e: stopping tablet replica
I20250109 08:58:17.265420 20363 raft_consensus.cc:2238] T f5f85ad7c7c94acfaf9d550d55fce48c P 787ddcc824c2490a808fc5a53452dc5e [term 2 LEADER]: Raft consensus shutting down.
I20250109 08:58:17.266170 20363 raft_consensus.cc:2267] T f5f85ad7c7c94acfaf9d550d55fce48c P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:17.267575 20363 tablet_replica.cc:331] T 90a4a2381cbe40f0b204be325ee9fea1 P 787ddcc824c2490a808fc5a53452dc5e: stopping tablet replica
I20250109 08:58:17.268043 20363 raft_consensus.cc:2238] T 90a4a2381cbe40f0b204be325ee9fea1 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:17.268414 20363 raft_consensus.cc:2267] T 90a4a2381cbe40f0b204be325ee9fea1 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:17.269774 20363 tablet_replica.cc:331] T 0f3bde1a1ccd4c359a2e80feca968453 P 787ddcc824c2490a808fc5a53452dc5e: stopping tablet replica
I20250109 08:58:17.270232 20363 raft_consensus.cc:2238] T 0f3bde1a1ccd4c359a2e80feca968453 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Raft consensus shutting down.
I20250109 08:58:17.270843 20363 raft_consensus.cc:2267] T 0f3bde1a1ccd4c359a2e80feca968453 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:17.272361 20363 tablet_replica.cc:331] T e8ff570914d74df8b87296010e9b864f P 787ddcc824c2490a808fc5a53452dc5e: stopping tablet replica
I20250109 08:58:17.272949 20363 raft_consensus.cc:2238] T e8ff570914d74df8b87296010e9b864f P 787ddcc824c2490a808fc5a53452dc5e [term 2 LEADER]: Raft consensus shutting down.
I20250109 08:58:17.273702 20363 raft_consensus.cc:2267] T e8ff570914d74df8b87296010e9b864f P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:17.275346 20363 tablet_replica.cc:331] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P 787ddcc824c2490a808fc5a53452dc5e: stopping tablet replica
W20250109 08:58:17.275620 22141 consensus_peers.cc:487] T 306c4892bbff4dbfbe5a0645294aa627 P 88935ab835544e9889d05a6587d8cf21 -> Peer 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731): Couldn't send request to peer 787ddcc824c2490a808fc5a53452dc5e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:17.275974 20363 raft_consensus.cc:2238] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Raft consensus shutting down.
I20250109 08:58:17.276747 20363 raft_consensus.cc:2267] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:17.278288 20363 tablet_replica.cc:331] T 75edb5d04fa14201ad711c1d95c81f64 P 787ddcc824c2490a808fc5a53452dc5e: stopping tablet replica
I20250109 08:58:17.278791 20363 raft_consensus.cc:2238] T 75edb5d04fa14201ad711c1d95c81f64 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Raft consensus shutting down.
I20250109 08:58:17.279403 20363 raft_consensus.cc:2267] T 75edb5d04fa14201ad711c1d95c81f64 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:17.280881 20363 tablet_replica.cc:331] T c99aacce09d44f889be5480854a22760 P 787ddcc824c2490a808fc5a53452dc5e: stopping tablet replica
I20250109 08:58:17.281353 20363 raft_consensus.cc:2238] T c99aacce09d44f889be5480854a22760 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:17.281706 20363 raft_consensus.cc:2267] T c99aacce09d44f889be5480854a22760 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:17.283196 20363 tablet_replica.cc:331] T 0e06021cd6624e50af2c914399ebd6c4 P 787ddcc824c2490a808fc5a53452dc5e: stopping tablet replica
I20250109 08:58:17.283680 20363 raft_consensus.cc:2238] T 0e06021cd6624e50af2c914399ebd6c4 P 787ddcc824c2490a808fc5a53452dc5e [term 2 LEADER]: Raft consensus shutting down.
I20250109 08:58:17.284484 20363 raft_consensus.cc:2267] T 0e06021cd6624e50af2c914399ebd6c4 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:17.285916 20363 tablet_replica.cc:331] T 80123146836349938344efc61333695f P 787ddcc824c2490a808fc5a53452dc5e: stopping tablet replica
I20250109 08:58:17.286438 20363 raft_consensus.cc:2238] T 80123146836349938344efc61333695f P 787ddcc824c2490a808fc5a53452dc5e [term 2 LEADER]: Raft consensus shutting down.
I20250109 08:58:17.287051 20363 raft_consensus.cc:2267] T 80123146836349938344efc61333695f P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:17.288635 20363 tablet_replica.cc:331] T ddd14cdbefd64936960e9ffb5b38b5f2 P 787ddcc824c2490a808fc5a53452dc5e: stopping tablet replica
I20250109 08:58:17.289086 20363 raft_consensus.cc:2238] T ddd14cdbefd64936960e9ffb5b38b5f2 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:17.289438 20363 raft_consensus.cc:2267] T ddd14cdbefd64936960e9ffb5b38b5f2 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:17.290853 20363 tablet_replica.cc:331] T e03846e461974d0d8944c344e715b86a P 787ddcc824c2490a808fc5a53452dc5e: stopping tablet replica
I20250109 08:58:17.291447 20363 raft_consensus.cc:2238] T e03846e461974d0d8944c344e715b86a P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:17.291893 20363 raft_consensus.cc:2267] T e03846e461974d0d8944c344e715b86a P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:17.293771 20363 tablet_replica.cc:331] T 79c1ed8ab551478291f1f24cc940cb27 P 787ddcc824c2490a808fc5a53452dc5e: stopping tablet replica
I20250109 08:58:17.294409 20363 raft_consensus.cc:2238] T 79c1ed8ab551478291f1f24cc940cb27 P 787ddcc824c2490a808fc5a53452dc5e [term 2 LEADER]: Raft consensus shutting down.
I20250109 08:58:17.295207 20363 raft_consensus.cc:2267] T 79c1ed8ab551478291f1f24cc940cb27 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:17.297367 20363 tablet_replica.cc:331] T b00671c87aac405a81c6543f4defd722 P 787ddcc824c2490a808fc5a53452dc5e: stopping tablet replica
I20250109 08:58:17.298004 20363 raft_consensus.cc:2238] T b00671c87aac405a81c6543f4defd722 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:17.298437 20363 raft_consensus.cc:2267] T b00671c87aac405a81c6543f4defd722 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:17.300019 20363 tablet_replica.cc:331] T 89e1eeb24fea483eb63bd98af66e0b6b P 787ddcc824c2490a808fc5a53452dc5e: stopping tablet replica
I20250109 08:58:17.300650 20363 raft_consensus.cc:2238] T 89e1eeb24fea483eb63bd98af66e0b6b P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Raft consensus shutting down.
I20250109 08:58:17.301409 20363 raft_consensus.cc:2267] T 89e1eeb24fea483eb63bd98af66e0b6b P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:17.303077 20363 tablet_replica.cc:331] T 20ea30dba4314290ae45d9b4c13bebdd P 787ddcc824c2490a808fc5a53452dc5e: stopping tablet replica
I20250109 08:58:17.303560 20363 raft_consensus.cc:2238] T 20ea30dba4314290ae45d9b4c13bebdd P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:17.303893 20363 raft_consensus.cc:2267] T 20ea30dba4314290ae45d9b4c13bebdd P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:17.305580 20363 tablet_replica.cc:331] T d355d166de534fb4a50c9f48f2df7469 P 787ddcc824c2490a808fc5a53452dc5e: stopping tablet replica
I20250109 08:58:17.306201 20363 raft_consensus.cc:2238] T d355d166de534fb4a50c9f48f2df7469 P 787ddcc824c2490a808fc5a53452dc5e [term 3 LEADER]: Raft consensus shutting down.
I20250109 08:58:17.307014 20363 raft_consensus.cc:2267] T d355d166de534fb4a50c9f48f2df7469 P 787ddcc824c2490a808fc5a53452dc5e [term 3 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:17.308835 20363 tablet_replica.cc:331] T f961d3f357944118a69f905621731552 P 787ddcc824c2490a808fc5a53452dc5e: stopping tablet replica
I20250109 08:58:17.309276 20363 raft_consensus.cc:2238] T f961d3f357944118a69f905621731552 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:17.309635 20363 raft_consensus.cc:2267] T f961d3f357944118a69f905621731552 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:17.311151 20363 tablet_replica.cc:331] T af330dcd4767423ea4c179be7f88d0c4 P 787ddcc824c2490a808fc5a53452dc5e: stopping tablet replica
I20250109 08:58:17.311631 20363 raft_consensus.cc:2238] T af330dcd4767423ea4c179be7f88d0c4 P 787ddcc824c2490a808fc5a53452dc5e [term 2 LEADER]: Raft consensus shutting down.
I20250109 08:58:17.312458 20363 pending_rounds.cc:62] T af330dcd4767423ea4c179be7f88d0c4 P 787ddcc824c2490a808fc5a53452dc5e: Trying to abort 1 pending ops.
I20250109 08:58:17.312678 20363 pending_rounds.cc:69] T af330dcd4767423ea4c179be7f88d0c4 P 787ddcc824c2490a808fc5a53452dc5e: Aborting op as it isn't in flight: id { term: 2 index: 4 } timestamp: 7112348032345448448 op_type: CHANGE_CONFIG_OP change_config_record { tablet_id: "af330dcd4767423ea4c179be7f88d0c4" old_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } } new_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } } }
I20250109 08:58:17.313452 20363 raft_consensus.cc:2911] T af330dcd4767423ea4c179be7f88d0c4 P 787ddcc824c2490a808fc5a53452dc5e [term 2 LEADER]: Aborting config change with OpId 2.4: Aborted: Op aborted
I20250109 08:58:17.313719 20363 raft_consensus.cc:2883] T af330dcd4767423ea4c179be7f88d0c4 P 787ddcc824c2490a808fc5a53452dc5e [term 2 LEADER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted
I20250109 08:58:17.313937 20363 raft_consensus.cc:2267] T af330dcd4767423ea4c179be7f88d0c4 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:17.315380 20363 tablet_replica.cc:331] T 7a7211b8d4374ad1afd82e8cf85bad81 P 787ddcc824c2490a808fc5a53452dc5e: stopping tablet replica
I20250109 08:58:17.315829 20363 raft_consensus.cc:2238] T 7a7211b8d4374ad1afd82e8cf85bad81 P 787ddcc824c2490a808fc5a53452dc5e [term 2 LEADER]: Raft consensus shutting down.
I20250109 08:58:17.316535 20363 raft_consensus.cc:2267] T 7a7211b8d4374ad1afd82e8cf85bad81 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:17.318122 20363 tablet_replica.cc:331] T 1b582ab8d8f044d9b9038a28e958feeb P 787ddcc824c2490a808fc5a53452dc5e: stopping tablet replica
I20250109 08:58:17.318562 20363 raft_consensus.cc:2238] T 1b582ab8d8f044d9b9038a28e958feeb P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:17.318895 20363 raft_consensus.cc:2267] T 1b582ab8d8f044d9b9038a28e958feeb P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:17.320390 20363 tablet_replica.cc:331] T 206785f29c824cddbe9bbc00d365ffb3 P 787ddcc824c2490a808fc5a53452dc5e: stopping tablet replica
I20250109 08:58:17.320838 20363 raft_consensus.cc:2238] T 206785f29c824cddbe9bbc00d365ffb3 P 787ddcc824c2490a808fc5a53452dc5e [term 3 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:17.321171 20363 raft_consensus.cc:2267] T 206785f29c824cddbe9bbc00d365ffb3 P 787ddcc824c2490a808fc5a53452dc5e [term 3 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:17.322675 20363 tablet_replica.cc:331] T 5ac5ffe3cd65463fb448e819e3978841 P 787ddcc824c2490a808fc5a53452dc5e: stopping tablet replica
I20250109 08:58:17.323073 20363 raft_consensus.cc:2238] T 5ac5ffe3cd65463fb448e819e3978841 P 787ddcc824c2490a808fc5a53452dc5e [term 1 LEADER]: Raft consensus shutting down.
I20250109 08:58:17.323705 20363 raft_consensus.cc:2267] T 5ac5ffe3cd65463fb448e819e3978841 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:17.325196 20363 tablet_replica.cc:331] T c761cb691bd241ae825917a3fbc3ad24 P 787ddcc824c2490a808fc5a53452dc5e: stopping tablet replica
I20250109 08:58:17.325625 20363 raft_consensus.cc:2238] T c761cb691bd241ae825917a3fbc3ad24 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:17.325955 20363 raft_consensus.cc:2267] T c761cb691bd241ae825917a3fbc3ad24 P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:17.327415 20363 tablet_replica.cc:331] T c18c1d46a6ca47598d7b161ed70a4e7a P 787ddcc824c2490a808fc5a53452dc5e: stopping tablet replica
I20250109 08:58:17.327828 20363 raft_consensus.cc:2238] T c18c1d46a6ca47598d7b161ed70a4e7a P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:17.328220 20363 raft_consensus.cc:2267] T c18c1d46a6ca47598d7b161ed70a4e7a P 787ddcc824c2490a808fc5a53452dc5e [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:17.330173 20363 tablet_replica.cc:331] T e38ec0ef84af4b798073eafd4d210c35 P 787ddcc824c2490a808fc5a53452dc5e: stopping tablet replica
I20250109 08:58:17.330586 20363 raft_consensus.cc:2238] T e38ec0ef84af4b798073eafd4d210c35 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:17.330996 20363 raft_consensus.cc:2267] T e38ec0ef84af4b798073eafd4d210c35 P 787ddcc824c2490a808fc5a53452dc5e [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:17.332677 20363 tablet_replica.cc:331] T 610441ef4b1d40c284eff3a714e4214e P 787ddcc824c2490a808fc5a53452dc5e: stopping tablet replica
I20250109 08:58:17.333060 20363 raft_consensus.cc:2238] T 610441ef4b1d40c284eff3a714e4214e P 787ddcc824c2490a808fc5a53452dc5e [term 3 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:17.333403 20363 raft_consensus.cc:2267] T 610441ef4b1d40c284eff3a714e4214e P 787ddcc824c2490a808fc5a53452dc5e [term 3 FOLLOWER]: Raft consensus is shut down!
W20250109 08:58:17.369410 22141 consensus_peers.cc:487] T 20ea30dba4314290ae45d9b4c13bebdd P 88935ab835544e9889d05a6587d8cf21 -> Peer 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731): Couldn't send request to peer 787ddcc824c2490a808fc5a53452dc5e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20250109 08:58:17.383610 22141 consensus_peers.cc:487] T 1d3d5d2a2c88417a97c9d57262a356dd P 88935ab835544e9889d05a6587d8cf21 -> Peer 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731): Couldn't send request to peer 787ddcc824c2490a808fc5a53452dc5e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:17.390199 23128 raft_consensus.cc:491] T 90978f185a374feba3bc19b54f9d97f5 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2b1a692435241ee988e0bd1661cd4c3)
I20250109 08:58:17.390774 23128 raft_consensus.cc:513] T 90978f185a374feba3bc19b54f9d97f5 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:17.392760 23128 leader_election.cc:290] T 90978f185a374feba3bc19b54f9d97f5 P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
W20250109 08:58:17.396730 22142 leader_election.cc:336] T 90978f185a374feba3bc19b54f9d97f5 P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Network error: Client connection negotiation failed: client connection to 127.19.226.194:45527: connect: Connection refused (error 111)
W20250109 08:58:17.402974 22141 consensus_peers.cc:487] T d52f79ecf037457cbcc09d5fa43456cf P 88935ab835544e9889d05a6587d8cf21 -> Peer 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731): Couldn't send request to peer 787ddcc824c2490a808fc5a53452dc5e. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.195:35731: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20250109 08:58:17.403921 22141 leader_election.cc:336] T 90978f185a374feba3bc19b54f9d97f5 P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Network error: Client connection negotiation failed: client connection to 127.19.226.193:42725: connect: Connection refused (error 111)
I20250109 08:58:17.404313 22141 leader_election.cc:304] T 90978f185a374feba3bc19b54f9d97f5 P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88935ab835544e9889d05a6587d8cf21; no voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:58:17.404966 23128 raft_consensus.cc:2743] T 90978f185a374feba3bc19b54f9d97f5 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20250109 08:58:17.406191 22141 consensus_peers.cc:487] T b00671c87aac405a81c6543f4defd722 P 88935ab835544e9889d05a6587d8cf21 -> Peer 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731): Couldn't send request to peer 787ddcc824c2490a808fc5a53452dc5e. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.195:35731: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20250109 08:58:17.406649 22141 consensus_peers.cc:487] T df9715570a6844d9a2686307db16eff1 P 88935ab835544e9889d05a6587d8cf21 -> Peer 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731): Couldn't send request to peer 787ddcc824c2490a808fc5a53452dc5e. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.195:35731: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:17.419636 20363 tablet_server.cc:195] TabletServer@127.19.226.195:0 shutdown complete.
W20250109 08:58:17.435346 22141 consensus_peers.cc:487] T 37280454805d47f98a415a8f0c16d869 P 88935ab835544e9889d05a6587d8cf21 -> Peer 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731): Couldn't send request to peer 787ddcc824c2490a808fc5a53452dc5e. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.195:35731: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20250109 08:58:17.457207 22141 consensus_peers.cc:487] T d015a10f95eb4f19b6d7c53cb201fa80 P 88935ab835544e9889d05a6587d8cf21 -> Peer 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731): Couldn't send request to peer 787ddcc824c2490a808fc5a53452dc5e. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.195:35731: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:17.464268 23128 raft_consensus.cc:491] T d6f988a231a74f6fa112da9a527d4a0b P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a2b1a692435241ee988e0bd1661cd4c3)
I20250109 08:58:17.464883 23128 raft_consensus.cc:513] T d6f988a231a74f6fa112da9a527d4a0b P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } }
I20250109 08:58:17.466923 23128 leader_election.cc:290] T d6f988a231a74f6fa112da9a527d4a0b P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725), a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527)
W20250109 08:58:17.467417 22141 consensus_peers.cc:487] T 239ca74a4c0a455a9db2652d7debb989 P 88935ab835544e9889d05a6587d8cf21 -> Peer 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731): Couldn't send request to peer 787ddcc824c2490a808fc5a53452dc5e. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.195:35731: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20250109 08:58:17.470928 22141 leader_election.cc:336] T d6f988a231a74f6fa112da9a527d4a0b P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Network error: Client connection negotiation failed: client connection to 127.19.226.193:42725: connect: Connection refused (error 111)
W20250109 08:58:17.471604 22142 leader_election.cc:336] T d6f988a231a74f6fa112da9a527d4a0b P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer a2b1a692435241ee988e0bd1661cd4c3 (127.19.226.194:45527): Network error: Client connection negotiation failed: client connection to 127.19.226.194:45527: connect: Connection refused (error 111)
I20250109 08:58:17.471953 22142 leader_election.cc:304] T d6f988a231a74f6fa112da9a527d4a0b P 88935ab835544e9889d05a6587d8cf21 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88935ab835544e9889d05a6587d8cf21; no voters: 8bf6e26e98b14566899578a990b27d47, a2b1a692435241ee988e0bd1661cd4c3
I20250109 08:58:17.472520 23128 raft_consensus.cc:2743] T d6f988a231a74f6fa112da9a527d4a0b P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20250109 08:58:17.478279 22141 consensus_peers.cc:487] T 20ea30dba4314290ae45d9b4c13bebdd P 88935ab835544e9889d05a6587d8cf21 -> Peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Couldn't send request to peer 8bf6e26e98b14566899578a990b27d47. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.193:42725: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20250109 08:58:17.501641 22141 consensus_peers.cc:487] T 206785f29c824cddbe9bbc00d365ffb3 P 88935ab835544e9889d05a6587d8cf21 -> Peer 8bf6e26e98b14566899578a990b27d47 (127.19.226.193:42725): Couldn't send request to peer 8bf6e26e98b14566899578a990b27d47. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.193:42725: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20250109 08:58:17.503587 20363 tablet_server.cc:178] TabletServer@127.19.226.196:0 shutting down...
I20250109 08:58:17.531433 20363 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250109 08:58:17.532222 20363 tablet_replica.cc:331] T 3b5049c00b0b4dd89c49fa5d13db169e P 88935ab835544e9889d05a6587d8cf21: stopping tablet replica
I20250109 08:58:17.532843 20363 raft_consensus.cc:2238] T 3b5049c00b0b4dd89c49fa5d13db169e P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:17.533282 20363 raft_consensus.cc:2267] T 3b5049c00b0b4dd89c49fa5d13db169e P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:17.534912 20363 tablet_replica.cc:331] T 0e06021cd6624e50af2c914399ebd6c4 P 88935ab835544e9889d05a6587d8cf21: stopping tablet replica
I20250109 08:58:17.535442 20363 raft_consensus.cc:2238] T 0e06021cd6624e50af2c914399ebd6c4 P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:17.535899 20363 raft_consensus.cc:2267] T 0e06021cd6624e50af2c914399ebd6c4 P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:17.537362 20363 tablet_replica.cc:331] T 7781c00473024307987096b596da12ac P 88935ab835544e9889d05a6587d8cf21: stopping tablet replica
I20250109 08:58:17.537851 20363 raft_consensus.cc:2238] T 7781c00473024307987096b596da12ac P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:17.538234 20363 raft_consensus.cc:2267] T 7781c00473024307987096b596da12ac P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:17.539717 20363 tablet_replica.cc:331] T 64c667f887ea41b4a484ee565b21fa16 P 88935ab835544e9889d05a6587d8cf21: stopping tablet replica
I20250109 08:58:17.540190 20363 raft_consensus.cc:2238] T 64c667f887ea41b4a484ee565b21fa16 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:17.540627 20363 raft_consensus.cc:2267] T 64c667f887ea41b4a484ee565b21fa16 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:17.542188 20363 tablet_replica.cc:331] T 5ac5ffe3cd65463fb448e819e3978841 P 88935ab835544e9889d05a6587d8cf21: stopping tablet replica
I20250109 08:58:17.542692 20363 raft_consensus.cc:2238] T 5ac5ffe3cd65463fb448e819e3978841 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:17.543221 20363 raft_consensus.cc:2267] T 5ac5ffe3cd65463fb448e819e3978841 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:17.545084 20363 tablet_replica.cc:331] T 1b582ab8d8f044d9b9038a28e958feeb P 88935ab835544e9889d05a6587d8cf21: stopping tablet replica
I20250109 08:58:17.545722 20363 raft_consensus.cc:2238] T 1b582ab8d8f044d9b9038a28e958feeb P 88935ab835544e9889d05a6587d8cf21 [term 2 LEADER]: Raft consensus shutting down.
I20250109 08:58:17.546448 20363 raft_consensus.cc:2267] T 1b582ab8d8f044d9b9038a28e958feeb P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Raft consensus is shut down!
W20250109 08:58:17.548185 22141 consensus_peers.cc:487] T eb4516b9a16041c2adf34be520768f3f P 88935ab835544e9889d05a6587d8cf21 -> Peer 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731): Couldn't send request to peer 787ddcc824c2490a808fc5a53452dc5e. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.195:35731: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:17.548218 20363 tablet_replica.cc:331] T 206785f29c824cddbe9bbc00d365ffb3 P 88935ab835544e9889d05a6587d8cf21: stopping tablet replica
W20250109 08:58:17.548703 22141 consensus_peers.cc:487] T 5717a02ef53a4b22a96f431ca6c20d93 P 88935ab835544e9889d05a6587d8cf21 -> Peer 787ddcc824c2490a808fc5a53452dc5e (127.19.226.195:35731): Couldn't send request to peer 787ddcc824c2490a808fc5a53452dc5e. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.195:35731: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:17.549127 20363 raft_consensus.cc:2238] T 206785f29c824cddbe9bbc00d365ffb3 P 88935ab835544e9889d05a6587d8cf21 [term 3 LEADER]: Raft consensus shutting down.
I20250109 08:58:17.549849 20363 raft_consensus.cc:2267] T 206785f29c824cddbe9bbc00d365ffb3 P 88935ab835544e9889d05a6587d8cf21 [term 3 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:17.551409 20363 tablet_replica.cc:331] T 180b4d539a144a38905bf340ecb24ef4 P 88935ab835544e9889d05a6587d8cf21: stopping tablet replica
I20250109 08:58:17.551887 20363 raft_consensus.cc:2238] T 180b4d539a144a38905bf340ecb24ef4 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:17.552311 20363 raft_consensus.cc:2267] T 180b4d539a144a38905bf340ecb24ef4 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:17.553841 20363 tablet_replica.cc:331] T d8493108cba04514bd2236192cef1079 P 88935ab835544e9889d05a6587d8cf21: stopping tablet replica
I20250109 08:58:17.554313 20363 raft_consensus.cc:2238] T d8493108cba04514bd2236192cef1079 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:17.554917 20363 raft_consensus.cc:2267] T d8493108cba04514bd2236192cef1079 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:17.556561 20363 tablet_replica.cc:331] T 81d74e7d4472497aac5463a4b47e4fb9 P 88935ab835544e9889d05a6587d8cf21: stopping tablet replica
I20250109 08:58:17.557056 20363 raft_consensus.cc:2238] T 81d74e7d4472497aac5463a4b47e4fb9 P 88935ab835544e9889d05a6587d8cf21 [term 2 LEADER]: Raft consensus shutting down.
I20250109 08:58:17.557816 20363 raft_consensus.cc:2267] T 81d74e7d4472497aac5463a4b47e4fb9 P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:17.559327 20363 tablet_replica.cc:331] T 56bbc306275e4a409e028a7ca1dd1602 P 88935ab835544e9889d05a6587d8cf21: stopping tablet replica
I20250109 08:58:17.559832 20363 raft_consensus.cc:2238] T 56bbc306275e4a409e028a7ca1dd1602 P 88935ab835544e9889d05a6587d8cf21 [term 3 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:17.560259 20363 raft_consensus.cc:2267] T 56bbc306275e4a409e028a7ca1dd1602 P 88935ab835544e9889d05a6587d8cf21 [term 3 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:17.561764 20363 tablet_replica.cc:331] T eb4516b9a16041c2adf34be520768f3f P 88935ab835544e9889d05a6587d8cf21: stopping tablet replica
I20250109 08:58:17.562220 20363 raft_consensus.cc:2238] T eb4516b9a16041c2adf34be520768f3f P 88935ab835544e9889d05a6587d8cf21 [term 3 LEADER]: Raft consensus shutting down.
I20250109 08:58:17.562889 20363 raft_consensus.cc:2267] T eb4516b9a16041c2adf34be520768f3f P 88935ab835544e9889d05a6587d8cf21 [term 3 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:17.564445 20363 tablet_replica.cc:331] T d6f988a231a74f6fa112da9a527d4a0b P 88935ab835544e9889d05a6587d8cf21: stopping tablet replica
I20250109 08:58:17.564932 20363 raft_consensus.cc:2238] T d6f988a231a74f6fa112da9a527d4a0b P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:17.565352 20363 raft_consensus.cc:2267] T d6f988a231a74f6fa112da9a527d4a0b P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:17.566875 20363 tablet_replica.cc:331] T 126a163a89c448be89ccb9490e9f0168 P 88935ab835544e9889d05a6587d8cf21: stopping tablet replica
I20250109 08:58:17.567351 20363 raft_consensus.cc:2238] T 126a163a89c448be89ccb9490e9f0168 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:17.567780 20363 raft_consensus.cc:2267] T 126a163a89c448be89ccb9490e9f0168 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:17.569432 20363 tablet_replica.cc:331] T 239ca74a4c0a455a9db2652d7debb989 P 88935ab835544e9889d05a6587d8cf21: stopping tablet replica
I20250109 08:58:17.569927 20363 raft_consensus.cc:2238] T 239ca74a4c0a455a9db2652d7debb989 P 88935ab835544e9889d05a6587d8cf21 [term 2 LEADER]: Raft consensus shutting down.
I20250109 08:58:17.570554 20363 raft_consensus.cc:2267] T 239ca74a4c0a455a9db2652d7debb989 P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:17.572117 20363 tablet_replica.cc:331] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P 88935ab835544e9889d05a6587d8cf21: stopping tablet replica
I20250109 08:58:17.572641 20363 raft_consensus.cc:2238] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:17.573036 20363 raft_consensus.cc:2267] T b54d6381a2cc41dfb59b1f6b99c0c5d4 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:17.574587 20363 tablet_replica.cc:331] T 5717a02ef53a4b22a96f431ca6c20d93 P 88935ab835544e9889d05a6587d8cf21: stopping tablet replica
I20250109 08:58:17.575066 20363 raft_consensus.cc:2238] T 5717a02ef53a4b22a96f431ca6c20d93 P 88935ab835544e9889d05a6587d8cf21 [term 2 LEADER]: Raft consensus shutting down.
I20250109 08:58:17.575731 20363 raft_consensus.cc:2267] T 5717a02ef53a4b22a96f431ca6c20d93 P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:17.577292 20363 tablet_replica.cc:331] T 0cc4fcda876242c396b804edebebb01f P 88935ab835544e9889d05a6587d8cf21: stopping tablet replica
I20250109 08:58:17.577776 20363 raft_consensus.cc:2238] T 0cc4fcda876242c396b804edebebb01f P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:17.578202 20363 raft_consensus.cc:2267] T 0cc4fcda876242c396b804edebebb01f P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:17.579623 20363 tablet_replica.cc:331] T 8c0459d2eee64f0a83bd05ad5974f506 P 88935ab835544e9889d05a6587d8cf21: stopping tablet replica
I20250109 08:58:17.580143 20363 raft_consensus.cc:2238] T 8c0459d2eee64f0a83bd05ad5974f506 P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:17.580561 20363 raft_consensus.cc:2267] T 8c0459d2eee64f0a83bd05ad5974f506 P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:17.581982 20363 tablet_replica.cc:331] T d015a10f95eb4f19b6d7c53cb201fa80 P 88935ab835544e9889d05a6587d8cf21: stopping tablet replica
I20250109 08:58:17.582470 20363 raft_consensus.cc:2238] T d015a10f95eb4f19b6d7c53cb201fa80 P 88935ab835544e9889d05a6587d8cf21 [term 2 LEADER]: Raft consensus shutting down.
I20250109 08:58:17.583094 20363 raft_consensus.cc:2267] T d015a10f95eb4f19b6d7c53cb201fa80 P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:17.584558 20363 tablet_replica.cc:331] T d355d166de534fb4a50c9f48f2df7469 P 88935ab835544e9889d05a6587d8cf21: stopping tablet replica
I20250109 08:58:17.585038 20363 raft_consensus.cc:2238] T d355d166de534fb4a50c9f48f2df7469 P 88935ab835544e9889d05a6587d8cf21 [term 3 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:17.585419 20363 raft_consensus.cc:2267] T d355d166de534fb4a50c9f48f2df7469 P 88935ab835544e9889d05a6587d8cf21 [term 3 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:17.586855 20363 tablet_replica.cc:331] T 71ac0243d72845659291184835652dde P 88935ab835544e9889d05a6587d8cf21: stopping tablet replica
I20250109 08:58:17.587380 20363 raft_consensus.cc:2238] T 71ac0243d72845659291184835652dde P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:17.587792 20363 raft_consensus.cc:2267] T 71ac0243d72845659291184835652dde P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:17.589308 20363 tablet_replica.cc:331] T b00671c87aac405a81c6543f4defd722 P 88935ab835544e9889d05a6587d8cf21: stopping tablet replica
I20250109 08:58:17.589780 20363 raft_consensus.cc:2238] T b00671c87aac405a81c6543f4defd722 P 88935ab835544e9889d05a6587d8cf21 [term 2 LEADER]: Raft consensus shutting down.
I20250109 08:58:17.590409 20363 raft_consensus.cc:2267] T b00671c87aac405a81c6543f4defd722 P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:17.591847 20363 tablet_replica.cc:331] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 88935ab835544e9889d05a6587d8cf21: stopping tablet replica
I20250109 08:58:17.592299 20363 raft_consensus.cc:2238] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:17.592695 20363 raft_consensus.cc:2267] T 5d31dbfb2b4a4a6ca4500f92df9dbb98 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:17.594167 20363 tablet_replica.cc:331] T 363fa693a5114f2bb22f4ab67722a1c6 P 88935ab835544e9889d05a6587d8cf21: stopping tablet replica
I20250109 08:58:17.594652 20363 raft_consensus.cc:2238] T 363fa693a5114f2bb22f4ab67722a1c6 P 88935ab835544e9889d05a6587d8cf21 [term 2 LEARNER]: Raft consensus shutting down.
I20250109 08:58:17.594976 20363 raft_consensus.cc:2267] T 363fa693a5114f2bb22f4ab67722a1c6 P 88935ab835544e9889d05a6587d8cf21 [term 2 LEARNER]: Raft consensus is shut down!
I20250109 08:58:17.596490 20363 tablet_replica.cc:331] T e8ff570914d74df8b87296010e9b864f P 88935ab835544e9889d05a6587d8cf21: stopping tablet replica
I20250109 08:58:17.596964 20363 raft_consensus.cc:2238] T e8ff570914d74df8b87296010e9b864f P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:17.597290 20363 raft_consensus.cc:2267] T e8ff570914d74df8b87296010e9b864f P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:17.599018 20363 tablet_replica.cc:331] T 0f3bde1a1ccd4c359a2e80feca968453 P 88935ab835544e9889d05a6587d8cf21: stopping tablet replica
I20250109 08:58:17.599714 20363 raft_consensus.cc:2238] T 0f3bde1a1ccd4c359a2e80feca968453 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:17.600265 20363 raft_consensus.cc:2267] T 0f3bde1a1ccd4c359a2e80feca968453 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:17.602278 20363 tablet_replica.cc:331] T 90978f185a374feba3bc19b54f9d97f5 P 88935ab835544e9889d05a6587d8cf21: stopping tablet replica
I20250109 08:58:17.602725 20363 raft_consensus.cc:2238] T 90978f185a374feba3bc19b54f9d97f5 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:17.603070 20363 raft_consensus.cc:2267] T 90978f185a374feba3bc19b54f9d97f5 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:17.604560 20363 tablet_replica.cc:331] T 37280454805d47f98a415a8f0c16d869 P 88935ab835544e9889d05a6587d8cf21: stopping tablet replica
I20250109 08:58:17.604976 20363 raft_consensus.cc:2238] T 37280454805d47f98a415a8f0c16d869 P 88935ab835544e9889d05a6587d8cf21 [term 3 LEADER]: Raft consensus shutting down.
I20250109 08:58:17.605599 20363 raft_consensus.cc:2267] T 37280454805d47f98a415a8f0c16d869 P 88935ab835544e9889d05a6587d8cf21 [term 3 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:17.607117 20363 tablet_replica.cc:331] T af330dcd4767423ea4c179be7f88d0c4 P 88935ab835544e9889d05a6587d8cf21: stopping tablet replica
I20250109 08:58:17.607563 20363 raft_consensus.cc:2238] T af330dcd4767423ea4c179be7f88d0c4 P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:17.607834 20363 pending_rounds.cc:62] T af330dcd4767423ea4c179be7f88d0c4 P 88935ab835544e9889d05a6587d8cf21: Trying to abort 1 pending ops.
I20250109 08:58:17.607964 20363 pending_rounds.cc:69] T af330dcd4767423ea4c179be7f88d0c4 P 88935ab835544e9889d05a6587d8cf21: Aborting op as it isn't in flight: id { term: 2 index: 4 } timestamp: 7112348032345448448 op_type: CHANGE_CONFIG_OP change_config_record { tablet_id: "af330dcd4767423ea4c179be7f88d0c4" old_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: NON_VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: true } } } new_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8bf6e26e98b14566899578a990b27d47" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42725 } attrs { replace: true } } peers { permanent_uuid: "a2b1a692435241ee988e0bd1661cd4c3" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 45527 } } peers { permanent_uuid: "787ddcc824c2490a808fc5a53452dc5e" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35731 } } peers { permanent_uuid: "88935ab835544e9889d05a6587d8cf21" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 44363 } attrs { promote: false } } } }
I20250109 08:58:17.608673 20363 raft_consensus.cc:2911] T af330dcd4767423ea4c179be7f88d0c4 P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Aborting config change with OpId 2.4: Aborted: Op aborted
I20250109 08:58:17.608917 20363 raft_consensus.cc:2883] T af330dcd4767423ea4c179be7f88d0c4 P 88935ab835544e9889d05a6587d8cf21 [term 2 LEARNER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted
I20250109 08:58:17.609109 20363 raft_consensus.cc:2267] T af330dcd4767423ea4c179be7f88d0c4 P 88935ab835544e9889d05a6587d8cf21 [term 2 LEARNER]: Raft consensus is shut down!
I20250109 08:58:17.610555 20363 tablet_replica.cc:331] T f5f85ad7c7c94acfaf9d550d55fce48c P 88935ab835544e9889d05a6587d8cf21: stopping tablet replica
I20250109 08:58:17.610993 20363 raft_consensus.cc:2238] T f5f85ad7c7c94acfaf9d550d55fce48c P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:17.611313 20363 raft_consensus.cc:2267] T f5f85ad7c7c94acfaf9d550d55fce48c P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:17.612799 20363 tablet_replica.cc:331] T 306c4892bbff4dbfbe5a0645294aa627 P 88935ab835544e9889d05a6587d8cf21: stopping tablet replica
I20250109 08:58:17.613226 20363 raft_consensus.cc:2238] T 306c4892bbff4dbfbe5a0645294aa627 P 88935ab835544e9889d05a6587d8cf21 [term 2 LEADER]: Raft consensus shutting down.
I20250109 08:58:17.613859 20363 raft_consensus.cc:2267] T 306c4892bbff4dbfbe5a0645294aa627 P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:17.615334 20363 tablet_replica.cc:331] T df9715570a6844d9a2686307db16eff1 P 88935ab835544e9889d05a6587d8cf21: stopping tablet replica
I20250109 08:58:17.615777 20363 raft_consensus.cc:2238] T df9715570a6844d9a2686307db16eff1 P 88935ab835544e9889d05a6587d8cf21 [term 2 LEADER]: Raft consensus shutting down.
I20250109 08:58:17.616415 20363 raft_consensus.cc:2267] T df9715570a6844d9a2686307db16eff1 P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:17.617964 20363 tablet_replica.cc:331] T db34a00c02b84ee988edc7d6e344c1f5 P 88935ab835544e9889d05a6587d8cf21: stopping tablet replica
I20250109 08:58:17.618417 20363 raft_consensus.cc:2238] T db34a00c02b84ee988edc7d6e344c1f5 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:17.618814 20363 raft_consensus.cc:2267] T db34a00c02b84ee988edc7d6e344c1f5 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:17.620321 20363 tablet_replica.cc:331] T 2935d69446cd458e92046d2b2f1b74ab P 88935ab835544e9889d05a6587d8cf21: stopping tablet replica
I20250109 08:58:17.620838 20363 raft_consensus.cc:2238] T 2935d69446cd458e92046d2b2f1b74ab P 88935ab835544e9889d05a6587d8cf21 [term 2 LEADER]: Raft consensus shutting down.
I20250109 08:58:17.621486 20363 raft_consensus.cc:2267] T 2935d69446cd458e92046d2b2f1b74ab P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:17.623046 20363 tablet_replica.cc:331] T d52f79ecf037457cbcc09d5fa43456cf P 88935ab835544e9889d05a6587d8cf21: stopping tablet replica
I20250109 08:58:17.623659 20363 raft_consensus.cc:2238] T d52f79ecf037457cbcc09d5fa43456cf P 88935ab835544e9889d05a6587d8cf21 [term 2 LEADER]: Raft consensus shutting down.
I20250109 08:58:17.624286 20363 raft_consensus.cc:2267] T d52f79ecf037457cbcc09d5fa43456cf P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:17.625887 20363 tablet_replica.cc:331] T 20ea30dba4314290ae45d9b4c13bebdd P 88935ab835544e9889d05a6587d8cf21: stopping tablet replica
I20250109 08:58:17.626369 20363 raft_consensus.cc:2238] T 20ea30dba4314290ae45d9b4c13bebdd P 88935ab835544e9889d05a6587d8cf21 [term 2 LEADER]: Raft consensus shutting down.
I20250109 08:58:17.627007 20363 raft_consensus.cc:2267] T 20ea30dba4314290ae45d9b4c13bebdd P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:17.628571 20363 tablet_replica.cc:331] T 1d3d5d2a2c88417a97c9d57262a356dd P 88935ab835544e9889d05a6587d8cf21: stopping tablet replica
I20250109 08:58:17.629020 20363 raft_consensus.cc:2238] T 1d3d5d2a2c88417a97c9d57262a356dd P 88935ab835544e9889d05a6587d8cf21 [term 3 LEADER]: Raft consensus shutting down.
I20250109 08:58:17.629768 20363 raft_consensus.cc:2267] T 1d3d5d2a2c88417a97c9d57262a356dd P 88935ab835544e9889d05a6587d8cf21 [term 3 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:17.631266 20363 tablet_replica.cc:331] T e601a034e10e418bac8683443d86a1c1 P 88935ab835544e9889d05a6587d8cf21: stopping tablet replica
I20250109 08:58:17.631737 20363 raft_consensus.cc:2238] T e601a034e10e418bac8683443d86a1c1 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:17.632157 20363 raft_consensus.cc:2267] T e601a034e10e418bac8683443d86a1c1 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:17.633628 20363 tablet_replica.cc:331] T 97304b06c7fa40ef8345a72c796c47f8 P 88935ab835544e9889d05a6587d8cf21: stopping tablet replica
I20250109 08:58:17.634057 20363 raft_consensus.cc:2238] T 97304b06c7fa40ef8345a72c796c47f8 P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:17.634479 20363 raft_consensus.cc:2267] T 97304b06c7fa40ef8345a72c796c47f8 P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:17.636051 20363 tablet_replica.cc:331] T 7c19ed9fdde5463f84e43bea7b860d03 P 88935ab835544e9889d05a6587d8cf21: stopping tablet replica
I20250109 08:58:17.636487 20363 raft_consensus.cc:2238] T 7c19ed9fdde5463f84e43bea7b860d03 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:17.636886 20363 raft_consensus.cc:2267] T 7c19ed9fdde5463f84e43bea7b860d03 P 88935ab835544e9889d05a6587d8cf21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:17.638342 20363 tablet_replica.cc:331] T 90a4a2381cbe40f0b204be325ee9fea1 P 88935ab835544e9889d05a6587d8cf21: stopping tablet replica
I20250109 08:58:17.638744 20363 raft_consensus.cc:2238] T 90a4a2381cbe40f0b204be325ee9fea1 P 88935ab835544e9889d05a6587d8cf21 [term 2 LEADER]: Raft consensus shutting down.
I20250109 08:58:17.639326 20363 raft_consensus.cc:2267] T 90a4a2381cbe40f0b204be325ee9fea1 P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:17.641086 20363 tablet_replica.cc:331] T edc9db96e0ff4aa5839a6922f3eb5f24 P 88935ab835544e9889d05a6587d8cf21: stopping tablet replica
I20250109 08:58:17.641463 20363 raft_consensus.cc:2238] T edc9db96e0ff4aa5839a6922f3eb5f24 P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:17.641825 20363 raft_consensus.cc:2267] T edc9db96e0ff4aa5839a6922f3eb5f24 P 88935ab835544e9889d05a6587d8cf21 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:17.711130 20363 tablet_server.cc:195] TabletServer@127.19.226.196:0 shutdown complete.
I20250109 08:58:17.761425 20363 master.cc:537] Master@127.19.226.254:38499 shutting down...
I20250109 08:58:17.785259 20363 raft_consensus.cc:2238] T 00000000000000000000000000000000 P 1f1a09d9b3e04cfcb8db842978c0cf7b [term 1 LEADER]: Raft consensus shutting down.
I20250109 08:58:17.785737 20363 raft_consensus.cc:2267] T 00000000000000000000000000000000 P 1f1a09d9b3e04cfcb8db842978c0cf7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:17.785998 20363 tablet_replica.cc:331] T 00000000000000000000000000000000 P 1f1a09d9b3e04cfcb8db842978c0cf7b: stopping tablet replica
I20250109 08:58:17.824266 20363 master.cc:559] Master@127.19.226.254:38499 shutdown complete.
[ OK ] LeaderRebalancerTest.AddTserver (51149 ms)
[ RUN ] LeaderRebalancerTest.RestartTserver
I20250109 08:58:17.907577 20363 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.19.226.254:46425
I20250109 08:58:17.908917 20363 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250109 08:58:17.914259 23178 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250109 08:58:17.914489 23179 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250109 08:58:17.915486 23181 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250109 08:58:17.916152 20363 server_base.cc:1034] running on GCE node
I20250109 08:58:17.917459 20363 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250109 08:58:17.917677 20363 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250109 08:58:17.917814 20363 hybrid_clock.cc:648] HybridClock initialized: now 1736413097917798 us; error 0 us; skew 500 ppm
I20250109 08:58:17.918287 20363 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250109 08:58:17.920794 20363 webserver.cc:458] Webserver started at http://127.19.226.254:33729/ using document root <none> and password file <none>
I20250109 08:58:17.921252 20363 fs_manager.cc:362] Metadata directory not provided
I20250109 08:58:17.921417 20363 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250109 08:58:17.921659 20363 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250109 08:58:17.922696 20363 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1736412987538469-20363-0/minicluster-data/master-0-root/instance:
uuid: "7dbdd4b28a9d481eb01d950816966c31"
format_stamp: "Formatted at 2025-01-09 08:58:17 on dist-test-slave-g8ds"
I20250109 08:58:17.926929 20363 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20250109 08:58:17.930336 23186 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250109 08:58:17.931092 20363 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20250109 08:58:17.931340 20363 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1736412987538469-20363-0/minicluster-data/master-0-root
uuid: "7dbdd4b28a9d481eb01d950816966c31"
format_stamp: "Formatted at 2025-01-09 08:58:17 on dist-test-slave-g8ds"
I20250109 08:58:17.931610 20363 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1736412987538469-20363-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1736412987538469-20363-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1736412987538469-20363-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250109 08:58:17.956744 20363 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250109 08:58:17.957880 20363 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250109 08:58:17.993273 20363 rpc_server.cc:307] RPC server started. Bound to: 127.19.226.254:46425
I20250109 08:58:17.993373 23237 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.226.254:46425 every 8 connection(s)
I20250109 08:58:17.996933 23238 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:18.006247 23238 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7dbdd4b28a9d481eb01d950816966c31: Bootstrap starting.
I20250109 08:58:18.010655 23238 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7dbdd4b28a9d481eb01d950816966c31: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:18.014539 23238 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7dbdd4b28a9d481eb01d950816966c31: No bootstrap required, opened a new log
I20250109 08:58:18.016469 23238 raft_consensus.cc:357] T 00000000000000000000000000000000 P 7dbdd4b28a9d481eb01d950816966c31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7dbdd4b28a9d481eb01d950816966c31" member_type: VOTER }
I20250109 08:58:18.016836 23238 raft_consensus.cc:383] T 00000000000000000000000000000000 P 7dbdd4b28a9d481eb01d950816966c31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:18.017009 23238 raft_consensus.cc:738] T 00000000000000000000000000000000 P 7dbdd4b28a9d481eb01d950816966c31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dbdd4b28a9d481eb01d950816966c31, State: Initialized, Role: FOLLOWER
I20250109 08:58:18.017495 23238 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7dbdd4b28a9d481eb01d950816966c31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7dbdd4b28a9d481eb01d950816966c31" member_type: VOTER }
I20250109 08:58:18.018033 23238 raft_consensus.cc:397] T 00000000000000000000000000000000 P 7dbdd4b28a9d481eb01d950816966c31 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250109 08:58:18.018231 23238 raft_consensus.cc:491] T 00000000000000000000000000000000 P 7dbdd4b28a9d481eb01d950816966c31 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250109 08:58:18.018440 23238 raft_consensus.cc:3054] T 00000000000000000000000000000000 P 7dbdd4b28a9d481eb01d950816966c31 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:18.022423 23238 raft_consensus.cc:513] T 00000000000000000000000000000000 P 7dbdd4b28a9d481eb01d950816966c31 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7dbdd4b28a9d481eb01d950816966c31" member_type: VOTER }
I20250109 08:58:18.022858 23238 leader_election.cc:304] T 00000000000000000000000000000000 P 7dbdd4b28a9d481eb01d950816966c31 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7dbdd4b28a9d481eb01d950816966c31; no voters:
I20250109 08:58:18.023840 23238 leader_election.cc:290] T 00000000000000000000000000000000 P 7dbdd4b28a9d481eb01d950816966c31 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250109 08:58:18.024112 23241 raft_consensus.cc:2798] T 00000000000000000000000000000000 P 7dbdd4b28a9d481eb01d950816966c31 [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:58:18.025360 23241 raft_consensus.cc:695] T 00000000000000000000000000000000 P 7dbdd4b28a9d481eb01d950816966c31 [term 1 LEADER]: Becoming Leader. State: Replica: 7dbdd4b28a9d481eb01d950816966c31, State: Running, Role: LEADER
I20250109 08:58:18.025977 23241 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7dbdd4b28a9d481eb01d950816966c31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7dbdd4b28a9d481eb01d950816966c31" member_type: VOTER }
I20250109 08:58:18.026646 23238 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7dbdd4b28a9d481eb01d950816966c31 [sys.catalog]: configured and running, proceeding with master startup.
I20250109 08:58:18.030099 23243 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7dbdd4b28a9d481eb01d950816966c31 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7dbdd4b28a9d481eb01d950816966c31. Latest consensus state: current_term: 1 leader_uuid: "7dbdd4b28a9d481eb01d950816966c31" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7dbdd4b28a9d481eb01d950816966c31" member_type: VOTER } }
I20250109 08:58:18.030356 23242 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7dbdd4b28a9d481eb01d950816966c31 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7dbdd4b28a9d481eb01d950816966c31" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7dbdd4b28a9d481eb01d950816966c31" member_type: VOTER } }
I20250109 08:58:18.030717 23243 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7dbdd4b28a9d481eb01d950816966c31 [sys.catalog]: This master's current role is: LEADER
I20250109 08:58:18.030968 23242 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7dbdd4b28a9d481eb01d950816966c31 [sys.catalog]: This master's current role is: LEADER
I20250109 08:58:18.033156 23248 catalog_manager.cc:1476] Loading table and tablet metadata into memory...
I20250109 08:58:18.040308 23248 catalog_manager.cc:1485] Initializing Kudu cluster ID...
I20250109 08:58:18.041945 20363 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20250109 08:58:18.048418 23248 catalog_manager.cc:1348] Generated new cluster ID: 2680f6a831954965a2d58f46b4c6fdf2
I20250109 08:58:18.048702 23248 catalog_manager.cc:1496] Initializing Kudu internal certificate authority...
I20250109 08:58:18.071223 23248 catalog_manager.cc:1371] Generated new certificate authority record
I20250109 08:58:18.072456 23248 catalog_manager.cc:1505] Loading token signing keys...
I20250109 08:58:18.086733 23248 catalog_manager.cc:5899] T 00000000000000000000000000000000 P 7dbdd4b28a9d481eb01d950816966c31: Generated new TSK 0
I20250109 08:58:18.087311 23248 catalog_manager.cc:1515] Initializing in-progress tserver states...
I20250109 08:58:18.107964 20363 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250109 08:58:18.113636 23259 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250109 08:58:18.114563 23260 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250109 08:58:18.116209 23262 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250109 08:58:18.116905 20363 server_base.cc:1034] running on GCE node
I20250109 08:58:18.117972 20363 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250109 08:58:18.118161 20363 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250109 08:58:18.118302 20363 hybrid_clock.cc:648] HybridClock initialized: now 1736413098118288 us; error 0 us; skew 500 ppm
I20250109 08:58:18.118841 20363 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250109 08:58:18.121205 20363 webserver.cc:458] Webserver started at http://127.19.226.193:38143/ using document root <none> and password file <none>
I20250109 08:58:18.121642 20363 fs_manager.cc:362] Metadata directory not provided
I20250109 08:58:18.121819 20363 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250109 08:58:18.122054 20363 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250109 08:58:18.123108 20363 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1736412987538469-20363-0/minicluster-data/ts-0-root/instance:
uuid: "235ad6af04c34c88a5261daa3604fada"
format_stamp: "Formatted at 2025-01-09 08:58:18 on dist-test-slave-g8ds"
I20250109 08:58:18.127492 20363 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20250109 08:58:18.130767 23267 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250109 08:58:18.131503 20363 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20250109 08:58:18.131769 20363 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1736412987538469-20363-0/minicluster-data/ts-0-root
uuid: "235ad6af04c34c88a5261daa3604fada"
format_stamp: "Formatted at 2025-01-09 08:58:18 on dist-test-slave-g8ds"
I20250109 08:58:18.132033 20363 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1736412987538469-20363-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1736412987538469-20363-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1736412987538469-20363-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250109 08:58:18.143998 20363 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250109 08:58:18.145218 20363 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250109 08:58:18.146699 20363 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250109 08:58:18.148952 20363 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250109 08:58:18.149127 20363 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250109 08:58:18.149356 20363 ts_tablet_manager.cc:610] Registered 0 tablets
I20250109 08:58:18.149493 20363 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250109 08:58:18.186993 20363 rpc_server.cc:307] RPC server started. Bound to: 127.19.226.193:42691
I20250109 08:58:18.187083 23329 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.226.193:42691 every 8 connection(s)
I20250109 08:58:18.191357 20363 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250109 08:58:18.198499 23334 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250109 08:58:18.200299 23335 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250109 08:58:18.202044 23337 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250109 08:58:18.202564 20363 server_base.cc:1034] running on GCE node
I20250109 08:58:18.202800 23330 heartbeater.cc:346] Connected to a master server at 127.19.226.254:46425
I20250109 08:58:18.203275 23330 heartbeater.cc:463] Registering TS with master...
I20250109 08:58:18.204273 23330 heartbeater.cc:510] Master 127.19.226.254:46425 requested a full tablet report, sending...
I20250109 08:58:18.204983 20363 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250109 08:58:18.205205 20363 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250109 08:58:18.205374 20363 hybrid_clock.cc:648] HybridClock initialized: now 1736413098205357 us; error 0 us; skew 500 ppm
I20250109 08:58:18.205878 20363 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250109 08:58:18.206125 23203 ts_manager.cc:194] Registered new tserver with Master: 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691)
I20250109 08:58:18.207891 23203 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:34130
I20250109 08:58:18.208554 20363 webserver.cc:458] Webserver started at http://127.19.226.194:44297/ using document root <none> and password file <none>
I20250109 08:58:18.209038 20363 fs_manager.cc:362] Metadata directory not provided
I20250109 08:58:18.209210 20363 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250109 08:58:18.209493 20363 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250109 08:58:18.210734 20363 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1736412987538469-20363-0/minicluster-data/ts-1-root/instance:
uuid: "ed1efbce9ee3401e99a5260edeae4e7c"
format_stamp: "Formatted at 2025-01-09 08:58:18 on dist-test-slave-g8ds"
I20250109 08:58:18.214823 20363 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20250109 08:58:18.217933 23342 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250109 08:58:18.218585 20363 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20250109 08:58:18.218843 20363 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1736412987538469-20363-0/minicluster-data/ts-1-root
uuid: "ed1efbce9ee3401e99a5260edeae4e7c"
format_stamp: "Formatted at 2025-01-09 08:58:18 on dist-test-slave-g8ds"
I20250109 08:58:18.219097 20363 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1736412987538469-20363-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1736412987538469-20363-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1736412987538469-20363-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250109 08:58:18.227962 20363 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250109 08:58:18.229080 20363 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250109 08:58:18.230378 20363 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250109 08:58:18.232386 20363 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250109 08:58:18.232582 20363 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250109 08:58:18.232793 20363 ts_tablet_manager.cc:610] Registered 0 tablets
I20250109 08:58:18.232937 20363 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250109 08:58:18.269416 20363 rpc_server.cc:307] RPC server started. Bound to: 127.19.226.194:41365
I20250109 08:58:18.269505 23404 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.226.194:41365 every 8 connection(s)
I20250109 08:58:18.273797 20363 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250109 08:58:18.281615 23408 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250109 08:58:18.281737 23409 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250109 08:58:18.283293 23411 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250109 08:58:18.283921 20363 server_base.cc:1034] running on GCE node
I20250109 08:58:18.284997 20363 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20250109 08:58:18.285037 23405 heartbeater.cc:346] Connected to a master server at 127.19.226.254:46425
W20250109 08:58:18.285351 20363 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250109 08:58:18.285516 23405 heartbeater.cc:463] Registering TS with master...
I20250109 08:58:18.285569 20363 hybrid_clock.cc:648] HybridClock initialized: now 1736413098285541 us; error 0 us; skew 500 ppm
I20250109 08:58:18.286424 20363 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250109 08:58:18.286499 23405 heartbeater.cc:510] Master 127.19.226.254:46425 requested a full tablet report, sending...
I20250109 08:58:18.288581 23203 ts_manager.cc:194] Registered new tserver with Master: ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365)
I20250109 08:58:18.289183 20363 webserver.cc:458] Webserver started at http://127.19.226.195:44477/ using document root <none> and password file <none>
I20250109 08:58:18.289670 20363 fs_manager.cc:362] Metadata directory not provided
I20250109 08:58:18.289842 20363 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250109 08:58:18.289973 23203 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:34144
I20250109 08:58:18.290074 20363 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250109 08:58:18.291447 20363 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1736412987538469-20363-0/minicluster-data/ts-2-root/instance:
uuid: "cf55383bef6649eb9e78cb2d2512dedc"
format_stamp: "Formatted at 2025-01-09 08:58:18 on dist-test-slave-g8ds"
I20250109 08:58:18.295939 20363 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20250109 08:58:18.298911 23416 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250109 08:58:18.299624 20363 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20250109 08:58:18.299896 20363 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1736412987538469-20363-0/minicluster-data/ts-2-root
uuid: "cf55383bef6649eb9e78cb2d2512dedc"
format_stamp: "Formatted at 2025-01-09 08:58:18 on dist-test-slave-g8ds"
I20250109 08:58:18.300153 20363 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1736412987538469-20363-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1736412987538469-20363-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1736412987538469-20363-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250109 08:58:18.316840 20363 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250109 08:58:18.318039 20363 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250109 08:58:18.319484 20363 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250109 08:58:18.321843 20363 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250109 08:58:18.322037 20363 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250109 08:58:18.322247 20363 ts_tablet_manager.cc:610] Registered 0 tablets
I20250109 08:58:18.322393 20363 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250109 08:58:18.359484 20363 rpc_server.cc:307] RPC server started. Bound to: 127.19.226.195:35551
I20250109 08:58:18.359578 23478 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.226.195:35551 every 8 connection(s)
I20250109 08:58:18.364048 20363 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250109 08:58:18.370610 23482 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250109 08:58:18.372769 23483 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250109 08:58:18.373759 23479 heartbeater.cc:346] Connected to a master server at 127.19.226.254:46425
I20250109 08:58:18.374244 23479 heartbeater.cc:463] Registering TS with master...
I20250109 08:58:18.375150 23479 heartbeater.cc:510] Master 127.19.226.254:46425 requested a full tablet report, sending...
I20250109 08:58:18.377487 23203 ts_manager.cc:194] Registered new tserver with Master: cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551)
I20250109 08:58:18.378386 20363 server_base.cc:1034] running on GCE node
W20250109 08:58:18.378583 23485 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250109 08:58:18.379173 23203 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:34156
I20250109 08:58:18.379621 20363 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250109 08:58:18.379910 20363 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250109 08:58:18.380116 20363 hybrid_clock.cc:648] HybridClock initialized: now 1736413098380097 us; error 0 us; skew 500 ppm
I20250109 08:58:18.380825 20363 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250109 08:58:18.383131 20363 webserver.cc:458] Webserver started at http://127.19.226.196:32883/ using document root <none> and password file <none>
I20250109 08:58:18.383538 20363 fs_manager.cc:362] Metadata directory not provided
I20250109 08:58:18.383693 20363 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250109 08:58:18.383889 20363 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250109 08:58:18.384893 20363 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1736412987538469-20363-0/minicluster-data/ts-3-root/instance:
uuid: "05a74e1eb2474b3d8b3a2da1ef127e01"
format_stamp: "Formatted at 2025-01-09 08:58:18 on dist-test-slave-g8ds"
I20250109 08:58:18.388823 20363 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20250109 08:58:18.391760 23490 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250109 08:58:18.392475 20363 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20250109 08:58:18.392747 20363 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1736412987538469-20363-0/minicluster-data/ts-3-root
uuid: "05a74e1eb2474b3d8b3a2da1ef127e01"
format_stamp: "Formatted at 2025-01-09 08:58:18 on dist-test-slave-g8ds"
I20250109 08:58:18.393004 20363 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1736412987538469-20363-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1736412987538469-20363-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1736412987538469-20363-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250109 08:58:18.414824 20363 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250109 08:58:18.415884 20363 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250109 08:58:18.417259 20363 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250109 08:58:18.419323 20363 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250109 08:58:18.419515 20363 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250109 08:58:18.419734 20363 ts_tablet_manager.cc:610] Registered 0 tablets
I20250109 08:58:18.419881 20363 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250109 08:58:18.457778 20363 rpc_server.cc:307] RPC server started. Bound to: 127.19.226.196:41829
I20250109 08:58:18.457860 23552 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.226.196:41829 every 8 connection(s)
I20250109 08:58:18.470100 23553 heartbeater.cc:346] Connected to a master server at 127.19.226.254:46425
I20250109 08:58:18.470459 23553 heartbeater.cc:463] Registering TS with master...
I20250109 08:58:18.471099 23553 heartbeater.cc:510] Master 127.19.226.254:46425 requested a full tablet report, sending...
I20250109 08:58:18.472852 23203 ts_manager.cc:194] Registered new tserver with Master: 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829)
I20250109 08:58:18.473932 20363 internal_mini_cluster.cc:371] 4 TS(s) registered with all masters after 0.01316082s
I20250109 08:58:18.474081 23203 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:34170
I20250109 08:58:18.476214 20363 test_util.cc:274] Using random seed: 770376354
I20250109 08:58:18.497565 23203 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34174:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250109 08:58:18.730345 23518 tablet_service.cc:1467] Processing CreateTablet for tablet bd6fcd838b0249cabd5f1f95e5538ab6 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION VALUES < 36398027
I20250109 08:58:18.731737 23518 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd6fcd838b0249cabd5f1f95e5538ab6. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:18.733494 23517 tablet_service.cc:1467] Processing CreateTablet for tablet 41e7f0cc38c4414ab88c92c1cdeb3984 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20250109 08:58:18.734803 23517 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 41e7f0cc38c4414ab88c92c1cdeb3984. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:18.735121 23513 tablet_service.cc:1467] Processing CreateTablet for tablet 8c5e3d0155d140c4b32f162d3173f5c9 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20250109 08:58:18.736351 23513 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8c5e3d0155d140c4b32f162d3173f5c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:18.737354 23516 tablet_service.cc:1467] Processing CreateTablet for tablet 34caa5988a734b65a2eb512747d833b7 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20250109 08:58:18.738201 23512 tablet_service.cc:1467] Processing CreateTablet for tablet ea2b0762940e44dbbf69b9a163ace846 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20250109 08:58:18.733374 23514 tablet_service.cc:1467] Processing CreateTablet for tablet 82ea7dc042dd4d05b260d508f4422c3c (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20250109 08:58:18.740226 23511 tablet_service.cc:1467] Processing CreateTablet for tablet 5ccc0d42d4a148d7819366cea0b754a7 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20250109 08:58:18.742435 23510 tablet_service.cc:1467] Processing CreateTablet for tablet 9b0bbf2e4a424fb5ac13a3b55627b048 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20250109 08:58:18.744997 23509 tablet_service.cc:1467] Processing CreateTablet for tablet ed79047393d2425f8ecff31ea08b9cad (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20250109 08:58:18.738632 23516 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34caa5988a734b65a2eb512747d833b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:18.746062 23515 tablet_service.cc:1467] Processing CreateTablet for tablet 970a82e754db47b6b1a19551d4ba43d8 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20250109 08:58:18.747316 23515 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 970a82e754db47b6b1a19551d4ba43d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:18.749325 23509 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed79047393d2425f8ecff31ea08b9cad. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:18.749953 23510 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b0bbf2e4a424fb5ac13a3b55627b048. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:18.750533 23514 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82ea7dc042dd4d05b260d508f4422c3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:18.751137 23511 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ccc0d42d4a148d7819366cea0b754a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:18.752677 23512 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea2b0762940e44dbbf69b9a163ace846. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:18.815973 23511 tablet_service.cc:1467] Processing CreateTablet for tablet d5fa255ab5f24750a0afa6d81ec869f2 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20250109 08:58:18.816066 23509 tablet_service.cc:1467] Processing CreateTablet for tablet 9b245cb3666c421c884cf20161a22831 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20250109 08:58:18.821393 23509 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b245cb3666c421c884cf20161a22831. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:18.821413 23569 tablet_bootstrap.cc:492] T ed79047393d2425f8ecff31ea08b9cad P 05a74e1eb2474b3d8b3a2da1ef127e01: Bootstrap starting.
I20250109 08:58:18.822983 23512 tablet_service.cc:1467] Processing CreateTablet for tablet 4df64aae69ca41dc91286cae1603000f (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20250109 08:58:18.825317 23518 tablet_service.cc:1467] Processing CreateTablet for tablet 5c37c58abaa54656b31c72160b4dec9e (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20250109 08:58:18.826349 23515 tablet_service.cc:1467] Processing CreateTablet for tablet 940653b6d55c46439827697b9aadf16b (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20250109 08:58:18.829506 23514 tablet_service.cc:1467] Processing CreateTablet for tablet 1c5b5beea4a840eb9677097cacee3408 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20250109 08:58:18.831753 23569 tablet_bootstrap.cc:654] T ed79047393d2425f8ecff31ea08b9cad P 05a74e1eb2474b3d8b3a2da1ef127e01: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:18.830188 23513 tablet_service.cc:1467] Processing CreateTablet for tablet b572a6a3dfc04ea199bc703ddad8af67 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20250109 08:58:18.826764 23512 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4df64aae69ca41dc91286cae1603000f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:18.834133 23510 tablet_service.cc:1467] Processing CreateTablet for tablet 8de6b2fb09bc46ce9e4033af06dffd04 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20250109 08:58:18.835333 23511 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5fa255ab5f24750a0afa6d81ec869f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:18.835963 23515 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 940653b6d55c46439827697b9aadf16b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:18.837749 23516 tablet_service.cc:1467] Processing CreateTablet for tablet 2ab70546dda143b5b5db15c3875aa348 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20250109 08:58:18.841434 23517 tablet_service.cc:1467] Processing CreateTablet for tablet cc9886333c0343798226d59758a1ea5e (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20250109 08:58:18.845922 23569 tablet_bootstrap.cc:492] T ed79047393d2425f8ecff31ea08b9cad P 05a74e1eb2474b3d8b3a2da1ef127e01: No bootstrap required, opened a new log
I20250109 08:58:18.846490 23569 ts_tablet_manager.cc:1397] T ed79047393d2425f8ecff31ea08b9cad P 05a74e1eb2474b3d8b3a2da1ef127e01: Time spent bootstrapping tablet: real 0.025s user 0.009s sys 0.000s
I20250109 08:58:18.849181 23569 raft_consensus.cc:357] T ed79047393d2425f8ecff31ea08b9cad P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:18.849924 23569 raft_consensus.cc:383] T ed79047393d2425f8ecff31ea08b9cad P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:18.850203 23569 raft_consensus.cc:738] T ed79047393d2425f8ecff31ea08b9cad P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05a74e1eb2474b3d8b3a2da1ef127e01, State: Initialized, Role: FOLLOWER
I20250109 08:58:18.852303 23444 tablet_service.cc:1467] Processing CreateTablet for tablet bd6fcd838b0249cabd5f1f95e5538ab6 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION VALUES < 36398027
I20250109 08:58:18.853694 23444 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd6fcd838b0249cabd5f1f95e5538ab6. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:18.856446 23569 consensus_queue.cc:260] T ed79047393d2425f8ecff31ea08b9cad P 05a74e1eb2474b3d8b3a2da1ef127e01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:18.858981 23509 tablet_service.cc:1467] Processing CreateTablet for tablet 5831288451724d4ca1114ec5e5f01b78 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20250109 08:58:18.860085 23511 tablet_service.cc:1467] Processing CreateTablet for tablet 08214ee9c4c14bda92bdc6fa8231c313 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20250109 08:58:18.861438 23510 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8de6b2fb09bc46ce9e4033af06dffd04. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:18.862066 23516 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ab70546dda143b5b5db15c3875aa348. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:18.864253 23514 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c5b5beea4a840eb9677097cacee3408. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:18.866080 23511 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08214ee9c4c14bda92bdc6fa8231c313. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:18.873880 23513 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b572a6a3dfc04ea199bc703ddad8af67. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:18.876690 23517 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc9886333c0343798226d59758a1ea5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:18.878432 23518 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5c37c58abaa54656b31c72160b4dec9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:18.881332 23509 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5831288451724d4ca1114ec5e5f01b78. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:18.895267 23569 ts_tablet_manager.cc:1428] T ed79047393d2425f8ecff31ea08b9cad P 05a74e1eb2474b3d8b3a2da1ef127e01: Time spent starting tablet: real 0.048s user 0.010s sys 0.012s
I20250109 08:58:18.895947 23553 heartbeater.cc:502] Master 127.19.226.254:46425 was elected leader, sending a full tablet report...
I20250109 08:58:18.898398 23443 tablet_service.cc:1467] Processing CreateTablet for tablet 1340cf8ee2cb423f875276137da17e8b (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20250109 08:58:18.899663 23443 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1340cf8ee2cb423f875276137da17e8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:18.900416 23367 tablet_service.cc:1467] Processing CreateTablet for tablet 6eef9bb3c645428d9f1cb843bae9a264 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20250109 08:58:18.901381 23441 tablet_service.cc:1467] Processing CreateTablet for tablet 6eef9bb3c645428d9f1cb843bae9a264 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20250109 08:58:18.901856 23367 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6eef9bb3c645428d9f1cb843bae9a264. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:18.903990 23439 tablet_service.cc:1467] Processing CreateTablet for tablet 8c5e3d0155d140c4b32f162d3173f5c9 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20250109 08:58:18.905278 23439 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8c5e3d0155d140c4b32f162d3173f5c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:18.905551 23366 tablet_service.cc:1467] Processing CreateTablet for tablet 970a82e754db47b6b1a19551d4ba43d8 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20250109 08:58:18.906952 23366 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 970a82e754db47b6b1a19551d4ba43d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:18.909366 23370 tablet_service.cc:1467] Processing CreateTablet for tablet 1340cf8ee2cb423f875276137da17e8b (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20250109 08:58:18.910684 23370 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1340cf8ee2cb423f875276137da17e8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:18.910543 23364 tablet_service.cc:1467] Processing CreateTablet for tablet 2e3d34c0d09843d2be54e47f3ccc16d8 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20250109 08:58:18.911819 23364 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e3d34c0d09843d2be54e47f3ccc16d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:18.913317 23438 tablet_service.cc:1467] Processing CreateTablet for tablet 2e3d34c0d09843d2be54e47f3ccc16d8 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20250109 08:58:18.913461 23363 tablet_service.cc:1467] Processing CreateTablet for tablet ea2b0762940e44dbbf69b9a163ace846 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20250109 08:58:18.914603 23438 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e3d34c0d09843d2be54e47f3ccc16d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:18.914767 23363 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea2b0762940e44dbbf69b9a163ace846. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:18.917371 23437 tablet_service.cc:1467] Processing CreateTablet for tablet 5ccc0d42d4a148d7819366cea0b754a7 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20250109 08:58:18.917366 23362 tablet_service.cc:1467] Processing CreateTablet for tablet 9b0bbf2e4a424fb5ac13a3b55627b048 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20250109 08:58:18.918036 23295 tablet_service.cc:1467] Processing CreateTablet for tablet bd6fcd838b0249cabd5f1f95e5538ab6 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION VALUES < 36398027
I20250109 08:58:18.918669 23437 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ccc0d42d4a148d7819366cea0b754a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:18.918972 23362 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b0bbf2e4a424fb5ac13a3b55627b048. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:18.919289 23295 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd6fcd838b0249cabd5f1f95e5538ab6. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:18.909437 23365 tablet_service.cc:1467] Processing CreateTablet for tablet 8c5e3d0155d140c4b32f162d3173f5c9 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20250109 08:58:18.920387 23368 tablet_service.cc:1467] Processing CreateTablet for tablet 34caa5988a734b65a2eb512747d833b7 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20250109 08:58:18.921048 23365 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8c5e3d0155d140c4b32f162d3173f5c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:18.921739 23368 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34caa5988a734b65a2eb512747d833b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:18.923540 23442 tablet_service.cc:1467] Processing CreateTablet for tablet 34caa5988a734b65a2eb512747d833b7 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20250109 08:58:18.925029 23369 tablet_service.cc:1467] Processing CreateTablet for tablet 41e7f0cc38c4414ab88c92c1cdeb3984 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20250109 08:58:18.926313 23369 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 41e7f0cc38c4414ab88c92c1cdeb3984. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:18.927755 23440 tablet_service.cc:1467] Processing CreateTablet for tablet 82ea7dc042dd4d05b260d508f4422c3c (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20250109 08:58:18.928484 23361 tablet_service.cc:1467] Processing CreateTablet for tablet 3208a99957554f6db341654856980c8a (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20250109 08:58:18.929687 23361 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3208a99957554f6db341654856980c8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:18.933041 23442 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34caa5988a734b65a2eb512747d833b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:18.937044 23441 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6eef9bb3c645428d9f1cb843bae9a264. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:18.940821 23440 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82ea7dc042dd4d05b260d508f4422c3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:18.939288 23511 tablet_service.cc:1467] Processing CreateTablet for tablet 24ce131f7bb143bea1581d5b0c2a7109 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20250109 08:58:18.944151 23511 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 24ce131f7bb143bea1581d5b0c2a7109. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:18.948112 23436 tablet_service.cc:1467] Processing CreateTablet for tablet 9b0bbf2e4a424fb5ac13a3b55627b048 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20250109 08:58:18.949487 23436 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b0bbf2e4a424fb5ac13a3b55627b048. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:18.950791 23514 tablet_service.cc:1467] Processing CreateTablet for tablet a686bf4d947c4b638049e6b88c59ae09 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20250109 08:58:18.952316 23435 tablet_service.cc:1467] Processing CreateTablet for tablet 3208a99957554f6db341654856980c8a (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20250109 08:58:18.960297 23435 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3208a99957554f6db341654856980c8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:18.957348 23513 tablet_service.cc:1467] Processing CreateTablet for tablet d87d3b4755344b82b0aec7ffd1ce785c (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20250109 08:58:18.958902 23516 tablet_service.cc:1467] Processing CreateTablet for tablet e5251927614642639bd46ecd5373d2a4 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
W20250109 08:58:18.961627 23554 tablet.cc:2367] T ed79047393d2425f8ecff31ea08b9cad P 05a74e1eb2474b3d8b3a2da1ef127e01: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20250109 08:58:18.962467 23514 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a686bf4d947c4b638049e6b88c59ae09. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:18.966760 23513 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d87d3b4755344b82b0aec7ffd1ce785c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:18.970021 23569 tablet_bootstrap.cc:492] T 5ccc0d42d4a148d7819366cea0b754a7 P 05a74e1eb2474b3d8b3a2da1ef127e01: Bootstrap starting.
I20250109 08:58:18.976289 23574 tablet_bootstrap.cc:492] T bd6fcd838b0249cabd5f1f95e5538ab6 P 235ad6af04c34c88a5261daa3604fada: Bootstrap starting.
I20250109 08:58:18.981156 23511 tablet_service.cc:1467] Processing CreateTablet for tablet 0dab6c23ce444dda9427901a98f07b15 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20250109 08:58:18.987959 23517 tablet_service.cc:1467] Processing CreateTablet for tablet 86b82e19759c4fd0b471a0dbff839aa5 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20250109 08:58:18.989217 23516 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e5251927614642639bd46ecd5373d2a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:18.995823 23438 tablet_service.cc:1467] Processing CreateTablet for tablet a686bf4d947c4b638049e6b88c59ae09 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20250109 08:58:19.006588 23571 raft_consensus.cc:491] T ed79047393d2425f8ecff31ea08b9cad P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:19.025158 23571 raft_consensus.cc:513] T ed79047393d2425f8ecff31ea08b9cad P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:19.027720 23571 leader_election.cc:290] T ed79047393d2425f8ecff31ea08b9cad P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691), ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365)
I20250109 08:58:19.028416 23439 tablet_service.cc:1467] Processing CreateTablet for tablet d5fa255ab5f24750a0afa6d81ec869f2 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20250109 08:58:19.029803 23439 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5fa255ab5f24750a0afa6d81ec869f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:19.030506 23574 tablet_bootstrap.cc:654] T bd6fcd838b0249cabd5f1f95e5538ab6 P 235ad6af04c34c88a5261daa3604fada: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:19.031533 23572 tablet_bootstrap.cc:492] T bd6fcd838b0249cabd5f1f95e5538ab6 P cf55383bef6649eb9e78cb2d2512dedc: Bootstrap starting.
I20250109 08:58:19.037588 23444 tablet_service.cc:1467] Processing CreateTablet for tablet a54ca9f0d6e844e9a8b1d9cd0bb2a026 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20250109 08:58:19.038928 23444 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a54ca9f0d6e844e9a8b1d9cd0bb2a026. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:19.040315 23517 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86b82e19759c4fd0b471a0dbff839aa5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:19.043186 23515 tablet_service.cc:1467] Processing CreateTablet for tablet 5f0d500a91b54851bda52e5ccf741fd6 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20250109 08:58:19.049115 23572 tablet_bootstrap.cc:654] T bd6fcd838b0249cabd5f1f95e5538ab6 P cf55383bef6649eb9e78cb2d2512dedc: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:19.055883 23515 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5f0d500a91b54851bda52e5ccf741fd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:19.056564 23511 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0dab6c23ce444dda9427901a98f07b15. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:19.059087 23509 tablet_service.cc:1467] Processing CreateTablet for tablet 750dd1e1055f4570b0aecdda3ebe8772 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20250109 08:58:19.061705 23513 tablet_service.cc:1467] Processing CreateTablet for tablet 94f2be307f704c75bceb213e52307cfb (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20250109 08:58:19.062507 23509 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 750dd1e1055f4570b0aecdda3ebe8772. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:19.063064 23513 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 94f2be307f704c75bceb213e52307cfb. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:19.061705 23512 tablet_service.cc:1467] Processing CreateTablet for tablet fc1038255dd64cb5ba2425037d22f1f7 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20250109 08:58:19.069139 23512 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc1038255dd64cb5ba2425037d22f1f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:19.070658 23514 tablet_service.cc:1467] Processing CreateTablet for tablet 11aa76a3a30643708fececed660a5250 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20250109 08:58:19.071920 23514 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11aa76a3a30643708fececed660a5250. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:19.081276 23517 tablet_service.cc:1467] Processing CreateTablet for tablet ecc833ad1afa4079a75e41b1a807c923 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20250109 08:58:19.082587 23569 tablet_bootstrap.cc:654] T 5ccc0d42d4a148d7819366cea0b754a7 P 05a74e1eb2474b3d8b3a2da1ef127e01: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:19.083673 23513 tablet_service.cc:1467] Processing CreateTablet for tablet a7d2ab61d238471c9f287ac1d5db4ed0 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20250109 08:58:19.085727 23513 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a7d2ab61d238471c9f287ac1d5db4ed0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:19.085623 23510 tablet_service.cc:1467] Processing CreateTablet for tablet 77597d68423749e7b87f9f0db7a1ff3b (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20250109 08:58:19.086812 23516 tablet_service.cc:1467] Processing CreateTablet for tablet 444507f7812441ceaeebc44bc8f06293 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20250109 08:58:19.088080 23516 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 444507f7812441ceaeebc44bc8f06293. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:19.089146 23511 tablet_service.cc:1467] Processing CreateTablet for tablet 1f9c6525267145d6bb73dde32ed9b7d4 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20250109 08:58:19.090430 23511 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f9c6525267145d6bb73dde32ed9b7d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:19.090304 23518 tablet_service.cc:1467] Processing CreateTablet for tablet 29828d9befb845d5bd49a80e018502e3 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20250109 08:58:19.083643 23515 tablet_service.cc:1467] Processing CreateTablet for tablet 060daed1d3334c6188413dd381b82bf9 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20250109 08:58:19.091634 23518 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 29828d9befb845d5bd49a80e018502e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:19.093406 23517 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ecc833ad1afa4079a75e41b1a807c923. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:19.093457 23569 tablet_bootstrap.cc:492] T 5ccc0d42d4a148d7819366cea0b754a7 P 05a74e1eb2474b3d8b3a2da1ef127e01: No bootstrap required, opened a new log
I20250109 08:58:19.093991 23569 ts_tablet_manager.cc:1397] T 5ccc0d42d4a148d7819366cea0b754a7 P 05a74e1eb2474b3d8b3a2da1ef127e01: Time spent bootstrapping tablet: real 0.198s user 0.018s sys 0.003s
I20250109 08:58:19.094064 23515 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 060daed1d3334c6188413dd381b82bf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:19.096858 23569 raft_consensus.cc:357] T 5ccc0d42d4a148d7819366cea0b754a7 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:19.097604 23569 raft_consensus.cc:383] T 5ccc0d42d4a148d7819366cea0b754a7 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:19.097904 23569 raft_consensus.cc:738] T 5ccc0d42d4a148d7819366cea0b754a7 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05a74e1eb2474b3d8b3a2da1ef127e01, State: Initialized, Role: FOLLOWER
I20250109 08:58:19.098577 23569 consensus_queue.cc:260] T 5ccc0d42d4a148d7819366cea0b754a7 P 05a74e1eb2474b3d8b3a2da1ef127e01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:19.101137 23569 ts_tablet_manager.cc:1428] T 5ccc0d42d4a148d7819366cea0b754a7 P 05a74e1eb2474b3d8b3a2da1ef127e01: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20250109 08:58:19.102077 23569 tablet_bootstrap.cc:492] T ea2b0762940e44dbbf69b9a163ace846 P 05a74e1eb2474b3d8b3a2da1ef127e01: Bootstrap starting.
I20250109 08:58:19.098582 23572 tablet_bootstrap.cc:492] T bd6fcd838b0249cabd5f1f95e5538ab6 P cf55383bef6649eb9e78cb2d2512dedc: No bootstrap required, opened a new log
I20250109 08:58:19.104174 23572 ts_tablet_manager.cc:1397] T bd6fcd838b0249cabd5f1f95e5538ab6 P cf55383bef6649eb9e78cb2d2512dedc: Time spent bootstrapping tablet: real 0.073s user 0.011s sys 0.000s
I20250109 08:58:19.105053 23574 tablet_bootstrap.cc:492] T bd6fcd838b0249cabd5f1f95e5538ab6 P 235ad6af04c34c88a5261daa3604fada: No bootstrap required, opened a new log
I20250109 08:58:19.105518 23574 ts_tablet_manager.cc:1397] T bd6fcd838b0249cabd5f1f95e5538ab6 P 235ad6af04c34c88a5261daa3604fada: Time spent bootstrapping tablet: real 0.130s user 0.013s sys 0.021s
I20250109 08:58:19.106909 23572 raft_consensus.cc:357] T bd6fcd838b0249cabd5f1f95e5538ab6 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:19.107667 23572 raft_consensus.cc:383] T bd6fcd838b0249cabd5f1f95e5538ab6 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:19.107950 23572 raft_consensus.cc:738] T bd6fcd838b0249cabd5f1f95e5538ab6 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf55383bef6649eb9e78cb2d2512dedc, State: Initialized, Role: FOLLOWER
I20250109 08:58:19.108098 23574 raft_consensus.cc:357] T bd6fcd838b0249cabd5f1f95e5538ab6 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
W20250109 08:58:19.147938 23251 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20250109 08:58:19.154081 23294 tablet_service.cc:1467] Processing CreateTablet for tablet 41e7f0cc38c4414ab88c92c1cdeb3984 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20250109 08:58:19.156296 23294 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 41e7f0cc38c4414ab88c92c1cdeb3984. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:19.158181 23286 tablet_service.cc:1467] Processing CreateTablet for tablet ed79047393d2425f8ecff31ea08b9cad (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20250109 08:58:19.159783 23286 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed79047393d2425f8ecff31ea08b9cad. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:19.161684 23287 tablet_service.cc:1467] Processing CreateTablet for tablet 3208a99957554f6db341654856980c8a (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20250109 08:58:19.163414 23287 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3208a99957554f6db341654856980c8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:19.166085 23288 tablet_service.cc:1467] Processing CreateTablet for tablet 5ccc0d42d4a148d7819366cea0b754a7 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20250109 08:58:19.167737 23288 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ccc0d42d4a148d7819366cea0b754a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:19.170008 23289 tablet_service.cc:1467] Processing CreateTablet for tablet ea2b0762940e44dbbf69b9a163ace846 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20250109 08:58:19.171901 23289 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea2b0762940e44dbbf69b9a163ace846. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:19.173935 23290 tablet_service.cc:1467] Processing CreateTablet for tablet 2e3d34c0d09843d2be54e47f3ccc16d8 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20250109 08:58:19.175679 23290 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e3d34c0d09843d2be54e47f3ccc16d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:19.177671 23291 tablet_service.cc:1467] Processing CreateTablet for tablet 82ea7dc042dd4d05b260d508f4422c3c (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20250109 08:58:19.179133 23291 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82ea7dc042dd4d05b260d508f4422c3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:19.182024 23292 tablet_service.cc:1467] Processing CreateTablet for tablet 970a82e754db47b6b1a19551d4ba43d8 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20250109 08:58:19.185137 23293 tablet_service.cc:1467] Processing CreateTablet for tablet 6eef9bb3c645428d9f1cb843bae9a264 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20250109 08:58:19.188319 23295 tablet_service.cc:1467] Processing CreateTablet for tablet 1340cf8ee2cb423f875276137da17e8b (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20250109 08:58:19.040279 23443 tablet_service.cc:1467] Processing CreateTablet for tablet 5c37c58abaa54656b31c72160b4dec9e (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20250109 08:58:19.210772 23330 heartbeater.cc:502] Master 127.19.226.254:46425 was elected leader, sending a full tablet report...
I20250109 08:58:19.213801 23292 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 970a82e754db47b6b1a19551d4ba43d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:19.216744 23293 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6eef9bb3c645428d9f1cb843bae9a264. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:19.224781 23295 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1340cf8ee2cb423f875276137da17e8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:19.233513 23510 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77597d68423749e7b87f9f0db7a1ff3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:19.108840 23574 raft_consensus.cc:383] T bd6fcd838b0249cabd5f1f95e5538ab6 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:19.235100 23574 raft_consensus.cc:738] T bd6fcd838b0249cabd5f1f95e5538ab6 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Initialized, Role: FOLLOWER
I20250109 08:58:19.236668 23516 tablet_service.cc:1467] Processing CreateTablet for tablet 950e07e9cc794b8ab9ec5bd97d7e8e59 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20250109 08:58:19.236198 23574 consensus_queue.cc:260] T bd6fcd838b0249cabd5f1f95e5538ab6 P 235ad6af04c34c88a5261daa3604fada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:19.238104 23516 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 950e07e9cc794b8ab9ec5bd97d7e8e59. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:19.275167 23366 tablet_service.cc:1467] Processing CreateTablet for tablet 4df64aae69ca41dc91286cae1603000f (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20250109 08:58:19.284456 23366 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4df64aae69ca41dc91286cae1603000f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:19.285495 23572 consensus_queue.cc:260] T bd6fcd838b0249cabd5f1f95e5538ab6 P cf55383bef6649eb9e78cb2d2512dedc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:19.287799 23438 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a686bf4d947c4b638049e6b88c59ae09. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:19.289137 23440 tablet_service.cc:1467] Processing CreateTablet for tablet 940653b6d55c46439827697b9aadf16b (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20250109 08:58:19.291484 23440 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 940653b6d55c46439827697b9aadf16b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:19.292824 23405 heartbeater.cc:502] Master 127.19.226.254:46425 was elected leader, sending a full tablet report...
I20250109 08:58:19.293603 23435 tablet_service.cc:1467] Processing CreateTablet for tablet e6b9aaeda9a947d4a87fedd774301cc0 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20250109 08:58:19.294929 23437 tablet_service.cc:1467] Processing CreateTablet for tablet 9b245cb3666c421c884cf20161a22831 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20250109 08:58:19.295483 23435 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e6b9aaeda9a947d4a87fedd774301cc0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:19.296228 23437 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b245cb3666c421c884cf20161a22831. 1 dirs total, 0 dirs full, 0 dirs failed
W20250109 08:58:19.298760 23188 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.19.226.196:41829, user_credentials={real_user=slave}} blocked reactor thread for 44794.5us
I20250109 08:58:19.299365 23365 tablet_service.cc:1467] Processing CreateTablet for tablet ed79047393d2425f8ecff31ea08b9cad (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20250109 08:58:19.311499 23580 tablet_bootstrap.cc:492] T 970a82e754db47b6b1a19551d4ba43d8 P ed1efbce9ee3401e99a5260edeae4e7c: Bootstrap starting.
I20250109 08:58:19.312062 23365 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed79047393d2425f8ecff31ea08b9cad. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:19.321806 23443 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5c37c58abaa54656b31c72160b4dec9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:19.322484 23442 tablet_service.cc:1467] Processing CreateTablet for tablet b572a6a3dfc04ea199bc703ddad8af67 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20250109 08:58:19.323756 23442 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b572a6a3dfc04ea199bc703ddad8af67. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:19.336025 23361 tablet_service.cc:1467] Processing CreateTablet for tablet a54ca9f0d6e844e9a8b1d9cd0bb2a026 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20250109 08:58:19.338523 23517 tablet_service.cc:1467] Processing CreateTablet for tablet 1ac4317290b24209a76d2e19e68ee45b (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20250109 08:58:19.339798 23517 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ac4317290b24209a76d2e19e68ee45b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:19.341995 23364 tablet_service.cc:1467] Processing CreateTablet for tablet d5fa255ab5f24750a0afa6d81ec869f2 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20250109 08:58:19.347035 23509 tablet_service.cc:1467] Processing CreateTablet for tablet ee703ea575234c659da04f21ef3cdd70 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20250109 08:58:19.355283 23370 tablet_service.cc:1467] Processing CreateTablet for tablet 940653b6d55c46439827697b9aadf16b (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20250109 08:58:19.362922 23514 tablet_service.cc:1467] Processing CreateTablet for tablet bfb140bb3a8b44bd8b4e1fcf0946162b (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20250109 08:58:19.366075 23569 tablet_bootstrap.cc:654] T ea2b0762940e44dbbf69b9a163ace846 P 05a74e1eb2474b3d8b3a2da1ef127e01: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:19.367512 23293 tablet_service.cc:1467] Processing CreateTablet for tablet 9b245cb3666c421c884cf20161a22831 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20250109 08:58:19.368984 23293 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b245cb3666c421c884cf20161a22831. 1 dirs total, 0 dirs full, 0 dirs failed
W20250109 08:58:19.372490 23188 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.19.226.196:41829, user_credentials={real_user=slave}} blocked reactor thread for 68302.9us
I20250109 08:58:19.373754 23440 tablet_service.cc:1467] Processing CreateTablet for tablet cc9886333c0343798226d59758a1ea5e (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20250109 08:58:19.375183 23440 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc9886333c0343798226d59758a1ea5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:19.382012 23479 heartbeater.cc:502] Master 127.19.226.254:46425 was elected leader, sending a full tablet report...
I20250109 08:58:19.388868 23436 tablet_service.cc:1467] Processing CreateTablet for tablet 5831288451724d4ca1114ec5e5f01b78 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20250109 08:58:19.390218 23436 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5831288451724d4ca1114ec5e5f01b78. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:19.392771 23362 tablet_service.cc:1467] Processing CreateTablet for tablet e6b9aaeda9a947d4a87fedd774301cc0 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20250109 08:58:19.394043 23362 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e6b9aaeda9a947d4a87fedd774301cc0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:19.399999 23366 tablet_service.cc:1467] Processing CreateTablet for tablet 1c5b5beea4a840eb9677097cacee3408 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20250109 08:58:19.402729 23365 tablet_service.cc:1467] Processing CreateTablet for tablet 8de6b2fb09bc46ce9e4033af06dffd04 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20250109 08:58:19.407860 23365 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8de6b2fb09bc46ce9e4033af06dffd04. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:19.348356 23509 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee703ea575234c659da04f21ef3cdd70. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:19.360085 23441 tablet_service.cc:1467] Processing CreateTablet for tablet 8de6b2fb09bc46ce9e4033af06dffd04 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20250109 08:58:19.416074 23441 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8de6b2fb09bc46ce9e4033af06dffd04. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:19.372242 23439 tablet_service.cc:1467] Processing CreateTablet for tablet 86d4074509744900abe94402467a7d85 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20250109 08:58:19.417495 23439 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86d4074509744900abe94402467a7d85. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:19.420818 23366 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c5b5beea4a840eb9677097cacee3408. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:19.426208 23363 tablet_service.cc:1467] Processing CreateTablet for tablet 2ab70546dda143b5b5db15c3875aa348 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20250109 08:58:19.427488 23363 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ab70546dda143b5b5db15c3875aa348. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:19.430655 23293 tablet_service.cc:1467] Processing CreateTablet for tablet a54ca9f0d6e844e9a8b1d9cd0bb2a026 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20250109 08:58:19.433704 23294 tablet_service.cc:1467] Processing CreateTablet for tablet e6b9aaeda9a947d4a87fedd774301cc0 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20250109 08:58:19.434437 23293 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a54ca9f0d6e844e9a8b1d9cd0bb2a026. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:19.432968 23290 tablet_service.cc:1467] Processing CreateTablet for tablet 5c37c58abaa54656b31c72160b4dec9e (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20250109 08:58:19.432968 23289 tablet_service.cc:1467] Processing CreateTablet for tablet 4df64aae69ca41dc91286cae1603000f (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20250109 08:58:19.435842 23290 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5c37c58abaa54656b31c72160b4dec9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:19.443674 23365 tablet_service.cc:1467] Processing CreateTablet for tablet 5831288451724d4ca1114ec5e5f01b78 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20250109 08:58:19.445197 23365 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5831288451724d4ca1114ec5e5f01b78. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:19.457779 23442 tablet_service.cc:1467] Processing CreateTablet for tablet a09d8dffc2e446f1acd3c7f6cc1705bb (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20250109 08:58:19.459165 23442 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a09d8dffc2e446f1acd3c7f6cc1705bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:19.488142 23289 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4df64aae69ca41dc91286cae1603000f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:19.493256 23367 tablet_service.cc:1467] Processing CreateTablet for tablet a09d8dffc2e446f1acd3c7f6cc1705bb (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20250109 08:58:19.498093 23365 tablet_service.cc:1467] Processing CreateTablet for tablet 08214ee9c4c14bda92bdc6fa8231c313 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20250109 08:58:19.502910 23292 tablet_service.cc:1467] Processing CreateTablet for tablet b572a6a3dfc04ea199bc703ddad8af67 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20250109 08:58:19.504215 23368 tablet_service.cc:1467] Processing CreateTablet for tablet 86d4074509744900abe94402467a7d85 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20250109 08:58:19.505460 23581 raft_consensus.cc:491] T bd6fcd838b0249cabd5f1f95e5538ab6 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:19.489284 23364 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5fa255ab5f24750a0afa6d81ec869f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:19.489105 23514 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bfb140bb3a8b44bd8b4e1fcf0946162b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:19.513324 23361 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a54ca9f0d6e844e9a8b1d9cd0bb2a026. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:19.512964 23291 tablet_service.cc:1467] Processing CreateTablet for tablet 1c5b5beea4a840eb9677097cacee3408 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20250109 08:58:19.514626 23294 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e6b9aaeda9a947d4a87fedd774301cc0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:19.515429 23583 raft_consensus.cc:491] T bd6fcd838b0249cabd5f1f95e5538ab6 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:19.545189 23365 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08214ee9c4c14bda92bdc6fa8231c313. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:19.546054 23292 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b572a6a3dfc04ea199bc703ddad8af67. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:19.515214 23438 tablet_service.cc:1467] Processing CreateTablet for tablet 24ce131f7bb143bea1581d5b0c2a7109 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20250109 08:58:19.549214 23438 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 24ce131f7bb143bea1581d5b0c2a7109. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:19.549520 23291 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c5b5beea4a840eb9677097cacee3408. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:19.545234 23583 raft_consensus.cc:513] T bd6fcd838b0249cabd5f1f95e5538ab6 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
W20250109 08:58:19.520534 23188 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.19.226.196:41829, user_credentials={real_user=slave}} blocked reactor thread for 83858.8us
I20250109 08:58:19.528592 23581 raft_consensus.cc:513] T bd6fcd838b0249cabd5f1f95e5538ab6 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:19.529285 23380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed79047393d2425f8ecff31ea08b9cad" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" is_pre_election: true
I20250109 08:58:19.557392 23581 leader_election.cc:290] T bd6fcd838b0249cabd5f1f95e5538ab6 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829), cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551)
W20250109 08:58:19.530385 23190 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.19.226.195:35551, user_credentials={real_user=slave}} blocked reactor thread for 58016.7us
W20250109 08:58:19.561877 23494 leader_election.cc:343] T ed79047393d2425f8ecff31ea08b9cad P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:19.531113 23443 tablet_service.cc:1467] Processing CreateTablet for tablet db111e41c1be497e8e673a2c42b9ba66 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20250109 08:58:19.562566 23440 tablet_service.cc:1467] Processing CreateTablet for tablet 0dab6c23ce444dda9427901a98f07b15 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20250109 08:58:19.563174 23443 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db111e41c1be497e8e673a2c42b9ba66. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:19.564028 23370 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 940653b6d55c46439827697b9aadf16b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:19.565341 23368 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86d4074509744900abe94402467a7d85. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:19.569629 23574 ts_tablet_manager.cc:1428] T bd6fcd838b0249cabd5f1f95e5538ab6 P 235ad6af04c34c88a5261daa3604fada: Time spent starting tablet: real 0.464s user 0.029s sys 0.044s
I20250109 08:58:19.532184 23369 tablet_service.cc:1467] Processing CreateTablet for tablet a686bf4d947c4b638049e6b88c59ae09 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20250109 08:58:19.533828 23436 tablet_service.cc:1467] Processing CreateTablet for tablet e5251927614642639bd46ecd5373d2a4 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20250109 08:58:19.574278 23574 tablet_bootstrap.cc:492] T 6eef9bb3c645428d9f1cb843bae9a264 P 235ad6af04c34c88a5261daa3604fada: Bootstrap starting.
I20250109 08:58:19.571938 23583 leader_election.cc:290] T bd6fcd838b0249cabd5f1f95e5538ab6 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829), 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691)
I20250109 08:58:19.597774 23440 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0dab6c23ce444dda9427901a98f07b15. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:19.597550 23367 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a09d8dffc2e446f1acd3c7f6cc1705bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:19.610013 23295 tablet_service.cc:1467] Processing CreateTablet for tablet 86d4074509744900abe94402467a7d85 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20250109 08:58:19.611418 23295 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86d4074509744900abe94402467a7d85. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:19.612180 23572 ts_tablet_manager.cc:1428] T bd6fcd838b0249cabd5f1f95e5538ab6 P cf55383bef6649eb9e78cb2d2512dedc: Time spent starting tablet: real 0.508s user 0.013s sys 0.037s
W20250109 08:58:19.612525 23188 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.19.226.196:41829, user_credentials={real_user=slave}} blocked reactor thread for 44788.8us
W20250109 08:58:19.612555 23572 ts_tablet_manager.cc:1460] T bd6fcd838b0249cabd5f1f95e5538ab6 P cf55383bef6649eb9e78cb2d2512dedc: Tablet startup took 508ms
I20250109 08:58:19.621618 23287 tablet_service.cc:1467] Processing CreateTablet for tablet cc9886333c0343798226d59758a1ea5e (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20250109 08:58:19.622846 23287 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc9886333c0343798226d59758a1ea5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:19.632144 23293 tablet_service.cc:1467] Processing CreateTablet for tablet 24ce131f7bb143bea1581d5b0c2a7109 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20250109 08:58:19.633445 23293 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 24ce131f7bb143bea1581d5b0c2a7109. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:19.623021 23290 tablet_service.cc:1467] Processing CreateTablet for tablet a09d8dffc2e446f1acd3c7f6cc1705bb (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20250109 08:58:19.636247 23290 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a09d8dffc2e446f1acd3c7f6cc1705bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:19.636610 23288 tablet_service.cc:1467] Processing CreateTablet for tablet db111e41c1be497e8e673a2c42b9ba66 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20250109 08:58:19.637835 23288 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db111e41c1be497e8e673a2c42b9ba66. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:19.644979 23444 tablet_service.cc:1467] Processing CreateTablet for tablet 732e4aa69aae44e9b1153ac330a01a96 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20250109 08:58:19.647644 23366 tablet_service.cc:1467] Processing CreateTablet for tablet db111e41c1be497e8e673a2c42b9ba66 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20250109 08:58:19.642233 23369 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a686bf4d947c4b638049e6b88c59ae09. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:19.642756 23436 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e5251927614642639bd46ecd5373d2a4. 1 dirs total, 0 dirs full, 0 dirs failed
W20250109 08:58:19.680325 23190 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.19.226.195:35551, user_credentials={real_user=slave}} blocked reactor thread for 50971.5us
I20250109 08:58:19.691601 23574 tablet_bootstrap.cc:654] T 6eef9bb3c645428d9f1cb843bae9a264 P 235ad6af04c34c88a5261daa3604fada: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:19.692893 23594 raft_consensus.cc:491] T 5ccc0d42d4a148d7819366cea0b754a7 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:19.693418 23594 raft_consensus.cc:513] T 5ccc0d42d4a148d7819366cea0b754a7 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:19.702234 23293 tablet_service.cc:1467] Processing CreateTablet for tablet d87d3b4755344b82b0aec7ffd1ce785c (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20250109 08:58:19.703655 23293 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d87d3b4755344b82b0aec7ffd1ce785c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:19.703956 23439 tablet_service.cc:1467] Processing CreateTablet for tablet 5f0d500a91b54851bda52e5ccf741fd6 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20250109 08:58:19.711587 23305 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed79047393d2425f8ecff31ea08b9cad" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada" is_pre_election: true
W20250109 08:58:19.713284 23494 leader_election.cc:343] T ed79047393d2425f8ecff31ea08b9cad P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:19.713734 23494 leader_election.cc:304] T ed79047393d2425f8ecff31ea08b9cad P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01; no voters: 235ad6af04c34c88a5261daa3604fada, ed1efbce9ee3401e99a5260edeae4e7c
I20250109 08:58:19.724915 23289 tablet_service.cc:1467] Processing CreateTablet for tablet 08214ee9c4c14bda92bdc6fa8231c313 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20250109 08:58:19.726302 23289 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08214ee9c4c14bda92bdc6fa8231c313. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:19.727140 23293 tablet_service.cc:1467] Processing CreateTablet for tablet e5251927614642639bd46ecd5373d2a4 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20250109 08:58:19.728514 23293 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e5251927614642639bd46ecd5373d2a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:19.729138 23435 tablet_service.cc:1467] Processing CreateTablet for tablet 11aa76a3a30643708fececed660a5250 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20250109 08:58:19.733085 23528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd6fcd838b0249cabd5f1f95e5538ab6" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" is_pre_election: true
I20250109 08:58:19.733974 23441 tablet_service.cc:1467] Processing CreateTablet for tablet f1f9274aeda748f493f5752814fb262d (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
W20250109 08:58:19.734894 23269 leader_election.cc:343] T bd6fcd838b0249cabd5f1f95e5538ab6 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829): Illegal state: must be running to vote when last-logged opid is not known
W20250109 08:58:19.620888 23572 ts_tablet_manager.cc:1462] T bd6fcd838b0249cabd5f1f95e5538ab6 P cf55383bef6649eb9e78cb2d2512dedc: Trace:
0109 08:58:18.851084 (+ 0us) service_pool.cc:168] Inserting onto call queue
0109 08:58:18.851261 (+ 177us) service_pool.cc:225] Handling call
0109 08:58:18.853189 (+ 1928us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0109 08:58:18.853286 (+ 97us) ts_tablet_manager.cc:676] Creating new metadata...
0109 08:58:18.861958 (+ 8672us) tablet_metadata.cc:641] Metadata flushed
0109 08:58:18.869023 (+ 7065us) tablet_replica.cc:206] Creating consensus instance
0109 08:58:18.992024 (+123001us) inbound_call.cc:173] Queueing success response
0109 08:58:19.031189 (+ 39165us) ts_tablet_manager.cc:1373] Bootstrapping tablet
0109 08:58:19.104475 (+ 73286us) ts_tablet_manager.cc:1429] Starting tablet replica
0109 08:58:19.104921 (+ 446us) tablet_replica.cc:260] Starting instrumentation
0109 08:58:19.105483 (+ 562us) tablet_replica.cc:279] Starting consensus
Metrics: {"raft.queue_time_us":228348,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":111,"tablet-open.queue_time_us":161780,"thread_start_us":311133,"threads_started":2}
I20250109 08:58:19.742570 23572 tablet_bootstrap.cc:492] T 2e3d34c0d09843d2be54e47f3ccc16d8 P cf55383bef6649eb9e78cb2d2512dedc: Bootstrap starting.
W20250109 08:58:19.743367 23188 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.19.226.196:41829, user_credentials={real_user=slave}} blocked reactor thread for 61026.6us
I20250109 08:58:19.620568 23286 tablet_service.cc:1467] Processing CreateTablet for tablet 2ab70546dda143b5b5db15c3875aa348 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20250109 08:58:19.749015 23569 tablet_bootstrap.cc:492] T ea2b0762940e44dbbf69b9a163ace846 P 05a74e1eb2474b3d8b3a2da1ef127e01: No bootstrap required, opened a new log
I20250109 08:58:19.749184 23286 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ab70546dda143b5b5db15c3875aa348. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:19.749563 23569 ts_tablet_manager.cc:1397] T ea2b0762940e44dbbf69b9a163ace846 P 05a74e1eb2474b3d8b3a2da1ef127e01: Time spent bootstrapping tablet: real 0.648s user 0.013s sys 0.005s
I20250109 08:58:19.748157 23594 leader_election.cc:290] T 5ccc0d42d4a148d7819366cea0b754a7 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551), 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691)
I20250109 08:58:19.749163 23293 tablet_service.cc:1467] Processing CreateTablet for tablet 732e4aa69aae44e9b1153ac330a01a96 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20250109 08:58:19.759090 23293 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 732e4aa69aae44e9b1153ac330a01a96. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:19.764617 23444 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 732e4aa69aae44e9b1153ac330a01a96. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:19.768685 23366 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db111e41c1be497e8e673a2c42b9ba66. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:19.769685 23305 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd6fcd838b0249cabd5f1f95e5538ab6" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada" is_pre_election: true
I20250109 08:58:19.770361 23305 raft_consensus.cc:2463] T bd6fcd838b0249cabd5f1f95e5538ab6 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf55383bef6649eb9e78cb2d2512dedc in term 0.
I20250109 08:58:19.771660 23420 leader_election.cc:304] T bd6fcd838b0249cabd5f1f95e5538ab6 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 235ad6af04c34c88a5261daa3604fada, cf55383bef6649eb9e78cb2d2512dedc; no voters:
I20250109 08:58:19.758220 23594 raft_consensus.cc:2743] T ed79047393d2425f8ecff31ea08b9cad P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:58:19.772496 23583 raft_consensus.cc:2798] T bd6fcd838b0249cabd5f1f95e5538ab6 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:58:19.772874 23583 raft_consensus.cc:491] T bd6fcd838b0249cabd5f1f95e5538ab6 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:58:19.773201 23583 raft_consensus.cc:3054] T bd6fcd838b0249cabd5f1f95e5538ab6 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:19.773972 23286 tablet_service.cc:1467] Processing CreateTablet for tablet 86b82e19759c4fd0b471a0dbff839aa5 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20250109 08:58:19.775519 23286 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86b82e19759c4fd0b471a0dbff839aa5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:19.791328 23294 tablet_service.cc:1467] Processing CreateTablet for tablet 5f0d500a91b54851bda52e5ccf741fd6 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20250109 08:58:19.791498 23583 raft_consensus.cc:513] T bd6fcd838b0249cabd5f1f95e5538ab6 P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:19.792419 23569 raft_consensus.cc:357] T ea2b0762940e44dbbf69b9a163ace846 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:19.794502 23438 tablet_service.cc:1467] Processing CreateTablet for tablet de98c70e25494d2a8e581443be836bd3 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20250109 08:58:19.728152 23437 tablet_service.cc:1467] Processing CreateTablet for tablet 750dd1e1055f4570b0aecdda3ebe8772 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
W20250109 08:58:19.812680 23494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.226.193:42691, user_credentials={real_user=slave}} blocked reactor thread for 99443.9us
I20250109 08:58:19.811627 23569 raft_consensus.cc:383] T ea2b0762940e44dbbf69b9a163ace846 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:19.814431 23569 raft_consensus.cc:738] T ea2b0762940e44dbbf69b9a163ace846 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05a74e1eb2474b3d8b3a2da1ef127e01, State: Initialized, Role: FOLLOWER
I20250109 08:58:19.815095 23569 consensus_queue.cc:260] T ea2b0762940e44dbbf69b9a163ace846 P 05a74e1eb2474b3d8b3a2da1ef127e01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:19.816614 23364 tablet_service.cc:1467] Processing CreateTablet for tablet 732e4aa69aae44e9b1153ac330a01a96 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20250109 08:58:19.817567 23569 ts_tablet_manager.cc:1428] T ea2b0762940e44dbbf69b9a163ace846 P 05a74e1eb2474b3d8b3a2da1ef127e01: Time spent starting tablet: real 0.068s user 0.003s sys 0.004s
I20250109 08:58:19.817946 23364 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 732e4aa69aae44e9b1153ac330a01a96. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:19.818773 23363 tablet_service.cc:1467] Processing CreateTablet for tablet f1f9274aeda748f493f5752814fb262d (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20250109 08:58:19.819929 23363 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f1f9274aeda748f493f5752814fb262d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:19.823884 23361 tablet_service.cc:1467] Processing CreateTablet for tablet d87d3b4755344b82b0aec7ffd1ce785c (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20250109 08:58:19.825079 23361 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d87d3b4755344b82b0aec7ffd1ce785c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:19.825964 23569 tablet_bootstrap.cc:492] T 9b0bbf2e4a424fb5ac13a3b55627b048 P 05a74e1eb2474b3d8b3a2da1ef127e01: Bootstrap starting.
I20250109 08:58:19.827469 23294 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5f0d500a91b54851bda52e5ccf741fd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:19.827356 23442 tablet_service.cc:1467] Processing CreateTablet for tablet 94f2be307f704c75bceb213e52307cfb (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20250109 08:58:19.815155 23528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd6fcd838b0249cabd5f1f95e5538ab6" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" is_pre_election: true
I20250109 08:58:19.834955 23305 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd6fcd838b0249cabd5f1f95e5538ab6" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada"
I20250109 08:58:19.829524 23439 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5f0d500a91b54851bda52e5ccf741fd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:19.835587 23305 raft_consensus.cc:3054] T bd6fcd838b0249cabd5f1f95e5538ab6 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:19.837047 23304 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ccc0d42d4a148d7819366cea0b754a7" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada" is_pre_election: true
I20250109 08:58:19.838538 23362 tablet_service.cc:1467] Processing CreateTablet for tablet 86b82e19759c4fd0b471a0dbff839aa5 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20250109 08:58:19.839777 23362 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86b82e19759c4fd0b471a0dbff839aa5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:19.840180 23290 tablet_service.cc:1467] Processing CreateTablet for tablet a7d2ab61d238471c9f287ac1d5db4ed0 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20250109 08:58:19.842875 23580 tablet_bootstrap.cc:654] T 970a82e754db47b6b1a19551d4ba43d8 P ed1efbce9ee3401e99a5260edeae4e7c: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:19.815074 23368 tablet_service.cc:1467] Processing CreateTablet for tablet 750dd1e1055f4570b0aecdda3ebe8772 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20250109 08:58:19.844482 23365 tablet_service.cc:1467] Processing CreateTablet for tablet fc1038255dd64cb5ba2425037d22f1f7 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20250109 08:58:19.831765 23583 leader_election.cc:290] T bd6fcd838b0249cabd5f1f95e5538ab6 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 election: Requested vote from peers 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829), 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691)
I20250109 08:58:19.845486 23291 tablet_service.cc:1467] Processing CreateTablet for tablet f1f9274aeda748f493f5752814fb262d (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
W20250109 08:58:19.851491 23418 leader_election.cc:343] T bd6fcd838b0249cabd5f1f95e5538ab6 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:19.890908 23287 tablet_service.cc:1467] Processing CreateTablet for tablet de98c70e25494d2a8e581443be836bd3 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20250109 08:58:19.890832 23369 tablet_service.cc:1467] Processing CreateTablet for tablet 11aa76a3a30643708fececed660a5250 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20250109 08:58:19.892063 23454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd6fcd838b0249cabd5f1f95e5538ab6" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc" is_pre_election: true
I20250109 08:58:19.891587 23290 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a7d2ab61d238471c9f287ac1d5db4ed0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:19.892974 23454 raft_consensus.cc:2388] T bd6fcd838b0249cabd5f1f95e5538ab6 P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 235ad6af04c34c88a5261daa3604fada in current term 1: Already voted for candidate cf55383bef6649eb9e78cb2d2512dedc in this term.
I20250109 08:58:19.894389 23271 leader_election.cc:304] T bd6fcd838b0249cabd5f1f95e5538ab6 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 235ad6af04c34c88a5261daa3604fada; no voters: 05a74e1eb2474b3d8b3a2da1ef127e01, cf55383bef6649eb9e78cb2d2512dedc
I20250109 08:58:19.895330 23287 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de98c70e25494d2a8e581443be836bd3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:19.852407 23528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd6fcd838b0249cabd5f1f95e5538ab6" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01"
W20250109 08:58:19.852799 23494 leader_election.cc:343] T 5ccc0d42d4a148d7819366cea0b754a7 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Illegal state: must be running to vote when last-logged opid is not known
W20250109 08:58:19.896912 23494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.226.193:42691, user_credentials={real_user=slave}} blocked reactor thread for 44153.5us
I20250109 08:58:19.897713 23291 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f1f9274aeda748f493f5752814fb262d. 1 dirs total, 0 dirs full, 0 dirs failed
W20250109 08:58:19.899492 23418 leader_election.cc:343] T bd6fcd838b0249cabd5f1f95e5538ab6 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:19.827270 23292 tablet_service.cc:1467] Processing CreateTablet for tablet fc1038255dd64cb5ba2425037d22f1f7 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20250109 08:58:19.900899 23292 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc1038255dd64cb5ba2425037d22f1f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:19.853907 23437 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 750dd1e1055f4570b0aecdda3ebe8772. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:19.868569 23443 tablet_service.cc:1467] Processing CreateTablet for tablet a7d2ab61d238471c9f287ac1d5db4ed0 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20250109 08:58:19.903173 23442 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 94f2be307f704c75bceb213e52307cfb. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:19.904130 23443 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a7d2ab61d238471c9f287ac1d5db4ed0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:19.887670 23289 tablet_service.cc:1467] Processing CreateTablet for tablet 94f2be307f704c75bceb213e52307cfb (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20250109 08:58:19.889215 23368 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 750dd1e1055f4570b0aecdda3ebe8772. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:19.906093 23365 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc1038255dd64cb5ba2425037d22f1f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:19.908839 23289 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 94f2be307f704c75bceb213e52307cfb. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:19.920465 23438 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de98c70e25494d2a8e581443be836bd3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:19.924734 23435 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11aa76a3a30643708fececed660a5250. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:19.881282 23569 tablet_bootstrap.cc:654] T 9b0bbf2e4a424fb5ac13a3b55627b048 P 05a74e1eb2474b3d8b3a2da1ef127e01: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:19.878067 23305 raft_consensus.cc:2463] T bd6fcd838b0249cabd5f1f95e5538ab6 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf55383bef6649eb9e78cb2d2512dedc in term 1.
I20250109 08:58:19.939858 23586 raft_consensus.cc:2743] T bd6fcd838b0249cabd5f1f95e5538ab6 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:58:19.941107 23574 tablet_bootstrap.cc:492] T 6eef9bb3c645428d9f1cb843bae9a264 P 235ad6af04c34c88a5261daa3604fada: No bootstrap required, opened a new log
I20250109 08:58:19.941622 23574 ts_tablet_manager.cc:1397] T 6eef9bb3c645428d9f1cb843bae9a264 P 235ad6af04c34c88a5261daa3604fada: Time spent bootstrapping tablet: real 0.371s user 0.012s sys 0.030s
I20250109 08:58:19.942770 23420 leader_election.cc:304] T bd6fcd838b0249cabd5f1f95e5538ab6 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 235ad6af04c34c88a5261daa3604fada, cf55383bef6649eb9e78cb2d2512dedc; no voters: 05a74e1eb2474b3d8b3a2da1ef127e01
I20250109 08:58:19.943962 23583 raft_consensus.cc:2798] T bd6fcd838b0249cabd5f1f95e5538ab6 P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:58:19.944478 23583 raft_consensus.cc:695] T bd6fcd838b0249cabd5f1f95e5538ab6 P cf55383bef6649eb9e78cb2d2512dedc [term 1 LEADER]: Becoming Leader. State: Replica: cf55383bef6649eb9e78cb2d2512dedc, State: Running, Role: LEADER
I20250109 08:58:19.945312 23583 consensus_queue.cc:237] T bd6fcd838b0249cabd5f1f95e5538ab6 P cf55383bef6649eb9e78cb2d2512dedc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:19.904839 23370 tablet_service.cc:1467] Processing CreateTablet for tablet 0dab6c23ce444dda9427901a98f07b15 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20250109 08:58:19.934559 23441 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f1f9274aeda748f493f5752814fb262d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:19.934999 23369 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11aa76a3a30643708fececed660a5250. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:19.932168 23517 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:56864 (request call id 41) took 1016 ms. Trace:
I20250109 08:58:19.949506 23368 tablet_service.cc:1467] Processing CreateTablet for tablet de98c70e25494d2a8e581443be836bd3 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20250109 08:58:19.953738 23574 raft_consensus.cc:357] T 6eef9bb3c645428d9f1cb843bae9a264 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:19.957842 23574 raft_consensus.cc:383] T 6eef9bb3c645428d9f1cb843bae9a264 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:19.964977 23436 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:58512 (request call id 25) took 1009 ms. Trace:
I20250109 08:58:19.966076 23436 rpcz_store.cc:276] 0109 08:58:18.955942 (+ 0us) service_pool.cc:168] Inserting onto call queue
0109 08:58:19.532958 (+577016us) service_pool.cc:225] Handling call
0109 08:58:19.594423 (+ 61465us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0109 08:58:19.600115 (+ 5692us) ts_tablet_manager.cc:676] Creating new metadata...
0109 08:58:19.886941 (+286826us) tablet_metadata.cc:641] Metadata flushed
0109 08:58:19.964485 (+ 77544us) tablet_replica.cc:206] Creating consensus instance
0109 08:58:19.964916 (+ 431us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20250109 08:58:19.958135 23574 raft_consensus.cc:738] T 6eef9bb3c645428d9f1cb843bae9a264 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Initialized, Role: FOLLOWER
I20250109 08:58:19.969444 23574 consensus_queue.cc:260] T 6eef9bb3c645428d9f1cb843bae9a264 P 235ad6af04c34c88a5261daa3604fada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:19.969533 23439 tablet_service.cc:1467] Processing CreateTablet for tablet ecc833ad1afa4079a75e41b1a807c923 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20250109 08:58:19.970500 23444 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:58512 (request call id 27) took 1014 ms. Trace:
I20250109 08:58:19.972272 23436 tablet_service.cc:1467] Processing CreateTablet for tablet 1365130ef97c4581b623c22c39753831 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20250109 08:58:19.980486 23443 tablet_service.cc:1467] Processing CreateTablet for tablet 77597d68423749e7b87f9f0db7a1ff3b (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20250109 08:58:19.981556 23574 ts_tablet_manager.cc:1428] T 6eef9bb3c645428d9f1cb843bae9a264 P 235ad6af04c34c88a5261daa3604fada: Time spent starting tablet: real 0.040s user 0.005s sys 0.002s
I20250109 08:58:19.984149 23440 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:58512 (request call id 26) took 1028 ms. Trace:
I20250109 08:58:19.984583 23440 rpcz_store.cc:276] 0109 08:58:18.956204 (+ 0us) service_pool.cc:168] Inserting onto call queue
0109 08:58:19.561563 (+605359us) service_pool.cc:225] Handling call
0109 08:58:19.563456 (+ 1893us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0109 08:58:19.563550 (+ 94us) ts_tablet_manager.cc:676] Creating new metadata...
0109 08:58:19.818291 (+254741us) tablet_metadata.cc:641] Metadata flushed
0109 08:58:19.983652 (+165361us) tablet_replica.cc:206] Creating consensus instance
0109 08:58:19.984093 (+ 441us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20250109 08:58:19.986419 23440 tablet_service.cc:1467] Processing CreateTablet for tablet 1f9c6525267145d6bb73dde32ed9b7d4 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20250109 08:58:19.970890 23444 rpcz_store.cc:276] 0109 08:58:18.972323 (+ 0us) service_pool.cc:168] Inserting onto call queue
0109 08:58:19.643965 (+671642us) service_pool.cc:225] Handling call
0109 08:58:19.645915 (+ 1950us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0109 08:58:19.646009 (+ 94us) ts_tablet_manager.cc:676] Creating new metadata...
0109 08:58:19.885142 (+239133us) tablet_metadata.cc:641] Metadata flushed
0109 08:58:19.970045 (+ 84903us) tablet_replica.cc:206] Creating consensus instance
0109 08:58:19.970447 (+ 402us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20250109 08:58:19.989780 23444 tablet_service.cc:1467] Processing CreateTablet for tablet d714271c7ea64c0b8b13fcf63591ec07 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20250109 08:58:19.992909 23370 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0dab6c23ce444dda9427901a98f07b15. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:19.993552 23368 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de98c70e25494d2a8e581443be836bd3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:19.994268 23367 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:33278 (request call id 21) took 1058 ms. Trace:
I20250109 08:58:19.994696 23367 rpcz_store.cc:276] 0109 08:58:18.935767 (+ 0us) service_pool.cc:168] Inserting onto call queue
0109 08:58:19.471684 (+535917us) service_pool.cc:225] Handling call
0109 08:58:19.494092 (+ 22408us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0109 08:58:19.494182 (+ 90us) ts_tablet_manager.cc:676] Creating new metadata...
0109 08:58:19.875699 (+381517us) tablet_metadata.cc:641] Metadata flushed
0109 08:58:19.953581 (+ 77882us) tablet_replica.cc:206] Creating consensus instance
0109 08:58:19.994208 (+ 40627us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20250109 08:58:19.996031 23454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ccc0d42d4a148d7819366cea0b754a7" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc" is_pre_election: true
I20250109 08:58:19.995896 23295 tablet_service.cc:1467] Processing CreateTablet for tablet 444507f7812441ceaeebc44bc8f06293 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
W20250109 08:58:19.997613 23494 leader_election.cc:343] T 5ccc0d42d4a148d7819366cea0b754a7 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:19.998055 23494 leader_election.cc:304] T 5ccc0d42d4a148d7819366cea0b754a7 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01; no voters: 235ad6af04c34c88a5261daa3604fada, cf55383bef6649eb9e78cb2d2512dedc
I20250109 08:58:19.999392 23598 raft_consensus.cc:2743] T 5ccc0d42d4a148d7819366cea0b754a7 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:58:20.000833 23365 tablet_service.cc:1467] Processing CreateTablet for tablet ecc833ad1afa4079a75e41b1a807c923 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20250109 08:58:19.951112 23517 rpcz_store.cc:276] 0109 08:58:18.915732 (+ 0us) service_pool.cc:168] Inserting onto call queue
0109 08:58:19.337591 (+421859us) service_pool.cc:225] Handling call
0109 08:58:19.339373 (+ 1782us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0109 08:58:19.339461 (+ 88us) ts_tablet_manager.cc:676] Creating new metadata...
0109 08:58:19.751646 (+412185us) tablet_metadata.cc:641] Metadata flushed
0109 08:58:19.931641 (+179995us) tablet_replica.cc:206] Creating consensus instance
0109 08:58:19.932093 (+ 452us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20250109 08:58:20.006255 23365 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ecc833ad1afa4079a75e41b1a807c923. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:20.009847 23572 tablet_bootstrap.cc:654] T 2e3d34c0d09843d2be54e47f3ccc16d8 P cf55383bef6649eb9e78cb2d2512dedc: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:20.010612 23363 tablet_service.cc:1467] Processing CreateTablet for tablet 1365130ef97c4581b623c22c39753831 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20250109 08:58:20.011911 23363 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1365130ef97c4581b623c22c39753831. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:20.013995 23438 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:58512 (request call id 33) took 1036 ms. Trace:
I20250109 08:58:20.014263 23293 tablet_service.cc:1467] Processing CreateTablet for tablet 1365130ef97c4581b623c22c39753831 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20250109 08:58:20.019292 23367 tablet_service.cc:1467] Processing CreateTablet for tablet 444507f7812441ceaeebc44bc8f06293 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20250109 08:58:20.020605 23367 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 444507f7812441ceaeebc44bc8f06293. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:20.021004 23362 tablet_service.cc:1467] Processing CreateTablet for tablet 060daed1d3334c6188413dd381b82bf9 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20250109 08:58:20.022140 23362 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 060daed1d3334c6188413dd381b82bf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:20.026122 23574 tablet_bootstrap.cc:492] T ea2b0762940e44dbbf69b9a163ace846 P 235ad6af04c34c88a5261daa3604fada: Bootstrap starting.
I20250109 08:58:20.028687 23364 tablet_service.cc:1467] Processing CreateTablet for tablet 1f9c6525267145d6bb73dde32ed9b7d4 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20250109 08:58:20.029896 23364 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f9c6525267145d6bb73dde32ed9b7d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:20.030128 23290 tablet_service.cc:1467] Processing CreateTablet for tablet 060daed1d3334c6188413dd381b82bf9 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20250109 08:58:20.014396 23438 rpcz_store.cc:276] 0109 08:58:18.977145 (+ 0us) service_pool.cc:168] Inserting onto call queue
0109 08:58:19.793512 (+816367us) service_pool.cc:225] Handling call
0109 08:58:19.821629 (+ 28117us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0109 08:58:19.821717 (+ 88us) ts_tablet_manager.cc:676] Creating new metadata...
0109 08:58:19.999838 (+178121us) tablet_metadata.cc:641] Metadata flushed
0109 08:58:20.013516 (+ 13678us) tablet_replica.cc:206] Creating consensus instance
0109 08:58:20.013940 (+ 424us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20250109 08:58:20.034870 23438 tablet_service.cc:1467] Processing CreateTablet for tablet 950e07e9cc794b8ab9ec5bd97d7e8e59 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20250109 08:58:20.038777 23289 tablet_service.cc:1467] Processing CreateTablet for tablet 77597d68423749e7b87f9f0db7a1ff3b (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20250109 08:58:20.040036 23289 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77597d68423749e7b87f9f0db7a1ff3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:20.039333 23202 catalog_manager.cc:5526] T bd6fcd838b0249cabd5f1f95e5538ab6 P cf55383bef6649eb9e78cb2d2512dedc reported cstate change: term changed from 0 to 1, leader changed from <none> to cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195). New cstate: current_term: 1 leader_uuid: "cf55383bef6649eb9e78cb2d2512dedc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } health_report { overall_health: HEALTHY } } }
I20250109 08:58:20.043921 23290 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 060daed1d3334c6188413dd381b82bf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:20.045707 23286 tablet_service.cc:1467] Processing CreateTablet for tablet d714271c7ea64c0b8b13fcf63591ec07 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20250109 08:58:20.046991 23286 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d714271c7ea64c0b8b13fcf63591ec07. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:20.046811 23288 tablet_service.cc:1467] Processing CreateTablet for tablet 29828d9befb845d5bd49a80e018502e3 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20250109 08:58:20.047405 23368 tablet_service.cc:1467] Processing CreateTablet for tablet d714271c7ea64c0b8b13fcf63591ec07 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20250109 08:58:20.048071 23288 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 29828d9befb845d5bd49a80e018502e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:20.040169 23574 tablet_bootstrap.cc:654] T ea2b0762940e44dbbf69b9a163ace846 P 235ad6af04c34c88a5261daa3604fada: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:20.051015 23368 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d714271c7ea64c0b8b13fcf63591ec07. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:20.049551 23439 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ecc833ad1afa4079a75e41b1a807c923. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:20.065883 23438 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 950e07e9cc794b8ab9ec5bd97d7e8e59. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:20.066655 23436 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1365130ef97c4581b623c22c39753831. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:20.067225 23291 tablet_service.cc:1467] Processing CreateTablet for tablet 950e07e9cc794b8ab9ec5bd97d7e8e59 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20250109 08:58:20.068322 23443 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77597d68423749e7b87f9f0db7a1ff3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:20.072703 23440 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f9c6525267145d6bb73dde32ed9b7d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:20.075210 23444 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d714271c7ea64c0b8b13fcf63591ec07. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:20.095829 23569 tablet_bootstrap.cc:492] T 9b0bbf2e4a424fb5ac13a3b55627b048 P 05a74e1eb2474b3d8b3a2da1ef127e01: No bootstrap required, opened a new log
I20250109 08:58:20.096383 23569 ts_tablet_manager.cc:1397] T 9b0bbf2e4a424fb5ac13a3b55627b048 P 05a74e1eb2474b3d8b3a2da1ef127e01: Time spent bootstrapping tablet: real 0.271s user 0.008s sys 0.009s
I20250109 08:58:20.097555 23366 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:33278 (request call id 24) took 1128 ms. Trace:
I20250109 08:58:20.097988 23366 rpcz_store.cc:276] 0109 08:58:18.968869 (+ 0us) service_pool.cc:168] Inserting onto call queue
0109 08:58:19.646749 (+677880us) service_pool.cc:225] Handling call
0109 08:58:19.648535 (+ 1786us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0109 08:58:19.648631 (+ 96us) ts_tablet_manager.cc:676] Creating new metadata...
0109 08:58:19.868264 (+219633us) tablet_metadata.cc:641] Metadata flushed
0109 08:58:20.097045 (+228781us) tablet_replica.cc:206] Creating consensus instance
0109 08:58:20.097495 (+ 450us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20250109 08:58:20.065157 23293 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1365130ef97c4581b623c22c39753831. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:20.101028 23295 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 444507f7812441ceaeebc44bc8f06293. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:20.105783 23291 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 950e07e9cc794b8ab9ec5bd97d7e8e59. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:20.113445 23287 tablet_service.cc:1467] Processing CreateTablet for tablet ee703ea575234c659da04f21ef3cdd70 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20250109 08:58:20.114769 23287 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee703ea575234c659da04f21ef3cdd70. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:20.117462 23292 tablet_service.cc:1467] Processing CreateTablet for tablet 8d3947300cfc4523acdab19fc992f26a (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20250109 08:58:20.118803 23292 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d3947300cfc4523acdab19fc992f26a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:20.100266 23366 tablet_service.cc:1467] Processing CreateTablet for tablet 29828d9befb845d5bd49a80e018502e3 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20250109 08:58:20.147397 23366 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 29828d9befb845d5bd49a80e018502e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:20.149755 23574 tablet_bootstrap.cc:492] T ea2b0762940e44dbbf69b9a163ace846 P 235ad6af04c34c88a5261daa3604fada: No bootstrap required, opened a new log
I20250109 08:58:20.150281 23574 ts_tablet_manager.cc:1397] T ea2b0762940e44dbbf69b9a163ace846 P 235ad6af04c34c88a5261daa3604fada: Time spent bootstrapping tablet: real 0.124s user 0.011s sys 0.011s
W20250109 08:58:20.150900 23251 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20250109 08:58:20.151005 23365 tablet_service.cc:1467] Processing CreateTablet for tablet 1ac4317290b24209a76d2e19e68ee45b (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20250109 08:58:20.152292 23365 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ac4317290b24209a76d2e19e68ee45b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:20.153985 23574 raft_consensus.cc:357] T ea2b0762940e44dbbf69b9a163ace846 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:20.154763 23574 raft_consensus.cc:383] T ea2b0762940e44dbbf69b9a163ace846 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:20.155045 23574 raft_consensus.cc:738] T ea2b0762940e44dbbf69b9a163ace846 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Initialized, Role: FOLLOWER
I20250109 08:58:20.156397 23574 consensus_queue.cc:260] T ea2b0762940e44dbbf69b9a163ace846 P 235ad6af04c34c88a5261daa3604fada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:20.159987 23436 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:58512 (request call id 36) took 1175 ms. Trace:
I20250109 08:58:20.160477 23436 rpcz_store.cc:276] 0109 08:58:18.984821 (+ 0us) service_pool.cc:168] Inserting onto call queue
0109 08:58:19.969211 (+984390us) service_pool.cc:225] Handling call
0109 08:58:19.973140 (+ 3929us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0109 08:58:19.973237 (+ 97us) ts_tablet_manager.cc:676] Creating new metadata...
0109 08:58:20.087852 (+114615us) tablet_metadata.cc:641] Metadata flushed
0109 08:58:20.159451 (+ 71599us) tablet_replica.cc:206] Creating consensus instance
0109 08:58:20.159931 (+ 480us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20250109 08:58:20.141920 23586 raft_consensus.cc:491] T 6eef9bb3c645428d9f1cb843bae9a264 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:20.161614 23586 raft_consensus.cc:513] T 6eef9bb3c645428d9f1cb843bae9a264 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:20.163841 23586 leader_election.cc:290] T 6eef9bb3c645428d9f1cb843bae9a264 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365), cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551)
I20250109 08:58:20.144500 23435 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:58512 (request call id 32) took 1167 ms. Trace:
I20250109 08:58:20.169509 23454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6eef9bb3c645428d9f1cb843bae9a264" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc" is_pre_election: true
W20250109 08:58:20.171067 23271 leader_election.cc:343] T 6eef9bb3c645428d9f1cb843bae9a264 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:20.172937 23368 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:33278 (request call id 39) took 1016 ms. Trace:
I20250109 08:58:20.173388 23368 rpcz_store.cc:276] 0109 08:58:19.156822 (+ 0us) service_pool.cc:168] Inserting onto call queue
0109 08:58:20.046571 (+889749us) service_pool.cc:225] Handling call
0109 08:58:20.048162 (+ 1591us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0109 08:58:20.048263 (+ 101us) ts_tablet_manager.cc:676] Creating new metadata...
0109 08:58:20.105671 (+ 57408us) tablet_metadata.cc:641] Metadata flushed
0109 08:58:20.172392 (+ 66721us) tablet_replica.cc:206] Creating consensus instance
0109 08:58:20.172876 (+ 484us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20250109 08:58:20.174309 23363 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:33278 (request call id 36) took 1018 ms. Trace:
I20250109 08:58:20.174779 23363 rpcz_store.cc:276] 0109 08:58:19.155666 (+ 0us) service_pool.cc:168] Inserting onto call queue
0109 08:58:20.006704 (+851038us) service_pool.cc:225] Handling call
0109 08:58:20.011500 (+ 4796us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0109 08:58:20.011585 (+ 85us) ts_tablet_manager.cc:676] Creating new metadata...
0109 08:58:20.155261 (+143676us) tablet_metadata.cc:641] Metadata flushed
0109 08:58:20.173828 (+ 18567us) tablet_replica.cc:206] Creating consensus instance
0109 08:58:20.174249 (+ 421us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20250109 08:58:20.104691 23569 raft_consensus.cc:357] T 9b0bbf2e4a424fb5ac13a3b55627b048 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:20.180729 23569 raft_consensus.cc:383] T 9b0bbf2e4a424fb5ac13a3b55627b048 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:20.181073 23569 raft_consensus.cc:738] T 9b0bbf2e4a424fb5ac13a3b55627b048 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05a74e1eb2474b3d8b3a2da1ef127e01, State: Initialized, Role: FOLLOWER
I20250109 08:58:20.169917 23572 tablet_bootstrap.cc:492] T 2e3d34c0d09843d2be54e47f3ccc16d8 P cf55383bef6649eb9e78cb2d2512dedc: No bootstrap required, opened a new log
I20250109 08:58:20.183099 23572 ts_tablet_manager.cc:1397] T 2e3d34c0d09843d2be54e47f3ccc16d8 P cf55383bef6649eb9e78cb2d2512dedc: Time spent bootstrapping tablet: real 0.441s user 0.024s sys 0.029s
I20250109 08:58:20.174582 23286 tablet_service.cc:1467] Processing CreateTablet for tablet bfb140bb3a8b44bd8b4e1fcf0946162b (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20250109 08:58:20.185413 23435 rpcz_store.cc:276] 0109 08:58:18.976886 (+ 0us) service_pool.cc:168] Inserting onto call queue
0109 08:58:19.670252 (+693366us) service_pool.cc:225] Handling call
0109 08:58:19.729894 (+ 59642us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0109 08:58:19.729983 (+ 89us) ts_tablet_manager.cc:676] Creating new metadata...
0109 08:58:20.002372 (+272389us) tablet_metadata.cc:641] Metadata flushed
0109 08:58:20.092320 (+ 89948us) tablet_replica.cc:206] Creating consensus instance
0109 08:58:20.144338 (+ 52018us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20250109 08:58:20.189985 23569 consensus_queue.cc:260] T 9b0bbf2e4a424fb5ac13a3b55627b048 P 05a74e1eb2474b3d8b3a2da1ef127e01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:20.191082 23286 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bfb140bb3a8b44bd8b4e1fcf0946162b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:20.192557 23569 ts_tablet_manager.cc:1428] T 9b0bbf2e4a424fb5ac13a3b55627b048 P 05a74e1eb2474b3d8b3a2da1ef127e01: Time spent starting tablet: real 0.096s user 0.006s sys 0.002s
I20250109 08:58:20.194193 23435 tablet_service.cc:1467] Processing CreateTablet for tablet 8d3947300cfc4523acdab19fc992f26a (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20250109 08:58:20.156193 23364 tablet_service.cc:1467] Processing CreateTablet for tablet 8d3947300cfc4523acdab19fc992f26a (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20250109 08:58:20.197484 23364 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d3947300cfc4523acdab19fc992f26a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:20.198076 23569 tablet_bootstrap.cc:492] T bd6fcd838b0249cabd5f1f95e5538ab6 P 05a74e1eb2474b3d8b3a2da1ef127e01: Bootstrap starting.
I20250109 08:58:20.175263 23368 tablet_service.cc:1467] Processing CreateTablet for tablet ee703ea575234c659da04f21ef3cdd70 (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20250109 08:58:20.200479 23368 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee703ea575234c659da04f21ef3cdd70. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:20.204897 23569 tablet_bootstrap.cc:654] T bd6fcd838b0249cabd5f1f95e5538ab6 P 05a74e1eb2474b3d8b3a2da1ef127e01: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:20.197417 23435 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d3947300cfc4523acdab19fc992f26a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:20.215008 23586 raft_consensus.cc:491] T ea2b0762940e44dbbf69b9a163ace846 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:20.215576 23586 raft_consensus.cc:513] T ea2b0762940e44dbbf69b9a163ace846 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:20.226416 23441 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:58512 (request call id 30) took 1250 ms. Trace:
I20250109 08:58:20.226831 23441 rpcz_store.cc:276] 0109 08:58:18.976312 (+ 0us) service_pool.cc:168] Inserting onto call queue
0109 08:58:19.733091 (+756779us) service_pool.cc:225] Handling call
0109 08:58:19.734802 (+ 1711us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0109 08:58:19.734899 (+ 97us) ts_tablet_manager.cc:676] Creating new metadata...
0109 08:58:20.162378 (+427479us) tablet_metadata.cc:641] Metadata flushed
0109 08:58:20.214627 (+ 52249us) tablet_replica.cc:206] Creating consensus instance
0109 08:58:20.226339 (+ 11712us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20250109 08:58:20.227337 23574 ts_tablet_manager.cc:1428] T ea2b0762940e44dbbf69b9a163ace846 P 235ad6af04c34c88a5261daa3604fada: Time spent starting tablet: real 0.077s user 0.028s sys 0.015s
I20250109 08:58:20.228500 23574 tablet_bootstrap.cc:492] T 2e3d34c0d09843d2be54e47f3ccc16d8 P 235ad6af04c34c88a5261daa3604fada: Bootstrap starting.
I20250109 08:58:20.234354 23586 leader_election.cc:290] T ea2b0762940e44dbbf69b9a163ace846 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829), ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365)
I20250109 08:58:20.234239 23528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea2b0762940e44dbbf69b9a163ace846" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" is_pre_election: true
I20250109 08:58:20.235920 23528 raft_consensus.cc:2463] T ea2b0762940e44dbbf69b9a163ace846 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 235ad6af04c34c88a5261daa3604fada in term 0.
I20250109 08:58:20.237082 23269 leader_election.cc:304] T ea2b0762940e44dbbf69b9a163ace846 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01, 235ad6af04c34c88a5261daa3604fada; no voters:
I20250109 08:58:20.237934 23586 raft_consensus.cc:2798] T ea2b0762940e44dbbf69b9a163ace846 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:58:20.238008 23361 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:33278 (request call id 25) took 1268 ms. Trace:
I20250109 08:58:20.238310 23586 raft_consensus.cc:491] T ea2b0762940e44dbbf69b9a163ace846 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:58:20.238718 23586 raft_consensus.cc:3054] T ea2b0762940e44dbbf69b9a163ace846 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:20.225284 23580 tablet_bootstrap.cc:492] T 970a82e754db47b6b1a19551d4ba43d8 P ed1efbce9ee3401e99a5260edeae4e7c: No bootstrap required, opened a new log
I20250109 08:58:20.243805 23580 ts_tablet_manager.cc:1397] T 970a82e754db47b6b1a19551d4ba43d8 P ed1efbce9ee3401e99a5260edeae4e7c: Time spent bootstrapping tablet: real 0.933s user 0.013s sys 0.042s
I20250109 08:58:20.246948 23580 raft_consensus.cc:357] T 970a82e754db47b6b1a19551d4ba43d8 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:20.249743 23365 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:33278 (request call id 41) took 1092 ms. Trace:
I20250109 08:58:20.253099 23369 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:33278 (request call id 32) took 1099 ms. Trace:
I20250109 08:58:20.254139 23436 tablet_service.cc:1467] Processing CreateTablet for tablet 1ac4317290b24209a76d2e19e68ee45b (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20250109 08:58:20.233583 23437 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:58512 (request call id 29) took 1257 ms. Trace:
I20250109 08:58:20.234268 23574 tablet_bootstrap.cc:654] T 2e3d34c0d09843d2be54e47f3ccc16d8 P 235ad6af04c34c88a5261daa3604fada: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:20.256631 23580 raft_consensus.cc:383] T 970a82e754db47b6b1a19551d4ba43d8 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:20.275584 23441 tablet_service.cc:1467] Processing CreateTablet for tablet bfb140bb3a8b44bd8b4e1fcf0946162b (DEFAULT_TABLE table=test-workload [id=045cd56ed4ca4959ac5d7a817c7940c6]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20250109 08:58:20.260681 23365 rpcz_store.cc:276] 0109 08:58:19.157534 (+ 0us) service_pool.cc:168] Inserting onto call queue
0109 08:58:20.150109 (+992575us) service_pool.cc:225] Handling call
0109 08:58:20.151861 (+ 1752us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0109 08:58:20.151950 (+ 89us) ts_tablet_manager.cc:676] Creating new metadata...
0109 08:58:20.159825 (+ 7875us) tablet_metadata.cc:641] Metadata flushed
0109 08:58:20.249192 (+ 89367us) tablet_replica.cc:206] Creating consensus instance
0109 08:58:20.249682 (+ 490us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20250109 08:58:20.279804 23370 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:33278 (request call id 26) took 1310 ms. Trace:
I20250109 08:58:20.280210 23370 rpcz_store.cc:276] 0109 08:58:18.971864 (+ 0us) service_pool.cc:168] Inserting onto call queue
0109 08:58:19.903871 (+932007us) service_pool.cc:225] Handling call
0109 08:58:19.971103 (+ 67232us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0109 08:58:19.992523 (+ 21420us) ts_tablet_manager.cc:676] Creating new metadata...
0109 08:58:20.225367 (+232844us) tablet_metadata.cc:641] Metadata flushed
0109 08:58:20.279305 (+ 53938us) tablet_replica.cc:206] Creating consensus instance
0109 08:58:20.279745 (+ 440us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20250109 08:58:20.282444 23443 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:58512 (request call id 37) took 1297 ms. Trace:
I20250109 08:58:20.282836 23443 rpcz_store.cc:276] 0109 08:58:18.985017 (+ 0us) service_pool.cc:168] Inserting onto call queue
0109 08:58:19.979549 (+994532us) service_pool.cc:225] Handling call
0109 08:58:19.984064 (+ 4515us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0109 08:58:19.984174 (+ 110us) ts_tablet_manager.cc:676] Creating new metadata...
0109 08:58:20.092248 (+108074us) tablet_metadata.cc:641] Metadata flushed
0109 08:58:20.281928 (+189680us) tablet_replica.cc:206] Creating consensus instance
0109 08:58:20.282377 (+ 449us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":64,"spinlock_wait_cycles":18304}
I20250109 08:58:20.283425 23572 raft_consensus.cc:357] T 2e3d34c0d09843d2be54e47f3ccc16d8 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:20.284178 23572 raft_consensus.cc:383] T 2e3d34c0d09843d2be54e47f3ccc16d8 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:20.284480 23572 raft_consensus.cc:738] T 2e3d34c0d09843d2be54e47f3ccc16d8 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf55383bef6649eb9e78cb2d2512dedc, State: Initialized, Role: FOLLOWER
I20250109 08:58:20.285457 23362 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:33278 (request call id 37) took 1129 ms. Trace:
I20250109 08:58:20.285153 23572 consensus_queue.cc:260] T 2e3d34c0d09843d2be54e47f3ccc16d8 P cf55383bef6649eb9e78cb2d2512dedc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:20.270006 23580 raft_consensus.cc:738] T 970a82e754db47b6b1a19551d4ba43d8 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed1efbce9ee3401e99a5260edeae4e7c, State: Initialized, Role: FOLLOWER
I20250109 08:58:20.286669 23444 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:58512 (request call id 39) took 1017 ms. Trace:
I20250109 08:58:20.238577 23361 rpcz_store.cc:276] 0109 08:58:18.969179 (+ 0us) service_pool.cc:168] Inserting onto call queue
0109 08:58:19.813346 (+844167us) service_pool.cc:225] Handling call
0109 08:58:19.824678 (+ 11332us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0109 08:58:19.824762 (+ 84us) ts_tablet_manager.cc:676] Creating new metadata...
0109 08:58:19.978067 (+153305us) tablet_metadata.cc:641] Metadata flushed
0109 08:58:20.237392 (+259325us) tablet_replica.cc:206] Creating consensus instance
0109 08:58:20.237952 (+ 560us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20250109 08:58:20.287431 23362 rpcz_store.cc:276] 0109 08:58:19.156052 (+ 0us) service_pool.cc:168] Inserting onto call queue
0109 08:58:20.020117 (+864065us) service_pool.cc:225] Handling call
0109 08:58:20.021747 (+ 1630us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0109 08:58:20.021827 (+ 80us) ts_tablet_manager.cc:676] Creating new metadata...
0109 08:58:20.205829 (+184002us) tablet_metadata.cc:641] Metadata flushed
0109 08:58:20.273743 (+ 67914us) tablet_replica.cc:206] Creating consensus instance
0109 08:58:20.285388 (+ 11645us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20250109 08:58:20.295445 23436 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ac4317290b24209a76d2e19e68ee45b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:20.261026 23369 rpcz_store.cc:276] 0109 08:58:19.154170 (+ 0us) service_pool.cc:168] Inserting onto call queue
0109 08:58:19.889956 (+735786us) service_pool.cc:225] Handling call
0109 08:58:19.892412 (+ 2456us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0109 08:58:19.892511 (+ 99us) ts_tablet_manager.cc:676] Creating new metadata...
0109 08:58:20.104870 (+212359us) tablet_metadata.cc:641] Metadata flushed
0109 08:58:20.252637 (+147767us) tablet_replica.cc:206] Creating consensus instance
0109 08:58:20.253042 (+ 405us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20250109 08:58:20.264797 23437 rpcz_store.cc:276] 0109 08:58:18.976029 (+ 0us) service_pool.cc:168] Inserting onto call queue
0109 08:58:19.727306 (+751277us) service_pool.cc:225] Handling call
0109 08:58:19.814240 (+ 86934us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0109 08:58:19.814342 (+ 102us) ts_tablet_manager.cc:676] Creating new metadata...
0109 08:58:20.052223 (+237881us) tablet_metadata.cc:641] Metadata flushed
0109 08:58:20.233077 (+180854us) tablet_replica.cc:206] Creating consensus instance
0109 08:58:20.233526 (+ 449us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20250109 08:58:20.313426 23580 consensus_queue.cc:260] T 970a82e754db47b6b1a19551d4ba43d8 P ed1efbce9ee3401e99a5260edeae4e7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:20.316756 23441 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bfb140bb3a8b44bd8b4e1fcf0946162b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:20.325587 23366 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:33278 (request call id 40) took 1168 ms. Trace:
I20250109 08:58:20.326068 23366 rpcz_store.cc:276] 0109 08:58:19.157183 (+ 0us) service_pool.cc:168] Inserting onto call queue
0109 08:58:20.099321 (+942138us) service_pool.cc:225] Handling call
0109 08:58:20.146945 (+ 47624us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0109 08:58:20.147046 (+ 101us) ts_tablet_manager.cc:676] Creating new metadata...
0109 08:58:20.264456 (+117410us) tablet_metadata.cc:641] Metadata flushed
0109 08:58:20.325073 (+ 60617us) tablet_replica.cc:206] Creating consensus instance
0109 08:58:20.325521 (+ 448us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20250109 08:58:20.327412 23367 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:33278 (request call id 34) took 1172 ms. Trace:
I20250109 08:58:20.327836 23367 rpcz_store.cc:276] 0109 08:58:19.154905 (+ 0us) service_pool.cc:168] Inserting onto call queue
0109 08:58:19.995694 (+840789us) service_pool.cc:225] Handling call
0109 08:58:20.020141 (+ 24447us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0109 08:58:20.020239 (+ 98us) ts_tablet_manager.cc:676] Creating new metadata...
0109 08:58:20.225338 (+205099us) tablet_metadata.cc:641] Metadata flushed
0109 08:58:20.324732 (+ 99394us) tablet_replica.cc:206] Creating consensus instance
0109 08:58:20.327341 (+ 2609us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":2119}
I20250109 08:58:20.331049 23364 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:33278 (request call id 42) took 1010 ms. Trace:
I20250109 08:58:20.331508 23364 rpcz_store.cc:276] 0109 08:58:19.321080 (+ 0us) service_pool.cc:168] Inserting onto call queue
0109 08:58:20.155344 (+834264us) service_pool.cc:225] Handling call
0109 08:58:20.196986 (+ 41642us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0109 08:58:20.197085 (+ 99us) ts_tablet_manager.cc:676] Creating new metadata...
0109 08:58:20.241874 (+ 44789us) tablet_metadata.cc:641] Metadata flushed
0109 08:58:20.330494 (+ 88620us) tablet_replica.cc:206] Creating consensus instance
0109 08:58:20.330984 (+ 490us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20250109 08:58:20.333477 23572 ts_tablet_manager.cc:1428] T 2e3d34c0d09843d2be54e47f3ccc16d8 P cf55383bef6649eb9e78cb2d2512dedc: Time spent starting tablet: real 0.150s user 0.008s sys 0.000s
I20250109 08:58:20.295138 23444 rpcz_store.cc:276] 0109 08:58:19.268981 (+ 0us) service_pool.cc:168] Inserting onto call queue
0109 08:58:19.988915 (+719934us) service_pool.cc:225] Handling call
0109 08:58:19.990551 (+ 1636us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0109 08:58:19.990654 (+ 103us) ts_tablet_manager.cc:676] Creating new metadata...
0109 08:58:20.159283 (+168629us) tablet_metadata.cc:641] Metadata flushed
0109 08:58:20.286149 (+126866us) tablet_replica.cc:206] Creating consensus instance
0109 08:58:20.286613 (+ 464us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20250109 08:58:20.353081 23438 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:58512 (request call id 40) took 1083 ms. Trace:
I20250109 08:58:20.353622 23438 rpcz_store.cc:276] 0109 08:58:19.269363 (+ 0us) service_pool.cc:168] Inserting onto call queue
0109 08:58:20.034041 (+764678us) service_pool.cc:225] Handling call
0109 08:58:20.050873 (+ 16832us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0109 08:58:20.050966 (+ 93us) ts_tablet_manager.cc:676] Creating new metadata...
0109 08:58:20.287586 (+236620us) tablet_metadata.cc:641] Metadata flushed
0109 08:58:20.352527 (+ 64941us) tablet_replica.cc:206] Creating consensus instance
0109 08:58:20.353014 (+ 487us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20250109 08:58:20.358067 23440 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:58512 (request call id 38) took 1154 ms. Trace:
I20250109 08:58:20.358626 23440 rpcz_store.cc:276] 0109 08:58:19.254359 (+ 0us) service_pool.cc:168] Inserting onto call queue
0109 08:58:19.985490 (+731131us) service_pool.cc:225] Handling call
0109 08:58:19.987168 (+ 1678us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0109 08:58:19.987257 (+ 89us) ts_tablet_manager.cc:676] Creating new metadata...
0109 08:58:20.237494 (+250237us) tablet_metadata.cc:641] Metadata flushed
0109 08:58:20.357495 (+120001us) tablet_replica.cc:206] Creating consensus instance
0109 08:58:20.357998 (+ 503us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20250109 08:58:20.386269 23380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6eef9bb3c645428d9f1cb843bae9a264" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" is_pre_election: true
I20250109 08:58:20.386641 23379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea2b0762940e44dbbf69b9a163ace846" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" is_pre_election: true
I20250109 08:58:20.362200 23439 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:58512 (request call id 35) took 1377 ms. Trace:
I20250109 08:58:20.334524 23572 tablet_bootstrap.cc:492] T 5ccc0d42d4a148d7819366cea0b754a7 P cf55383bef6649eb9e78cb2d2512dedc: Bootstrap starting.
W20250109 08:58:20.387962 23271 leader_election.cc:343] T 6eef9bb3c645428d9f1cb843bae9a264 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:20.388458 23271 leader_election.cc:304] T 6eef9bb3c645428d9f1cb843bae9a264 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 235ad6af04c34c88a5261daa3604fada; no voters: cf55383bef6649eb9e78cb2d2512dedc, ed1efbce9ee3401e99a5260edeae4e7c
I20250109 08:58:20.367079 23436 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:58512 (request call id 41) took 1097 ms. Trace:
I20250109 08:58:20.374210 23569 tablet_bootstrap.cc:492] T bd6fcd838b0249cabd5f1f95e5538ab6 P 05a74e1eb2474b3d8b3a2da1ef127e01: No bootstrap required, opened a new log
I20250109 08:58:20.388800 23439 rpcz_store.cc:276] 0109 08:58:18.984559 (+ 0us) service_pool.cc:168] Inserting onto call queue
0109 08:58:19.968571 (+984012us) service_pool.cc:225] Handling call
0109 08:58:19.970537 (+ 1966us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0109 08:58:19.970673 (+ 136us) ts_tablet_manager.cc:676] Creating new metadata...
0109 08:58:20.252775 (+282102us) tablet_metadata.cc:641] Metadata flushed
0109 08:58:20.361661 (+108886us) tablet_replica.cc:206] Creating consensus instance
0109 08:58:20.362134 (+ 473us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20250109 08:58:20.389467 23569 ts_tablet_manager.cc:1397] T bd6fcd838b0249cabd5f1f95e5538ab6 P 05a74e1eb2474b3d8b3a2da1ef127e01: Time spent bootstrapping tablet: real 0.192s user 0.011s sys 0.017s
W20250109 08:58:20.389638 23271 leader_election.cc:343] T ea2b0762940e44dbbf69b9a163ace846 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:20.375147 23580 ts_tablet_manager.cc:1428] T 970a82e754db47b6b1a19551d4ba43d8 P ed1efbce9ee3401e99a5260edeae4e7c: Time spent starting tablet: real 0.131s user 0.003s sys 0.005s
I20250109 08:58:20.391033 23580 tablet_bootstrap.cc:492] T 8c5e3d0155d140c4b32f162d3173f5c9 P ed1efbce9ee3401e99a5260edeae4e7c: Bootstrap starting.
I20250109 08:58:20.376153 23574 tablet_bootstrap.cc:492] T 2e3d34c0d09843d2be54e47f3ccc16d8 P 235ad6af04c34c88a5261daa3604fada: No bootstrap required, opened a new log
I20250109 08:58:20.392850 23574 ts_tablet_manager.cc:1397] T 2e3d34c0d09843d2be54e47f3ccc16d8 P 235ad6af04c34c88a5261daa3604fada: Time spent bootstrapping tablet: real 0.165s user 0.045s sys 0.035s
I20250109 08:58:20.395412 23574 raft_consensus.cc:357] T 2e3d34c0d09843d2be54e47f3ccc16d8 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:20.396131 23574 raft_consensus.cc:383] T 2e3d34c0d09843d2be54e47f3ccc16d8 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:20.383464 23442 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:58512 (request call id 31) took 1406 ms. Trace:
I20250109 08:58:20.381752 23586 raft_consensus.cc:513] T ea2b0762940e44dbbf69b9a163ace846 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:20.400931 23442 rpcz_store.cc:276] 0109 08:58:18.976613 (+ 0us) service_pool.cc:168] Inserting onto call queue
0109 08:58:19.781110 (+804497us) service_pool.cc:225] Handling call
0109 08:58:19.845548 (+ 64438us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0109 08:58:19.845667 (+ 119us) ts_tablet_manager.cc:676] Creating new metadata...
0109 08:58:20.206965 (+361298us) tablet_metadata.cc:641] Metadata flushed
0109 08:58:20.383104 (+176139us) tablet_replica.cc:206] Creating consensus instance
0109 08:58:20.383421 (+ 317us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20250109 08:58:20.402149 23574 raft_consensus.cc:738] T 2e3d34c0d09843d2be54e47f3ccc16d8 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Initialized, Role: FOLLOWER
I20250109 08:58:20.402448 23572 tablet_bootstrap.cc:654] T 5ccc0d42d4a148d7819366cea0b754a7 P cf55383bef6649eb9e78cb2d2512dedc: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:20.403023 23574 consensus_queue.cc:260] T 2e3d34c0d09843d2be54e47f3ccc16d8 P 235ad6af04c34c88a5261daa3604fada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:20.403716 23528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea2b0762940e44dbbf69b9a163ace846" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01"
I20250109 08:58:20.404311 23528 raft_consensus.cc:3054] T ea2b0762940e44dbbf69b9a163ace846 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:20.406195 23580 tablet_bootstrap.cc:654] T 8c5e3d0155d140c4b32f162d3173f5c9 P ed1efbce9ee3401e99a5260edeae4e7c: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:20.396134 23583 consensus_queue.cc:1035] T bd6fcd838b0249cabd5f1f95e5538ab6 P cf55383bef6649eb9e78cb2d2512dedc [LEADER]: Connected to new peer: Peer: permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250109 08:58:20.396498 23436 rpcz_store.cc:276] 0109 08:58:19.269712 (+ 0us) service_pool.cc:168] Inserting onto call queue
0109 08:58:20.218046 (+948334us) service_pool.cc:225] Handling call
0109 08:58:20.269443 (+ 51397us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0109 08:58:20.272464 (+ 3021us) ts_tablet_manager.cc:676] Creating new metadata...
0109 08:58:20.359855 (+ 87391us) tablet_metadata.cc:641] Metadata flushed
0109 08:58:20.366590 (+ 6735us) tablet_replica.cc:206] Creating consensus instance
0109 08:58:20.367016 (+ 426us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20250109 08:58:20.397903 23606 raft_consensus.cc:2743] T 6eef9bb3c645428d9f1cb843bae9a264 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:58:20.402272 23586 leader_election.cc:290] T ea2b0762940e44dbbf69b9a163ace846 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 election: Requested vote from peers 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829), ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365)
I20250109 08:58:20.392225 23569 raft_consensus.cc:357] T bd6fcd838b0249cabd5f1f95e5538ab6 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:20.417403 23569 raft_consensus.cc:383] T bd6fcd838b0249cabd5f1f95e5538ab6 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:20.417783 23569 raft_consensus.cc:738] T bd6fcd838b0249cabd5f1f95e5538ab6 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05a74e1eb2474b3d8b3a2da1ef127e01, State: Initialized, Role: FOLLOWER
I20250109 08:58:20.418491 23569 consensus_queue.cc:260] T bd6fcd838b0249cabd5f1f95e5538ab6 P 05a74e1eb2474b3d8b3a2da1ef127e01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:20.427523 23528 raft_consensus.cc:2463] T ea2b0762940e44dbbf69b9a163ace846 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 235ad6af04c34c88a5261daa3604fada in term 1.
I20250109 08:58:20.429050 23569 ts_tablet_manager.cc:1428] T bd6fcd838b0249cabd5f1f95e5538ab6 P 05a74e1eb2474b3d8b3a2da1ef127e01: Time spent starting tablet: real 0.039s user 0.004s sys 0.003s
I20250109 08:58:20.428947 23269 leader_election.cc:304] T ea2b0762940e44dbbf69b9a163ace846 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01, 235ad6af04c34c88a5261daa3604fada; no voters:
I20250109 08:58:20.430087 23586 raft_consensus.cc:2798] T ea2b0762940e44dbbf69b9a163ace846 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:58:20.430605 23586 raft_consensus.cc:695] T ea2b0762940e44dbbf69b9a163ace846 P 235ad6af04c34c88a5261daa3604fada [term 1 LEADER]: Becoming Leader. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Running, Role: LEADER
I20250109 08:58:20.431113 23379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea2b0762940e44dbbf69b9a163ace846" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c"
W20250109 08:58:20.432355 23271 leader_election.cc:343] T ea2b0762940e44dbbf69b9a163ace846 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:20.430235 23569 tablet_bootstrap.cc:492] T 970a82e754db47b6b1a19551d4ba43d8 P 05a74e1eb2474b3d8b3a2da1ef127e01: Bootstrap starting.
I20250109 08:58:20.431474 23586 consensus_queue.cc:237] T ea2b0762940e44dbbf69b9a163ace846 P 235ad6af04c34c88a5261daa3604fada [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:20.439815 23569 tablet_bootstrap.cc:654] T 970a82e754db47b6b1a19551d4ba43d8 P 05a74e1eb2474b3d8b3a2da1ef127e01: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:20.443025 23580 tablet_bootstrap.cc:492] T 8c5e3d0155d140c4b32f162d3173f5c9 P ed1efbce9ee3401e99a5260edeae4e7c: No bootstrap required, opened a new log
I20250109 08:58:20.443585 23580 ts_tablet_manager.cc:1397] T 8c5e3d0155d140c4b32f162d3173f5c9 P ed1efbce9ee3401e99a5260edeae4e7c: Time spent bootstrapping tablet: real 0.053s user 0.024s sys 0.001s
I20250109 08:58:20.446239 23574 ts_tablet_manager.cc:1428] T 2e3d34c0d09843d2be54e47f3ccc16d8 P 235ad6af04c34c88a5261daa3604fada: Time spent starting tablet: real 0.053s user 0.008s sys 0.036s
I20250109 08:58:20.446626 23580 raft_consensus.cc:357] T 8c5e3d0155d140c4b32f162d3173f5c9 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:20.447525 23574 tablet_bootstrap.cc:492] T 41e7f0cc38c4414ab88c92c1cdeb3984 P 235ad6af04c34c88a5261daa3604fada: Bootstrap starting.
I20250109 08:58:20.447467 23580 raft_consensus.cc:383] T 8c5e3d0155d140c4b32f162d3173f5c9 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:20.449738 23572 tablet_bootstrap.cc:492] T 5ccc0d42d4a148d7819366cea0b754a7 P cf55383bef6649eb9e78cb2d2512dedc: No bootstrap required, opened a new log
I20250109 08:58:20.450251 23572 ts_tablet_manager.cc:1397] T 5ccc0d42d4a148d7819366cea0b754a7 P cf55383bef6649eb9e78cb2d2512dedc: Time spent bootstrapping tablet: real 0.116s user 0.019s sys 0.006s
I20250109 08:58:20.453034 23611 raft_consensus.cc:491] T 970a82e754db47b6b1a19551d4ba43d8 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:20.453573 23611 raft_consensus.cc:513] T 970a82e754db47b6b1a19551d4ba43d8 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:20.447988 23580 raft_consensus.cc:738] T 8c5e3d0155d140c4b32f162d3173f5c9 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed1efbce9ee3401e99a5260edeae4e7c, State: Initialized, Role: FOLLOWER
I20250109 08:58:20.456465 23574 tablet_bootstrap.cc:654] T 41e7f0cc38c4414ab88c92c1cdeb3984 P 235ad6af04c34c88a5261daa3604fada: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:20.459192 23611 leader_election.cc:290] T 970a82e754db47b6b1a19551d4ba43d8 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829), 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691)
I20250109 08:58:20.453258 23572 raft_consensus.cc:357] T 5ccc0d42d4a148d7819366cea0b754a7 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:20.463430 23572 raft_consensus.cc:383] T 5ccc0d42d4a148d7819366cea0b754a7 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:20.467201 23569 tablet_bootstrap.cc:492] T 970a82e754db47b6b1a19551d4ba43d8 P 05a74e1eb2474b3d8b3a2da1ef127e01: No bootstrap required, opened a new log
I20250109 08:58:20.468523 23572 raft_consensus.cc:738] T 5ccc0d42d4a148d7819366cea0b754a7 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf55383bef6649eb9e78cb2d2512dedc, State: Initialized, Role: FOLLOWER
I20250109 08:58:20.469928 23569 ts_tablet_manager.cc:1397] T 970a82e754db47b6b1a19551d4ba43d8 P 05a74e1eb2474b3d8b3a2da1ef127e01: Time spent bootstrapping tablet: real 0.040s user 0.018s sys 0.006s
I20250109 08:58:20.465080 23580 consensus_queue.cc:260] T 8c5e3d0155d140c4b32f162d3173f5c9 P ed1efbce9ee3401e99a5260edeae4e7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:20.473904 23574 tablet_bootstrap.cc:492] T 41e7f0cc38c4414ab88c92c1cdeb3984 P 235ad6af04c34c88a5261daa3604fada: No bootstrap required, opened a new log
I20250109 08:58:20.474371 23574 ts_tablet_manager.cc:1397] T 41e7f0cc38c4414ab88c92c1cdeb3984 P 235ad6af04c34c88a5261daa3604fada: Time spent bootstrapping tablet: real 0.027s user 0.011s sys 0.007s
I20250109 08:58:20.474496 23569 raft_consensus.cc:357] T 970a82e754db47b6b1a19551d4ba43d8 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:20.476536 23569 raft_consensus.cc:383] T 970a82e754db47b6b1a19551d4ba43d8 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:20.476833 23569 raft_consensus.cc:738] T 970a82e754db47b6b1a19551d4ba43d8 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05a74e1eb2474b3d8b3a2da1ef127e01, State: Initialized, Role: FOLLOWER
I20250109 08:58:20.477480 23569 consensus_queue.cc:260] T 970a82e754db47b6b1a19551d4ba43d8 P 05a74e1eb2474b3d8b3a2da1ef127e01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:20.480545 23580 ts_tablet_manager.cc:1428] T 8c5e3d0155d140c4b32f162d3173f5c9 P ed1efbce9ee3401e99a5260edeae4e7c: Time spent starting tablet: real 0.037s user 0.008s sys 0.002s
I20250109 08:58:20.481124 23569 ts_tablet_manager.cc:1428] T 970a82e754db47b6b1a19551d4ba43d8 P 05a74e1eb2474b3d8b3a2da1ef127e01: Time spent starting tablet: real 0.011s user 0.003s sys 0.003s
I20250109 08:58:20.478201 23574 raft_consensus.cc:357] T 41e7f0cc38c4414ab88c92c1cdeb3984 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:20.481552 23574 raft_consensus.cc:383] T 41e7f0cc38c4414ab88c92c1cdeb3984 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:20.481841 23574 raft_consensus.cc:738] T 41e7f0cc38c4414ab88c92c1cdeb3984 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Initialized, Role: FOLLOWER
I20250109 08:58:20.482508 23574 consensus_queue.cc:260] T 41e7f0cc38c4414ab88c92c1cdeb3984 P 235ad6af04c34c88a5261daa3604fada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:20.484802 23574 ts_tablet_manager.cc:1428] T 41e7f0cc38c4414ab88c92c1cdeb3984 P 235ad6af04c34c88a5261daa3604fada: Time spent starting tablet: real 0.010s user 0.008s sys 0.000s
I20250109 08:58:20.485759 23574 tablet_bootstrap.cc:492] T 9b245cb3666c421c884cf20161a22831 P 235ad6af04c34c88a5261daa3604fada: Bootstrap starting.
I20250109 08:58:20.488709 23586 raft_consensus.cc:491] T 41e7f0cc38c4414ab88c92c1cdeb3984 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:20.487903 23202 catalog_manager.cc:5526] T ea2b0762940e44dbbf69b9a163ace846 P 235ad6af04c34c88a5261daa3604fada reported cstate change: term changed from 0 to 1, leader changed from <none> to 235ad6af04c34c88a5261daa3604fada (127.19.226.193). New cstate: current_term: 1 leader_uuid: "235ad6af04c34c88a5261daa3604fada" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:20.489171 23586 raft_consensus.cc:513] T 41e7f0cc38c4414ab88c92c1cdeb3984 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:20.491165 23586 leader_election.cc:290] T 41e7f0cc38c4414ab88c92c1cdeb3984 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365), 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829)
I20250109 08:58:20.491137 23580 tablet_bootstrap.cc:492] T 3208a99957554f6db341654856980c8a P ed1efbce9ee3401e99a5260edeae4e7c: Bootstrap starting.
I20250109 08:58:20.491832 23574 tablet_bootstrap.cc:654] T 9b245cb3666c421c884cf20161a22831 P 235ad6af04c34c88a5261daa3604fada: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:20.492707 23572 consensus_queue.cc:260] T 5ccc0d42d4a148d7819366cea0b754a7 P cf55383bef6649eb9e78cb2d2512dedc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:20.497344 23379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41e7f0cc38c4414ab88c92c1cdeb3984" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" is_pre_election: true
I20250109 08:58:20.499874 23580 tablet_bootstrap.cc:654] T 3208a99957554f6db341654856980c8a P ed1efbce9ee3401e99a5260edeae4e7c: Neither blocks nor log segments found. Creating new log.
W20250109 08:58:20.501814 23271 leader_election.cc:343] T 41e7f0cc38c4414ab88c92c1cdeb3984 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:20.504397 23528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41e7f0cc38c4414ab88c92c1cdeb3984" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" is_pre_election: true
W20250109 08:58:20.506273 23269 leader_election.cc:343] T 41e7f0cc38c4414ab88c92c1cdeb3984 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:20.507115 23269 leader_election.cc:304] T 41e7f0cc38c4414ab88c92c1cdeb3984 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 235ad6af04c34c88a5261daa3604fada; no voters: 05a74e1eb2474b3d8b3a2da1ef127e01, ed1efbce9ee3401e99a5260edeae4e7c
I20250109 08:58:20.507997 23586 raft_consensus.cc:2743] T 41e7f0cc38c4414ab88c92c1cdeb3984 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:58:20.488355 23305 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "970a82e754db47b6b1a19551d4ba43d8" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada" is_pre_election: true
W20250109 08:58:20.509452 23346 leader_election.cc:343] T 970a82e754db47b6b1a19551d4ba43d8 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:20.512025 23569 tablet_bootstrap.cc:492] T 34caa5988a734b65a2eb512747d833b7 P 05a74e1eb2474b3d8b3a2da1ef127e01: Bootstrap starting.
I20250109 08:58:20.522842 23598 raft_consensus.cc:491] T bd6fcd838b0249cabd5f1f95e5538ab6 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:20.523361 23598 raft_consensus.cc:513] T bd6fcd838b0249cabd5f1f95e5538ab6 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:20.524516 23572 ts_tablet_manager.cc:1428] T 5ccc0d42d4a148d7819366cea0b754a7 P cf55383bef6649eb9e78cb2d2512dedc: Time spent starting tablet: real 0.074s user 0.007s sys 0.008s
I20250109 08:58:20.525487 23598 leader_election.cc:290] T bd6fcd838b0249cabd5f1f95e5538ab6 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691), cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551)
I20250109 08:58:20.528769 23454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd6fcd838b0249cabd5f1f95e5538ab6" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc" is_pre_election: true
I20250109 08:58:20.528103 23304 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd6fcd838b0249cabd5f1f95e5538ab6" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada" is_pre_election: true
I20250109 08:58:20.532053 23494 leader_election.cc:304] T bd6fcd838b0249cabd5f1f95e5538ab6 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01; no voters: 235ad6af04c34c88a5261daa3604fada, cf55383bef6649eb9e78cb2d2512dedc
I20250109 08:58:20.535348 23572 tablet_bootstrap.cc:492] T 8c5e3d0155d140c4b32f162d3173f5c9 P cf55383bef6649eb9e78cb2d2512dedc: Bootstrap starting.
I20250109 08:58:20.538764 23611 raft_consensus.cc:491] T 8c5e3d0155d140c4b32f162d3173f5c9 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:20.539075 23598 raft_consensus.cc:2743] T bd6fcd838b0249cabd5f1f95e5538ab6 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:58:20.539419 23611 raft_consensus.cc:513] T 8c5e3d0155d140c4b32f162d3173f5c9 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:20.541374 23598 raft_consensus.cc:491] T 970a82e754db47b6b1a19551d4ba43d8 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:20.541726 23611 leader_election.cc:290] T 8c5e3d0155d140c4b32f162d3173f5c9 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551), 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829)
I20250109 08:58:20.543758 23598 raft_consensus.cc:513] T 970a82e754db47b6b1a19551d4ba43d8 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:20.545140 23528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "970a82e754db47b6b1a19551d4ba43d8" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" is_pre_election: true
I20250109 08:58:20.545827 23528 raft_consensus.cc:2463] T 970a82e754db47b6b1a19551d4ba43d8 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed1efbce9ee3401e99a5260edeae4e7c in term 0.
I20250109 08:58:20.548036 23344 leader_election.cc:304] T 970a82e754db47b6b1a19551d4ba43d8 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01, ed1efbce9ee3401e99a5260edeae4e7c; no voters: 235ad6af04c34c88a5261daa3604fada
I20250109 08:58:20.549198 23598 leader_election.cc:290] T 970a82e754db47b6b1a19551d4ba43d8 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691), ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365)
I20250109 08:58:20.549948 23379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "970a82e754db47b6b1a19551d4ba43d8" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" is_pre_election: true
I20250109 08:58:20.550705 23379 raft_consensus.cc:2463] T 970a82e754db47b6b1a19551d4ba43d8 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05a74e1eb2474b3d8b3a2da1ef127e01 in term 0.
I20250109 08:58:20.552824 23528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c5e3d0155d140c4b32f162d3173f5c9" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" is_pre_election: true
I20250109 08:58:20.553382 23627 raft_consensus.cc:491] T 2e3d34c0d09843d2be54e47f3ccc16d8 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:20.551987 23304 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "970a82e754db47b6b1a19551d4ba43d8" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada" is_pre_election: true
I20250109 08:58:20.554543 23586 raft_consensus.cc:491] T 2e3d34c0d09843d2be54e47f3ccc16d8 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:20.554994 23586 raft_consensus.cc:513] T 2e3d34c0d09843d2be54e47f3ccc16d8 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
W20250109 08:58:20.555542 23344 leader_election.cc:343] T 8c5e3d0155d140c4b32f162d3173f5c9 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:20.556860 23586 leader_election.cc:290] T 2e3d34c0d09843d2be54e47f3ccc16d8 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365), cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551)
I20250109 08:58:20.557370 23494 leader_election.cc:304] T 970a82e754db47b6b1a19551d4ba43d8 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01, ed1efbce9ee3401e99a5260edeae4e7c; no voters:
W20250109 08:58:20.558280 23494 leader_election.cc:343] T 970a82e754db47b6b1a19551d4ba43d8 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:20.558535 23379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e3d34c0d09843d2be54e47f3ccc16d8" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" is_pre_election: true
I20250109 08:58:20.559022 23611 raft_consensus.cc:2798] T 970a82e754db47b6b1a19551d4ba43d8 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:58:20.559469 23611 raft_consensus.cc:491] T 970a82e754db47b6b1a19551d4ba43d8 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:58:20.559809 23611 raft_consensus.cc:3054] T 970a82e754db47b6b1a19551d4ba43d8 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:20.553831 23627 raft_consensus.cc:513] T 2e3d34c0d09843d2be54e47f3ccc16d8 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
W20250109 08:58:20.560245 23271 leader_election.cc:343] T 2e3d34c0d09843d2be54e47f3ccc16d8 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:20.563382 23304 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e3d34c0d09843d2be54e47f3ccc16d8" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada" is_pre_election: true
I20250109 08:58:20.564275 23304 raft_consensus.cc:2463] T 2e3d34c0d09843d2be54e47f3ccc16d8 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf55383bef6649eb9e78cb2d2512dedc in term 0.
I20250109 08:58:20.565567 23420 leader_election.cc:304] T 2e3d34c0d09843d2be54e47f3ccc16d8 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 235ad6af04c34c88a5261daa3604fada, cf55383bef6649eb9e78cb2d2512dedc; no voters:
I20250109 08:58:20.566478 23583 raft_consensus.cc:2798] T 2e3d34c0d09843d2be54e47f3ccc16d8 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:58:20.566850 23583 raft_consensus.cc:491] T 2e3d34c0d09843d2be54e47f3ccc16d8 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:58:20.567178 23583 raft_consensus.cc:3054] T 2e3d34c0d09843d2be54e47f3ccc16d8 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:20.570715 23598 raft_consensus.cc:2798] T 970a82e754db47b6b1a19551d4ba43d8 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:58:20.571151 23598 raft_consensus.cc:491] T 970a82e754db47b6b1a19551d4ba43d8 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:58:20.571484 23598 raft_consensus.cc:3054] T 970a82e754db47b6b1a19551d4ba43d8 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:20.574923 23583 raft_consensus.cc:513] T 2e3d34c0d09843d2be54e47f3ccc16d8 P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:20.577731 23631 raft_consensus.cc:491] T 9b0bbf2e4a424fb5ac13a3b55627b048 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:20.578373 23631 raft_consensus.cc:513] T 9b0bbf2e4a424fb5ac13a3b55627b048 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:20.581243 23379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b0bbf2e4a424fb5ac13a3b55627b048" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" is_pre_election: true
W20250109 08:58:20.582684 23494 leader_election.cc:343] T 9b0bbf2e4a424fb5ac13a3b55627b048 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:20.586113 23611 raft_consensus.cc:513] T 970a82e754db47b6b1a19551d4ba43d8 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:20.589744 23627 leader_election.cc:290] T 2e3d34c0d09843d2be54e47f3ccc16d8 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691), ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365)
I20250109 08:58:20.590394 23528 raft_consensus.cc:3054] T bd6fcd838b0249cabd5f1f95e5538ab6 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:20.591503 23454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e3d34c0d09843d2be54e47f3ccc16d8" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc" is_pre_election: true
I20250109 08:58:20.597134 23454 raft_consensus.cc:2388] T 2e3d34c0d09843d2be54e47f3ccc16d8 P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 235ad6af04c34c88a5261daa3604fada in current term 1: Already voted for candidate cf55383bef6649eb9e78cb2d2512dedc in this term.
I20250109 08:58:20.598354 23271 leader_election.cc:304] T 2e3d34c0d09843d2be54e47f3ccc16d8 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 235ad6af04c34c88a5261daa3604fada; no voters: cf55383bef6649eb9e78cb2d2512dedc, ed1efbce9ee3401e99a5260edeae4e7c
I20250109 08:58:20.599202 23586 raft_consensus.cc:3054] T 2e3d34c0d09843d2be54e47f3ccc16d8 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:20.604995 23611 leader_election.cc:290] T 970a82e754db47b6b1a19551d4ba43d8 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 election: Requested vote from peers 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829), 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691)
I20250109 08:58:20.605254 23527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "970a82e754db47b6b1a19551d4ba43d8" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01"
I20250109 08:58:20.596094 23598 raft_consensus.cc:513] T 970a82e754db47b6b1a19551d4ba43d8 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:20.608708 23305 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "970a82e754db47b6b1a19551d4ba43d8" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada"
W20250109 08:58:20.610056 23346 leader_election.cc:343] T 970a82e754db47b6b1a19551d4ba43d8 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:20.610850 23304 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "970a82e754db47b6b1a19551d4ba43d8" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada"
I20250109 08:58:20.611488 23379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "970a82e754db47b6b1a19551d4ba43d8" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c"
I20250109 08:58:20.612226 23379 raft_consensus.cc:2388] T 970a82e754db47b6b1a19551d4ba43d8 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 05a74e1eb2474b3d8b3a2da1ef127e01 in current term 1: Already voted for candidate ed1efbce9ee3401e99a5260edeae4e7c in this term.
W20250109 08:58:20.613530 23494 leader_election.cc:343] T 970a82e754db47b6b1a19551d4ba43d8 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:20.613756 23454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c5e3d0155d140c4b32f162d3173f5c9" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc" is_pre_election: true
I20250109 08:58:20.614020 23494 leader_election.cc:304] T 970a82e754db47b6b1a19551d4ba43d8 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01; no voters: 235ad6af04c34c88a5261daa3604fada, ed1efbce9ee3401e99a5260edeae4e7c
I20250109 08:58:20.614634 23527 raft_consensus.cc:2388] T 970a82e754db47b6b1a19551d4ba43d8 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate ed1efbce9ee3401e99a5260edeae4e7c in current term 1: Already voted for candidate 05a74e1eb2474b3d8b3a2da1ef127e01 in this term.
W20250109 08:58:20.615087 23346 leader_election.cc:343] T 8c5e3d0155d140c4b32f162d3173f5c9 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:20.615607 23346 leader_election.cc:304] T 8c5e3d0155d140c4b32f162d3173f5c9 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ed1efbce9ee3401e99a5260edeae4e7c; no voters: 05a74e1eb2474b3d8b3a2da1ef127e01, cf55383bef6649eb9e78cb2d2512dedc
I20250109 08:58:20.616676 23344 leader_election.cc:304] T 970a82e754db47b6b1a19551d4ba43d8 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ed1efbce9ee3401e99a5260edeae4e7c; no voters: 05a74e1eb2474b3d8b3a2da1ef127e01, 235ad6af04c34c88a5261daa3604fada
I20250109 08:58:20.618599 23635 raft_consensus.cc:2743] T 8c5e3d0155d140c4b32f162d3173f5c9 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:58:20.619547 23635 raft_consensus.cc:2743] T 970a82e754db47b6b1a19551d4ba43d8 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20250109 08:58:20.618571 23586 raft_consensus.cc:2743] T 2e3d34c0d09843d2be54e47f3ccc16d8 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:58:20.623284 23569 tablet_bootstrap.cc:654] T 34caa5988a734b65a2eb512747d833b7 P 05a74e1eb2474b3d8b3a2da1ef127e01: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:20.595597 23572 tablet_bootstrap.cc:654] T 8c5e3d0155d140c4b32f162d3173f5c9 P cf55383bef6649eb9e78cb2d2512dedc: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:20.629817 23583 leader_election.cc:290] T 2e3d34c0d09843d2be54e47f3ccc16d8 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 election: Requested vote from peers 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691), ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365)
I20250109 08:58:20.626531 23598 leader_election.cc:290] T 970a82e754db47b6b1a19551d4ba43d8 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 election: Requested vote from peers 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691), ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365)
I20250109 08:58:20.627635 23574 tablet_bootstrap.cc:492] T 9b245cb3666c421c884cf20161a22831 P 235ad6af04c34c88a5261daa3604fada: No bootstrap required, opened a new log
I20250109 08:58:20.631479 23454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b0bbf2e4a424fb5ac13a3b55627b048" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc" is_pre_election: true
I20250109 08:58:20.632056 23580 tablet_bootstrap.cc:492] T 3208a99957554f6db341654856980c8a P ed1efbce9ee3401e99a5260edeae4e7c: No bootstrap required, opened a new log
I20250109 08:58:20.630493 23304 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e3d34c0d09843d2be54e47f3ccc16d8" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada"
I20250109 08:58:20.632793 23580 ts_tablet_manager.cc:1397] T 3208a99957554f6db341654856980c8a P ed1efbce9ee3401e99a5260edeae4e7c: Time spent bootstrapping tablet: real 0.142s user 0.022s sys 0.017s
W20250109 08:58:20.632932 23494 leader_election.cc:343] T 9b0bbf2e4a424fb5ac13a3b55627b048 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:20.633380 23494 leader_election.cc:304] T 9b0bbf2e4a424fb5ac13a3b55627b048 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01; no voters: cf55383bef6649eb9e78cb2d2512dedc, ed1efbce9ee3401e99a5260edeae4e7c
I20250109 08:58:20.634289 23598 raft_consensus.cc:2743] T 9b0bbf2e4a424fb5ac13a3b55627b048 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:58:20.635576 23580 raft_consensus.cc:357] T 3208a99957554f6db341654856980c8a P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:20.636148 23580 raft_consensus.cc:383] T 3208a99957554f6db341654856980c8a P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:20.636483 23580 raft_consensus.cc:738] T 3208a99957554f6db341654856980c8a P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed1efbce9ee3401e99a5260edeae4e7c, State: Initialized, Role: FOLLOWER
I20250109 08:58:20.637074 23580 consensus_queue.cc:260] T 3208a99957554f6db341654856980c8a P ed1efbce9ee3401e99a5260edeae4e7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:20.630373 23631 leader_election.cc:290] T 9b0bbf2e4a424fb5ac13a3b55627b048 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365), cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551)
I20250109 08:58:20.632009 23574 ts_tablet_manager.cc:1397] T 9b245cb3666c421c884cf20161a22831 P 235ad6af04c34c88a5261daa3604fada: Time spent bootstrapping tablet: real 0.146s user 0.062s sys 0.079s
I20250109 08:58:20.639636 23574 raft_consensus.cc:357] T 9b245cb3666c421c884cf20161a22831 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:20.625972 23634 raft_consensus.cc:2743] T 970a82e754db47b6b1a19551d4ba43d8 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20250109 08:58:20.640630 23574 raft_consensus.cc:383] T 9b245cb3666c421c884cf20161a22831 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:20.640885 23574 raft_consensus.cc:738] T 9b245cb3666c421c884cf20161a22831 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Initialized, Role: FOLLOWER
I20250109 08:58:20.641469 23574 consensus_queue.cc:260] T 9b245cb3666c421c884cf20161a22831 P 235ad6af04c34c88a5261daa3604fada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:20.644021 23580 ts_tablet_manager.cc:1428] T 3208a99957554f6db341654856980c8a P ed1efbce9ee3401e99a5260edeae4e7c: Time spent starting tablet: real 0.011s user 0.005s sys 0.000s
I20250109 08:58:20.644665 23583 consensus_queue.cc:1035] T bd6fcd838b0249cabd5f1f95e5538ab6 P cf55383bef6649eb9e78cb2d2512dedc [LEADER]: Connected to new peer: Peer: permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250109 08:58:20.645506 23580 tablet_bootstrap.cc:492] T 2e3d34c0d09843d2be54e47f3ccc16d8 P ed1efbce9ee3401e99a5260edeae4e7c: Bootstrap starting.
I20250109 08:58:20.647922 23574 ts_tablet_manager.cc:1428] T 9b245cb3666c421c884cf20161a22831 P 235ad6af04c34c88a5261daa3604fada: Time spent starting tablet: real 0.010s user 0.002s sys 0.003s
I20250109 08:58:20.640018 23304 raft_consensus.cc:2463] T 2e3d34c0d09843d2be54e47f3ccc16d8 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf55383bef6649eb9e78cb2d2512dedc in term 1.
I20250109 08:58:20.648705 23379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e3d34c0d09843d2be54e47f3ccc16d8" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" is_pre_election: true
I20250109 08:58:20.649132 23574 tablet_bootstrap.cc:492] T 82ea7dc042dd4d05b260d508f4422c3c P 235ad6af04c34c88a5261daa3604fada: Bootstrap starting.
I20250109 08:58:20.650382 23380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e3d34c0d09843d2be54e47f3ccc16d8" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c"
I20250109 08:58:20.655087 23580 tablet_bootstrap.cc:654] T 2e3d34c0d09843d2be54e47f3ccc16d8 P ed1efbce9ee3401e99a5260edeae4e7c: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:20.657608 23574 tablet_bootstrap.cc:654] T 82ea7dc042dd4d05b260d508f4422c3c P 235ad6af04c34c88a5261daa3604fada: Neither blocks nor log segments found. Creating new log.
W20250109 08:58:20.660005 23420 leader_election.cc:343] T 2e3d34c0d09843d2be54e47f3ccc16d8 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:20.661455 23420 leader_election.cc:304] T 2e3d34c0d09843d2be54e47f3ccc16d8 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 235ad6af04c34c88a5261daa3604fada, cf55383bef6649eb9e78cb2d2512dedc; no voters:
I20250109 08:58:20.662452 23583 raft_consensus.cc:2798] T 2e3d34c0d09843d2be54e47f3ccc16d8 P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:58:20.663004 23583 raft_consensus.cc:695] T 2e3d34c0d09843d2be54e47f3ccc16d8 P cf55383bef6649eb9e78cb2d2512dedc [term 1 LEADER]: Becoming Leader. State: Replica: cf55383bef6649eb9e78cb2d2512dedc, State: Running, Role: LEADER
I20250109 08:58:20.663182 23572 tablet_bootstrap.cc:492] T 8c5e3d0155d140c4b32f162d3173f5c9 P cf55383bef6649eb9e78cb2d2512dedc: No bootstrap required, opened a new log
I20250109 08:58:20.663690 23572 ts_tablet_manager.cc:1397] T 8c5e3d0155d140c4b32f162d3173f5c9 P cf55383bef6649eb9e78cb2d2512dedc: Time spent bootstrapping tablet: real 0.129s user 0.010s sys 0.003s
W20250109 08:58:20.662580 23420 leader_election.cc:343] T 2e3d34c0d09843d2be54e47f3ccc16d8 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:20.664693 23583 consensus_queue.cc:237] T 2e3d34c0d09843d2be54e47f3ccc16d8 P cf55383bef6649eb9e78cb2d2512dedc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:20.666348 23569 tablet_bootstrap.cc:492] T 34caa5988a734b65a2eb512747d833b7 P 05a74e1eb2474b3d8b3a2da1ef127e01: No bootstrap required, opened a new log
I20250109 08:58:20.670943 23569 ts_tablet_manager.cc:1397] T 34caa5988a734b65a2eb512747d833b7 P 05a74e1eb2474b3d8b3a2da1ef127e01: Time spent bootstrapping tablet: real 0.159s user 0.009s sys 0.007s
I20250109 08:58:20.666596 23572 raft_consensus.cc:357] T 8c5e3d0155d140c4b32f162d3173f5c9 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:20.681057 23572 raft_consensus.cc:383] T 8c5e3d0155d140c4b32f162d3173f5c9 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:20.681401 23572 raft_consensus.cc:738] T 8c5e3d0155d140c4b32f162d3173f5c9 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf55383bef6649eb9e78cb2d2512dedc, State: Initialized, Role: FOLLOWER
I20250109 08:58:20.675626 23569 raft_consensus.cc:357] T 34caa5988a734b65a2eb512747d833b7 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:20.682343 23569 raft_consensus.cc:383] T 34caa5988a734b65a2eb512747d833b7 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:20.682672 23569 raft_consensus.cc:738] T 34caa5988a734b65a2eb512747d833b7 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05a74e1eb2474b3d8b3a2da1ef127e01, State: Initialized, Role: FOLLOWER
I20250109 08:58:20.684895 23569 consensus_queue.cc:260] T 34caa5988a734b65a2eb512747d833b7 P 05a74e1eb2474b3d8b3a2da1ef127e01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:20.687486 23569 ts_tablet_manager.cc:1428] T 34caa5988a734b65a2eb512747d833b7 P 05a74e1eb2474b3d8b3a2da1ef127e01: Time spent starting tablet: real 0.016s user 0.003s sys 0.002s
I20250109 08:58:20.684228 23572 consensus_queue.cc:260] T 8c5e3d0155d140c4b32f162d3173f5c9 P cf55383bef6649eb9e78cb2d2512dedc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:20.691241 23580 tablet_bootstrap.cc:492] T 2e3d34c0d09843d2be54e47f3ccc16d8 P ed1efbce9ee3401e99a5260edeae4e7c: No bootstrap required, opened a new log
I20250109 08:58:20.691881 23580 ts_tablet_manager.cc:1397] T 2e3d34c0d09843d2be54e47f3ccc16d8 P ed1efbce9ee3401e99a5260edeae4e7c: Time spent bootstrapping tablet: real 0.047s user 0.022s sys 0.017s
I20250109 08:58:20.693794 23572 ts_tablet_manager.cc:1428] T 8c5e3d0155d140c4b32f162d3173f5c9 P cf55383bef6649eb9e78cb2d2512dedc: Time spent starting tablet: real 0.030s user 0.010s sys 0.000s
I20250109 08:58:20.695070 23572 tablet_bootstrap.cc:492] T 1340cf8ee2cb423f875276137da17e8b P cf55383bef6649eb9e78cb2d2512dedc: Bootstrap starting.
I20250109 08:58:20.694821 23580 raft_consensus.cc:357] T 2e3d34c0d09843d2be54e47f3ccc16d8 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:20.695523 23580 raft_consensus.cc:383] T 2e3d34c0d09843d2be54e47f3ccc16d8 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:20.695740 23580 raft_consensus.cc:738] T 2e3d34c0d09843d2be54e47f3ccc16d8 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed1efbce9ee3401e99a5260edeae4e7c, State: Initialized, Role: FOLLOWER
I20250109 08:58:20.696264 23580 consensus_queue.cc:260] T 2e3d34c0d09843d2be54e47f3ccc16d8 P ed1efbce9ee3401e99a5260edeae4e7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:20.698594 23580 ts_tablet_manager.cc:1428] T 2e3d34c0d09843d2be54e47f3ccc16d8 P ed1efbce9ee3401e99a5260edeae4e7c: Time spent starting tablet: real 0.006s user 0.004s sys 0.004s
I20250109 08:58:20.698680 23569 tablet_bootstrap.cc:492] T 8c5e3d0155d140c4b32f162d3173f5c9 P 05a74e1eb2474b3d8b3a2da1ef127e01: Bootstrap starting.
I20250109 08:58:20.699673 23580 tablet_bootstrap.cc:492] T 9b0bbf2e4a424fb5ac13a3b55627b048 P ed1efbce9ee3401e99a5260edeae4e7c: Bootstrap starting.
I20250109 08:58:20.701647 23631 raft_consensus.cc:491] T 34caa5988a734b65a2eb512747d833b7 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:20.702163 23631 raft_consensus.cc:513] T 34caa5988a734b65a2eb512747d833b7 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:20.702939 23574 tablet_bootstrap.cc:492] T 82ea7dc042dd4d05b260d508f4422c3c P 235ad6af04c34c88a5261daa3604fada: No bootstrap required, opened a new log
I20250109 08:58:20.703469 23574 ts_tablet_manager.cc:1397] T 82ea7dc042dd4d05b260d508f4422c3c P 235ad6af04c34c88a5261daa3604fada: Time spent bootstrapping tablet: real 0.055s user 0.012s sys 0.021s
I20250109 08:58:20.705040 23380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34caa5988a734b65a2eb512747d833b7" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" is_pre_election: true
I20250109 08:58:20.706316 23574 raft_consensus.cc:357] T 82ea7dc042dd4d05b260d508f4422c3c P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:20.706620 23580 tablet_bootstrap.cc:654] T 9b0bbf2e4a424fb5ac13a3b55627b048 P ed1efbce9ee3401e99a5260edeae4e7c: Neither blocks nor log segments found. Creating new log.
W20250109 08:58:20.706727 23494 leader_election.cc:343] T 34caa5988a734b65a2eb512747d833b7 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:20.707100 23574 raft_consensus.cc:383] T 82ea7dc042dd4d05b260d508f4422c3c P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:20.707423 23574 raft_consensus.cc:738] T 82ea7dc042dd4d05b260d508f4422c3c P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Initialized, Role: FOLLOWER
I20250109 08:58:20.708242 23574 consensus_queue.cc:260] T 82ea7dc042dd4d05b260d508f4422c3c P 235ad6af04c34c88a5261daa3604fada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:20.709124 23454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34caa5988a734b65a2eb512747d833b7" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc" is_pre_election: true
I20250109 08:58:20.709870 23572 tablet_bootstrap.cc:654] T 1340cf8ee2cb423f875276137da17e8b P cf55383bef6649eb9e78cb2d2512dedc: Neither blocks nor log segments found. Creating new log.
W20250109 08:58:20.710649 23494 leader_election.cc:343] T 34caa5988a734b65a2eb512747d833b7 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:20.711107 23494 leader_election.cc:304] T 34caa5988a734b65a2eb512747d833b7 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01; no voters: cf55383bef6649eb9e78cb2d2512dedc, ed1efbce9ee3401e99a5260edeae4e7c
I20250109 08:58:20.711899 23634 raft_consensus.cc:2743] T 34caa5988a734b65a2eb512747d833b7 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:58:20.706596 23569 tablet_bootstrap.cc:654] T 8c5e3d0155d140c4b32f162d3173f5c9 P 05a74e1eb2474b3d8b3a2da1ef127e01: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:20.712679 23631 leader_election.cc:290] T 34caa5988a734b65a2eb512747d833b7 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365), cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551)
I20250109 08:58:20.713975 23201 catalog_manager.cc:5526] T 2e3d34c0d09843d2be54e47f3ccc16d8 P cf55383bef6649eb9e78cb2d2512dedc reported cstate change: term changed from 0 to 1, leader changed from <none> to cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195). New cstate: current_term: 1 leader_uuid: "cf55383bef6649eb9e78cb2d2512dedc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } health_report { overall_health: HEALTHY } } }
I20250109 08:58:20.719345 23574 ts_tablet_manager.cc:1428] T 82ea7dc042dd4d05b260d508f4422c3c P 235ad6af04c34c88a5261daa3604fada: Time spent starting tablet: real 0.016s user 0.008s sys 0.000s
I20250109 08:58:20.721576 23611 raft_consensus.cc:491] T 2e3d34c0d09843d2be54e47f3ccc16d8 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:20.722193 23611 raft_consensus.cc:513] T 2e3d34c0d09843d2be54e47f3ccc16d8 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:20.724391 23611 leader_election.cc:290] T 2e3d34c0d09843d2be54e47f3ccc16d8 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691), cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551)
I20250109 08:58:20.727308 23454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e3d34c0d09843d2be54e47f3ccc16d8" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc" is_pre_election: true
I20250109 08:58:20.727947 23304 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e3d34c0d09843d2be54e47f3ccc16d8" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada" is_pre_election: true
I20250109 08:58:20.728880 23304 raft_consensus.cc:2388] T 2e3d34c0d09843d2be54e47f3ccc16d8 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ed1efbce9ee3401e99a5260edeae4e7c in current term 1: Already voted for candidate cf55383bef6649eb9e78cb2d2512dedc in this term.
I20250109 08:58:20.734460 23346 leader_election.cc:304] T 2e3d34c0d09843d2be54e47f3ccc16d8 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ed1efbce9ee3401e99a5260edeae4e7c; no voters: 235ad6af04c34c88a5261daa3604fada, cf55383bef6649eb9e78cb2d2512dedc
I20250109 08:58:20.736953 23574 tablet_bootstrap.cc:492] T 970a82e754db47b6b1a19551d4ba43d8 P 235ad6af04c34c88a5261daa3604fada: Bootstrap starting.
I20250109 08:58:20.740238 23611 raft_consensus.cc:3054] T 2e3d34c0d09843d2be54e47f3ccc16d8 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:20.744678 23574 tablet_bootstrap.cc:654] T 970a82e754db47b6b1a19551d4ba43d8 P 235ad6af04c34c88a5261daa3604fada: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:20.753741 23611 raft_consensus.cc:2743] T 2e3d34c0d09843d2be54e47f3ccc16d8 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:58:20.757844 23572 tablet_bootstrap.cc:492] T 1340cf8ee2cb423f875276137da17e8b P cf55383bef6649eb9e78cb2d2512dedc: No bootstrap required, opened a new log
I20250109 08:58:20.758438 23572 ts_tablet_manager.cc:1397] T 1340cf8ee2cb423f875276137da17e8b P cf55383bef6649eb9e78cb2d2512dedc: Time spent bootstrapping tablet: real 0.064s user 0.007s sys 0.023s
I20250109 08:58:20.759405 23580 tablet_bootstrap.cc:492] T 9b0bbf2e4a424fb5ac13a3b55627b048 P ed1efbce9ee3401e99a5260edeae4e7c: No bootstrap required, opened a new log
I20250109 08:58:20.759405 23574 tablet_bootstrap.cc:492] T 970a82e754db47b6b1a19551d4ba43d8 P 235ad6af04c34c88a5261daa3604fada: No bootstrap required, opened a new log
I20250109 08:58:20.759953 23580 ts_tablet_manager.cc:1397] T 9b0bbf2e4a424fb5ac13a3b55627b048 P ed1efbce9ee3401e99a5260edeae4e7c: Time spent bootstrapping tablet: real 0.060s user 0.030s sys 0.017s
I20250109 08:58:20.760268 23574 ts_tablet_manager.cc:1397] T 970a82e754db47b6b1a19551d4ba43d8 P 235ad6af04c34c88a5261daa3604fada: Time spent bootstrapping tablet: real 0.024s user 0.006s sys 0.008s
I20250109 08:58:20.760753 23569 tablet_bootstrap.cc:492] T 8c5e3d0155d140c4b32f162d3173f5c9 P 05a74e1eb2474b3d8b3a2da1ef127e01: No bootstrap required, opened a new log
I20250109 08:58:20.761204 23569 ts_tablet_manager.cc:1397] T 8c5e3d0155d140c4b32f162d3173f5c9 P 05a74e1eb2474b3d8b3a2da1ef127e01: Time spent bootstrapping tablet: real 0.063s user 0.019s sys 0.012s
I20250109 08:58:20.763027 23580 raft_consensus.cc:357] T 9b0bbf2e4a424fb5ac13a3b55627b048 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:20.763255 23574 raft_consensus.cc:357] T 970a82e754db47b6b1a19551d4ba43d8 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:20.763832 23580 raft_consensus.cc:383] T 9b0bbf2e4a424fb5ac13a3b55627b048 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:20.764155 23574 raft_consensus.cc:383] T 970a82e754db47b6b1a19551d4ba43d8 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:20.764298 23580 raft_consensus.cc:738] T 9b0bbf2e4a424fb5ac13a3b55627b048 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed1efbce9ee3401e99a5260edeae4e7c, State: Initialized, Role: FOLLOWER
I20250109 08:58:20.764684 23574 raft_consensus.cc:738] T 970a82e754db47b6b1a19551d4ba43d8 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Initialized, Role: FOLLOWER
I20250109 08:58:20.765175 23580 consensus_queue.cc:260] T 9b0bbf2e4a424fb5ac13a3b55627b048 P ed1efbce9ee3401e99a5260edeae4e7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:20.765964 23572 raft_consensus.cc:357] T 1340cf8ee2cb423f875276137da17e8b P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:20.766697 23572 raft_consensus.cc:383] T 1340cf8ee2cb423f875276137da17e8b P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:20.766966 23572 raft_consensus.cc:738] T 1340cf8ee2cb423f875276137da17e8b P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf55383bef6649eb9e78cb2d2512dedc, State: Initialized, Role: FOLLOWER
I20250109 08:58:20.768049 23580 ts_tablet_manager.cc:1428] T 9b0bbf2e4a424fb5ac13a3b55627b048 P ed1efbce9ee3401e99a5260edeae4e7c: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20250109 08:58:20.767817 23572 consensus_queue.cc:260] T 1340cf8ee2cb423f875276137da17e8b P cf55383bef6649eb9e78cb2d2512dedc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:20.769094 23580 tablet_bootstrap.cc:492] T 1340cf8ee2cb423f875276137da17e8b P ed1efbce9ee3401e99a5260edeae4e7c: Bootstrap starting.
I20250109 08:58:20.765647 23574 consensus_queue.cc:260] T 970a82e754db47b6b1a19551d4ba43d8 P 235ad6af04c34c88a5261daa3604fada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:20.770630 23572 ts_tablet_manager.cc:1428] T 1340cf8ee2cb423f875276137da17e8b P cf55383bef6649eb9e78cb2d2512dedc: Time spent starting tablet: real 0.012s user 0.002s sys 0.005s
I20250109 08:58:20.763859 23569 raft_consensus.cc:357] T 8c5e3d0155d140c4b32f162d3173f5c9 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:20.771528 23574 ts_tablet_manager.cc:1428] T 970a82e754db47b6b1a19551d4ba43d8 P 235ad6af04c34c88a5261daa3604fada: Time spent starting tablet: real 0.011s user 0.006s sys 0.000s
I20250109 08:58:20.771752 23569 raft_consensus.cc:383] T 8c5e3d0155d140c4b32f162d3173f5c9 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:20.772099 23569 raft_consensus.cc:738] T 8c5e3d0155d140c4b32f162d3173f5c9 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05a74e1eb2474b3d8b3a2da1ef127e01, State: Initialized, Role: FOLLOWER
I20250109 08:58:20.773000 23574 tablet_bootstrap.cc:492] T 1340cf8ee2cb423f875276137da17e8b P 235ad6af04c34c88a5261daa3604fada: Bootstrap starting.
I20250109 08:58:20.775605 23580 tablet_bootstrap.cc:654] T 1340cf8ee2cb423f875276137da17e8b P ed1efbce9ee3401e99a5260edeae4e7c: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:20.772956 23569 consensus_queue.cc:260] T 8c5e3d0155d140c4b32f162d3173f5c9 P 05a74e1eb2474b3d8b3a2da1ef127e01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:20.777999 23569 ts_tablet_manager.cc:1428] T 8c5e3d0155d140c4b32f162d3173f5c9 P 05a74e1eb2474b3d8b3a2da1ef127e01: Time spent starting tablet: real 0.017s user 0.006s sys 0.000s
I20250109 08:58:20.778908 23569 tablet_bootstrap.cc:492] T 82ea7dc042dd4d05b260d508f4422c3c P 05a74e1eb2474b3d8b3a2da1ef127e01: Bootstrap starting.
I20250109 08:58:20.778944 23574 tablet_bootstrap.cc:654] T 1340cf8ee2cb423f875276137da17e8b P 235ad6af04c34c88a5261daa3604fada: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:20.775703 23572 tablet_bootstrap.cc:492] T 34caa5988a734b65a2eb512747d833b7 P cf55383bef6649eb9e78cb2d2512dedc: Bootstrap starting.
I20250109 08:58:20.787385 23572 tablet_bootstrap.cc:654] T 34caa5988a734b65a2eb512747d833b7 P cf55383bef6649eb9e78cb2d2512dedc: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:20.792874 23569 tablet_bootstrap.cc:654] T 82ea7dc042dd4d05b260d508f4422c3c P 05a74e1eb2474b3d8b3a2da1ef127e01: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:20.796440 23574 tablet_bootstrap.cc:492] T 1340cf8ee2cb423f875276137da17e8b P 235ad6af04c34c88a5261daa3604fada: No bootstrap required, opened a new log
I20250109 08:58:20.797031 23574 ts_tablet_manager.cc:1397] T 1340cf8ee2cb423f875276137da17e8b P 235ad6af04c34c88a5261daa3604fada: Time spent bootstrapping tablet: real 0.024s user 0.016s sys 0.000s
I20250109 08:58:20.799826 23574 raft_consensus.cc:357] T 1340cf8ee2cb423f875276137da17e8b P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:20.800277 23580 tablet_bootstrap.cc:492] T 1340cf8ee2cb423f875276137da17e8b P ed1efbce9ee3401e99a5260edeae4e7c: No bootstrap required, opened a new log
I20250109 08:58:20.800753 23574 raft_consensus.cc:383] T 1340cf8ee2cb423f875276137da17e8b P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:20.801169 23574 raft_consensus.cc:738] T 1340cf8ee2cb423f875276137da17e8b P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Initialized, Role: FOLLOWER
I20250109 08:58:20.801966 23574 consensus_queue.cc:260] T 1340cf8ee2cb423f875276137da17e8b P 235ad6af04c34c88a5261daa3604fada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:20.804203 23574 ts_tablet_manager.cc:1428] T 1340cf8ee2cb423f875276137da17e8b P 235ad6af04c34c88a5261daa3604fada: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20250109 08:58:20.804234 23580 ts_tablet_manager.cc:1397] T 1340cf8ee2cb423f875276137da17e8b P ed1efbce9ee3401e99a5260edeae4e7c: Time spent bootstrapping tablet: real 0.035s user 0.021s sys 0.010s
I20250109 08:58:20.805859 23574 tablet_bootstrap.cc:492] T 3208a99957554f6db341654856980c8a P 235ad6af04c34c88a5261daa3604fada: Bootstrap starting.
I20250109 08:58:20.807756 23569 tablet_bootstrap.cc:492] T 82ea7dc042dd4d05b260d508f4422c3c P 05a74e1eb2474b3d8b3a2da1ef127e01: No bootstrap required, opened a new log
I20250109 08:58:20.808241 23569 ts_tablet_manager.cc:1397] T 82ea7dc042dd4d05b260d508f4422c3c P 05a74e1eb2474b3d8b3a2da1ef127e01: Time spent bootstrapping tablet: real 0.030s user 0.010s sys 0.004s
I20250109 08:58:20.807884 23580 raft_consensus.cc:357] T 1340cf8ee2cb423f875276137da17e8b P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:20.808737 23580 raft_consensus.cc:383] T 1340cf8ee2cb423f875276137da17e8b P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:20.809047 23580 raft_consensus.cc:738] T 1340cf8ee2cb423f875276137da17e8b P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed1efbce9ee3401e99a5260edeae4e7c, State: Initialized, Role: FOLLOWER
I20250109 08:58:20.809718 23580 consensus_queue.cc:260] T 1340cf8ee2cb423f875276137da17e8b P ed1efbce9ee3401e99a5260edeae4e7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:20.812280 23574 tablet_bootstrap.cc:654] T 3208a99957554f6db341654856980c8a P 235ad6af04c34c88a5261daa3604fada: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:20.812292 23580 ts_tablet_manager.cc:1428] T 1340cf8ee2cb423f875276137da17e8b P ed1efbce9ee3401e99a5260edeae4e7c: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20250109 08:58:20.813638 23580 tablet_bootstrap.cc:492] T ed79047393d2425f8ecff31ea08b9cad P ed1efbce9ee3401e99a5260edeae4e7c: Bootstrap starting.
I20250109 08:58:20.819815 23580 tablet_bootstrap.cc:654] T ed79047393d2425f8ecff31ea08b9cad P ed1efbce9ee3401e99a5260edeae4e7c: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:20.822018 23569 raft_consensus.cc:357] T 82ea7dc042dd4d05b260d508f4422c3c P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:20.822824 23569 raft_consensus.cc:383] T 82ea7dc042dd4d05b260d508f4422c3c P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:20.823101 23569 raft_consensus.cc:738] T 82ea7dc042dd4d05b260d508f4422c3c P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05a74e1eb2474b3d8b3a2da1ef127e01, State: Initialized, Role: FOLLOWER
I20250109 08:58:20.825701 23583 raft_consensus.cc:491] T 8c5e3d0155d140c4b32f162d3173f5c9 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:20.826299 23583 raft_consensus.cc:513] T 8c5e3d0155d140c4b32f162d3173f5c9 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:20.828560 23583 leader_election.cc:290] T 8c5e3d0155d140c4b32f162d3173f5c9 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365), 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829)
I20250109 08:58:20.829519 23572 tablet_bootstrap.cc:492] T 34caa5988a734b65a2eb512747d833b7 P cf55383bef6649eb9e78cb2d2512dedc: No bootstrap required, opened a new log
I20250109 08:58:20.830050 23528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c5e3d0155d140c4b32f162d3173f5c9" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" is_pre_election: true
I20250109 08:58:20.830428 23569 consensus_queue.cc:260] T 82ea7dc042dd4d05b260d508f4422c3c P 05a74e1eb2474b3d8b3a2da1ef127e01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:20.830956 23528 raft_consensus.cc:2463] T 8c5e3d0155d140c4b32f162d3173f5c9 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf55383bef6649eb9e78cb2d2512dedc in term 0.
I20250109 08:58:20.833218 23380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c5e3d0155d140c4b32f162d3173f5c9" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" is_pre_election: true
I20250109 08:58:20.833511 23418 leader_election.cc:304] T 8c5e3d0155d140c4b32f162d3173f5c9 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01, cf55383bef6649eb9e78cb2d2512dedc; no voters:
I20250109 08:58:20.833958 23380 raft_consensus.cc:2463] T 8c5e3d0155d140c4b32f162d3173f5c9 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf55383bef6649eb9e78cb2d2512dedc in term 0.
I20250109 08:58:20.830113 23572 ts_tablet_manager.cc:1397] T 34caa5988a734b65a2eb512747d833b7 P cf55383bef6649eb9e78cb2d2512dedc: Time spent bootstrapping tablet: real 0.055s user 0.008s sys 0.006s
I20250109 08:58:20.834734 23583 raft_consensus.cc:2798] T 8c5e3d0155d140c4b32f162d3173f5c9 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:58:20.835130 23583 raft_consensus.cc:491] T 8c5e3d0155d140c4b32f162d3173f5c9 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:58:20.835538 23583 raft_consensus.cc:3054] T 8c5e3d0155d140c4b32f162d3173f5c9 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:20.837749 23572 raft_consensus.cc:357] T 34caa5988a734b65a2eb512747d833b7 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:20.838601 23572 raft_consensus.cc:383] T 34caa5988a734b65a2eb512747d833b7 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:20.838903 23572 raft_consensus.cc:738] T 34caa5988a734b65a2eb512747d833b7 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf55383bef6649eb9e78cb2d2512dedc, State: Initialized, Role: FOLLOWER
I20250109 08:58:20.839092 23574 tablet_bootstrap.cc:492] T 3208a99957554f6db341654856980c8a P 235ad6af04c34c88a5261daa3604fada: No bootstrap required, opened a new log
I20250109 08:58:20.839699 23574 ts_tablet_manager.cc:1397] T 3208a99957554f6db341654856980c8a P 235ad6af04c34c88a5261daa3604fada: Time spent bootstrapping tablet: real 0.034s user 0.012s sys 0.016s
I20250109 08:58:20.840175 23580 tablet_bootstrap.cc:492] T ed79047393d2425f8ecff31ea08b9cad P ed1efbce9ee3401e99a5260edeae4e7c: No bootstrap required, opened a new log
I20250109 08:58:20.839650 23572 consensus_queue.cc:260] T 34caa5988a734b65a2eb512747d833b7 P cf55383bef6649eb9e78cb2d2512dedc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:20.840720 23580 ts_tablet_manager.cc:1397] T ed79047393d2425f8ecff31ea08b9cad P ed1efbce9ee3401e99a5260edeae4e7c: Time spent bootstrapping tablet: real 0.027s user 0.014s sys 0.011s
I20250109 08:58:20.842207 23572 ts_tablet_manager.cc:1428] T 34caa5988a734b65a2eb512747d833b7 P cf55383bef6649eb9e78cb2d2512dedc: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20250109 08:58:20.843544 23580 raft_consensus.cc:357] T ed79047393d2425f8ecff31ea08b9cad P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:20.844331 23580 raft_consensus.cc:383] T ed79047393d2425f8ecff31ea08b9cad P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:20.844712 23580 raft_consensus.cc:738] T ed79047393d2425f8ecff31ea08b9cad P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed1efbce9ee3401e99a5260edeae4e7c, State: Initialized, Role: FOLLOWER
I20250109 08:58:20.845520 23574 raft_consensus.cc:357] T 3208a99957554f6db341654856980c8a P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:20.845468 23580 consensus_queue.cc:260] T ed79047393d2425f8ecff31ea08b9cad P ed1efbce9ee3401e99a5260edeae4e7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:20.846462 23574 raft_consensus.cc:383] T 3208a99957554f6db341654856980c8a P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:20.846812 23574 raft_consensus.cc:738] T 3208a99957554f6db341654856980c8a P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Initialized, Role: FOLLOWER
I20250109 08:58:20.847549 23574 consensus_queue.cc:260] T 3208a99957554f6db341654856980c8a P 235ad6af04c34c88a5261daa3604fada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:20.847833 23583 raft_consensus.cc:513] T 8c5e3d0155d140c4b32f162d3173f5c9 P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:20.848414 23580 ts_tablet_manager.cc:1428] T ed79047393d2425f8ecff31ea08b9cad P ed1efbce9ee3401e99a5260edeae4e7c: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20250109 08:58:20.849829 23580 tablet_bootstrap.cc:492] T 4df64aae69ca41dc91286cae1603000f P ed1efbce9ee3401e99a5260edeae4e7c: Bootstrap starting.
I20250109 08:58:20.850096 23586 raft_consensus.cc:491] T 1340cf8ee2cb423f875276137da17e8b P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:20.850579 23586 raft_consensus.cc:513] T 1340cf8ee2cb423f875276137da17e8b P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:20.851302 23569 ts_tablet_manager.cc:1428] T 82ea7dc042dd4d05b260d508f4422c3c P 05a74e1eb2474b3d8b3a2da1ef127e01: Time spent starting tablet: real 0.043s user 0.005s sys 0.001s
I20250109 08:58:20.849822 23583 leader_election.cc:290] T 8c5e3d0155d140c4b32f162d3173f5c9 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 election: Requested vote from peers ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365), 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829)
I20250109 08:58:20.853673 23379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1340cf8ee2cb423f875276137da17e8b" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" is_pre_election: true
I20250109 08:58:20.854334 23379 raft_consensus.cc:2463] T 1340cf8ee2cb423f875276137da17e8b P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 235ad6af04c34c88a5261daa3604fada in term 0.
I20250109 08:58:20.854712 23380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c5e3d0155d140c4b32f162d3173f5c9" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c"
I20250109 08:58:20.855376 23380 raft_consensus.cc:3054] T 8c5e3d0155d140c4b32f162d3173f5c9 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:20.855643 23271 leader_election.cc:304] T 1340cf8ee2cb423f875276137da17e8b P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 235ad6af04c34c88a5261daa3604fada, ed1efbce9ee3401e99a5260edeae4e7c; no voters:
I20250109 08:58:20.856683 23614 raft_consensus.cc:2798] T 1340cf8ee2cb423f875276137da17e8b P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:58:20.857043 23614 raft_consensus.cc:491] T 1340cf8ee2cb423f875276137da17e8b P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:58:20.857347 23614 raft_consensus.cc:3054] T 1340cf8ee2cb423f875276137da17e8b P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:20.857834 23528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c5e3d0155d140c4b32f162d3173f5c9" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01"
I20250109 08:58:20.858489 23528 raft_consensus.cc:3054] T 8c5e3d0155d140c4b32f162d3173f5c9 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:20.859742 23631 raft_consensus.cc:491] T 82ea7dc042dd4d05b260d508f4422c3c P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:20.860258 23631 raft_consensus.cc:513] T 82ea7dc042dd4d05b260d508f4422c3c P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:20.861955 23606 consensus_queue.cc:1035] T ea2b0762940e44dbbf69b9a163ace846 P 235ad6af04c34c88a5261daa3604fada [LEADER]: Connected to new peer: Peer: permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250109 08:58:20.862277 23631 leader_election.cc:290] T 82ea7dc042dd4d05b260d508f4422c3c P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551), 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691)
I20250109 08:58:20.864506 23454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82ea7dc042dd4d05b260d508f4422c3c" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc" is_pre_election: true
I20250109 08:58:20.865563 23586 leader_election.cc:290] T 1340cf8ee2cb423f875276137da17e8b P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365), cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551)
I20250109 08:58:20.866837 23304 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82ea7dc042dd4d05b260d508f4422c3c" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada" is_pre_election: true
I20250109 08:58:20.867537 23304 raft_consensus.cc:2463] T 82ea7dc042dd4d05b260d508f4422c3c P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05a74e1eb2474b3d8b3a2da1ef127e01 in term 0.
I20250109 08:58:20.868688 23611 raft_consensus.cc:491] T ed79047393d2425f8ecff31ea08b9cad P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:20.869172 23453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1340cf8ee2cb423f875276137da17e8b" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc" is_pre_election: true
I20250109 08:58:20.869501 23580 tablet_bootstrap.cc:654] T 4df64aae69ca41dc91286cae1603000f P ed1efbce9ee3401e99a5260edeae4e7c: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:20.869187 23611 raft_consensus.cc:513] T ed79047393d2425f8ecff31ea08b9cad P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:20.871206 23494 leader_election.cc:304] T 82ea7dc042dd4d05b260d508f4422c3c P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01, 235ad6af04c34c88a5261daa3604fada; no voters:
I20250109 08:58:20.871527 23569 tablet_bootstrap.cc:492] T 41e7f0cc38c4414ab88c92c1cdeb3984 P 05a74e1eb2474b3d8b3a2da1ef127e01: Bootstrap starting.
I20250109 08:58:20.871976 23453 raft_consensus.cc:2463] T 1340cf8ee2cb423f875276137da17e8b P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 235ad6af04c34c88a5261daa3604fada in term 0.
I20250109 08:58:20.877192 23526 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed79047393d2425f8ecff31ea08b9cad" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" is_pre_election: true
I20250109 08:58:20.877946 23526 raft_consensus.cc:2463] T ed79047393d2425f8ecff31ea08b9cad P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed1efbce9ee3401e99a5260edeae4e7c in term 0.
I20250109 08:58:20.878232 23569 tablet_bootstrap.cc:654] T 41e7f0cc38c4414ab88c92c1cdeb3984 P 05a74e1eb2474b3d8b3a2da1ef127e01: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:20.879089 23344 leader_election.cc:304] T ed79047393d2425f8ecff31ea08b9cad P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01, ed1efbce9ee3401e99a5260edeae4e7c; no voters:
I20250109 08:58:20.880004 23635 raft_consensus.cc:2798] T ed79047393d2425f8ecff31ea08b9cad P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:58:20.880359 23635 raft_consensus.cc:491] T ed79047393d2425f8ecff31ea08b9cad P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:58:20.880720 23635 raft_consensus.cc:3054] T ed79047393d2425f8ecff31ea08b9cad P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:20.881000 23580 tablet_bootstrap.cc:492] T 4df64aae69ca41dc91286cae1603000f P ed1efbce9ee3401e99a5260edeae4e7c: No bootstrap required, opened a new log
I20250109 08:58:20.881474 23580 ts_tablet_manager.cc:1397] T 4df64aae69ca41dc91286cae1603000f P ed1efbce9ee3401e99a5260edeae4e7c: Time spent bootstrapping tablet: real 0.032s user 0.011s sys 0.019s
I20250109 08:58:20.871634 23611 leader_election.cc:290] T ed79047393d2425f8ecff31ea08b9cad P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829), 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691)
I20250109 08:58:20.884069 23580 raft_consensus.cc:357] T 4df64aae69ca41dc91286cae1603000f P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:20.885926 23580 raft_consensus.cc:383] T 4df64aae69ca41dc91286cae1603000f P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:20.879016 23304 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed79047393d2425f8ecff31ea08b9cad" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada" is_pre_election: true
I20250109 08:58:20.886212 23580 raft_consensus.cc:738] T 4df64aae69ca41dc91286cae1603000f P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed1efbce9ee3401e99a5260edeae4e7c, State: Initialized, Role: FOLLOWER
I20250109 08:58:20.887002 23580 consensus_queue.cc:260] T 4df64aae69ca41dc91286cae1603000f P ed1efbce9ee3401e99a5260edeae4e7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:20.888562 23631 raft_consensus.cc:2798] T 82ea7dc042dd4d05b260d508f4422c3c P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20250109 08:58:20.888762 23346 leader_election.cc:343] T ed79047393d2425f8ecff31ea08b9cad P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:20.889079 23631 raft_consensus.cc:491] T 82ea7dc042dd4d05b260d508f4422c3c P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:58:20.889328 23580 ts_tablet_manager.cc:1428] T 4df64aae69ca41dc91286cae1603000f P ed1efbce9ee3401e99a5260edeae4e7c: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20250109 08:58:20.890249 23580 tablet_bootstrap.cc:492] T 6eef9bb3c645428d9f1cb843bae9a264 P ed1efbce9ee3401e99a5260edeae4e7c: Bootstrap starting.
I20250109 08:58:20.892558 23631 raft_consensus.cc:3054] T 82ea7dc042dd4d05b260d508f4422c3c P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:20.894899 23380 raft_consensus.cc:2463] T 8c5e3d0155d140c4b32f162d3173f5c9 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf55383bef6649eb9e78cb2d2512dedc in term 1.
I20250109 08:58:20.895473 23580 tablet_bootstrap.cc:654] T 6eef9bb3c645428d9f1cb843bae9a264 P ed1efbce9ee3401e99a5260edeae4e7c: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:20.897300 23528 raft_consensus.cc:2463] T 8c5e3d0155d140c4b32f162d3173f5c9 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf55383bef6649eb9e78cb2d2512dedc in term 1.
I20250109 08:58:20.898931 23420 leader_election.cc:304] T 8c5e3d0155d140c4b32f162d3173f5c9 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf55383bef6649eb9e78cb2d2512dedc, ed1efbce9ee3401e99a5260edeae4e7c; no voters:
I20250109 08:58:20.899792 23583 raft_consensus.cc:2798] T 8c5e3d0155d140c4b32f162d3173f5c9 P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:58:20.897764 23572 tablet_bootstrap.cc:492] T 82ea7dc042dd4d05b260d508f4422c3c P cf55383bef6649eb9e78cb2d2512dedc: Bootstrap starting.
W20250109 08:58:20.900208 23494 leader_election.cc:343] T 82ea7dc042dd4d05b260d508f4422c3c P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:20.900238 23583 raft_consensus.cc:695] T 8c5e3d0155d140c4b32f162d3173f5c9 P cf55383bef6649eb9e78cb2d2512dedc [term 1 LEADER]: Becoming Leader. State: Replica: cf55383bef6649eb9e78cb2d2512dedc, State: Running, Role: LEADER
I20250109 08:58:20.901366 23583 consensus_queue.cc:237] T 8c5e3d0155d140c4b32f162d3173f5c9 P cf55383bef6649eb9e78cb2d2512dedc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:20.905891 23635 raft_consensus.cc:513] T ed79047393d2425f8ecff31ea08b9cad P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:20.908123 23572 tablet_bootstrap.cc:654] T 82ea7dc042dd4d05b260d508f4422c3c P cf55383bef6649eb9e78cb2d2512dedc: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:20.911909 23528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed79047393d2425f8ecff31ea08b9cad" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01"
I20250109 08:58:20.912236 23304 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed79047393d2425f8ecff31ea08b9cad" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada"
I20250109 08:58:20.912671 23528 raft_consensus.cc:3054] T ed79047393d2425f8ecff31ea08b9cad P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Advancing to term 1
W20250109 08:58:20.913650 23346 leader_election.cc:343] T ed79047393d2425f8ecff31ea08b9cad P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:20.914834 23614 raft_consensus.cc:513] T 1340cf8ee2cb423f875276137da17e8b P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:20.919457 23528 raft_consensus.cc:2463] T ed79047393d2425f8ecff31ea08b9cad P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed1efbce9ee3401e99a5260edeae4e7c in term 1.
I20250109 08:58:20.920491 23574 ts_tablet_manager.cc:1428] T 3208a99957554f6db341654856980c8a P 235ad6af04c34c88a5261daa3604fada: Time spent starting tablet: real 0.080s user 0.006s sys 0.000s
I20250109 08:58:20.923205 23580 tablet_bootstrap.cc:492] T 6eef9bb3c645428d9f1cb843bae9a264 P ed1efbce9ee3401e99a5260edeae4e7c: No bootstrap required, opened a new log
I20250109 08:58:20.923691 23580 ts_tablet_manager.cc:1397] T 6eef9bb3c645428d9f1cb843bae9a264 P ed1efbce9ee3401e99a5260edeae4e7c: Time spent bootstrapping tablet: real 0.034s user 0.014s sys 0.000s
I20250109 08:58:20.924042 23574 tablet_bootstrap.cc:492] T ed79047393d2425f8ecff31ea08b9cad P 235ad6af04c34c88a5261daa3604fada: Bootstrap starting.
I20250109 08:58:20.924480 23635 leader_election.cc:290] T ed79047393d2425f8ecff31ea08b9cad P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 election: Requested vote from peers 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829), 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691)
I20250109 08:58:20.922231 23614 leader_election.cc:290] T 1340cf8ee2cb423f875276137da17e8b P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 election: Requested vote from peers ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365), cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551)
I20250109 08:58:20.925206 23631 raft_consensus.cc:513] T 82ea7dc042dd4d05b260d508f4422c3c P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:20.925686 23344 leader_election.cc:304] T ed79047393d2425f8ecff31ea08b9cad P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01, ed1efbce9ee3401e99a5260edeae4e7c; no voters: 235ad6af04c34c88a5261daa3604fada
I20250109 08:58:20.923915 23454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1340cf8ee2cb423f875276137da17e8b" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc"
I20250109 08:58:20.926146 23580 raft_consensus.cc:357] T 6eef9bb3c645428d9f1cb843bae9a264 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:20.926898 23580 raft_consensus.cc:383] T 6eef9bb3c645428d9f1cb843bae9a264 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:20.926925 23454 raft_consensus.cc:3054] T 1340cf8ee2cb423f875276137da17e8b P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:20.927291 23580 raft_consensus.cc:738] T 6eef9bb3c645428d9f1cb843bae9a264 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed1efbce9ee3401e99a5260edeae4e7c, State: Initialized, Role: FOLLOWER
I20250109 08:58:20.927806 23453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82ea7dc042dd4d05b260d508f4422c3c" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc"
I20250109 08:58:20.928056 23580 consensus_queue.cc:260] T 6eef9bb3c645428d9f1cb843bae9a264 P ed1efbce9ee3401e99a5260edeae4e7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:20.928743 23614 raft_consensus.cc:491] T 3208a99957554f6db341654856980c8a P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:20.929100 23611 raft_consensus.cc:2798] T ed79047393d2425f8ecff31ea08b9cad P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Leader election won for term 1
W20250109 08:58:20.929430 23494 leader_election.cc:343] T 82ea7dc042dd4d05b260d508f4422c3c P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:20.931900 23574 tablet_bootstrap.cc:654] T ed79047393d2425f8ecff31ea08b9cad P 235ad6af04c34c88a5261daa3604fada: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:20.931809 23304 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82ea7dc042dd4d05b260d508f4422c3c" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada"
I20250109 08:58:20.932472 23304 raft_consensus.cc:3054] T 82ea7dc042dd4d05b260d508f4422c3c P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:20.929184 23614 raft_consensus.cc:513] T 3208a99957554f6db341654856980c8a P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:20.935952 23580 ts_tablet_manager.cc:1428] T 6eef9bb3c645428d9f1cb843bae9a264 P ed1efbce9ee3401e99a5260edeae4e7c: Time spent starting tablet: real 0.012s user 0.008s sys 0.000s
I20250109 08:58:20.936336 23380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1340cf8ee2cb423f875276137da17e8b" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c"
I20250109 08:58:20.937070 23380 raft_consensus.cc:3054] T 1340cf8ee2cb423f875276137da17e8b P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:20.937078 23379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3208a99957554f6db341654856980c8a" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" is_pre_election: true
I20250109 08:58:20.938264 23453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3208a99957554f6db341654856980c8a" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc" is_pre_election: true
W20250109 08:58:20.939316 23271 consensus_peers.cc:487] T ea2b0762940e44dbbf69b9a163ace846 P 235ad6af04c34c88a5261daa3604fada -> Peer ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365): Couldn't send request to peer ed1efbce9ee3401e99a5260edeae4e7c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:20.939523 23627 raft_consensus.cc:491] T 5ccc0d42d4a148d7819366cea0b754a7 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:20.938385 23379 raft_consensus.cc:2463] T 3208a99957554f6db341654856980c8a P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 235ad6af04c34c88a5261daa3604fada in term 0.
W20250109 08:58:20.939993 23271 leader_election.cc:343] T 3208a99957554f6db341654856980c8a P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:20.939868 23304 raft_consensus.cc:2463] T 82ea7dc042dd4d05b260d508f4422c3c P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05a74e1eb2474b3d8b3a2da1ef127e01 in term 1.
I20250109 08:58:20.940824 23614 leader_election.cc:290] T 3208a99957554f6db341654856980c8a P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365), cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551)
I20250109 08:58:20.940274 23454 raft_consensus.cc:2463] T 1340cf8ee2cb423f875276137da17e8b P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 235ad6af04c34c88a5261daa3604fada in term 1.
I20250109 08:58:20.940011 23627 raft_consensus.cc:513] T 5ccc0d42d4a148d7819366cea0b754a7 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:20.940958 23631 leader_election.cc:290] T 82ea7dc042dd4d05b260d508f4422c3c P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 election: Requested vote from peers cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551), 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691)
I20250109 08:58:20.942952 23627 leader_election.cc:290] T 5ccc0d42d4a148d7819366cea0b754a7 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829), 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691)
I20250109 08:58:20.943006 23494 leader_election.cc:304] T 82ea7dc042dd4d05b260d508f4422c3c P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01, 235ad6af04c34c88a5261daa3604fada; no voters: cf55383bef6649eb9e78cb2d2512dedc
I20250109 08:58:20.945421 23304 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ccc0d42d4a148d7819366cea0b754a7" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada" is_pre_election: true
I20250109 08:58:20.946094 23528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ccc0d42d4a148d7819366cea0b754a7" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" is_pre_election: true
I20250109 08:58:20.946825 23528 raft_consensus.cc:2463] T 5ccc0d42d4a148d7819366cea0b754a7 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf55383bef6649eb9e78cb2d2512dedc in term 0.
W20250109 08:58:20.946734 23420 leader_election.cc:343] T 5ccc0d42d4a148d7819366cea0b754a7 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:20.947158 23271 leader_election.cc:304] T 1340cf8ee2cb423f875276137da17e8b P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 235ad6af04c34c88a5261daa3604fada, cf55383bef6649eb9e78cb2d2512dedc; no voters:
I20250109 08:58:20.948753 23614 raft_consensus.cc:2798] T 1340cf8ee2cb423f875276137da17e8b P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:58:20.948218 23418 leader_election.cc:304] T 5ccc0d42d4a148d7819366cea0b754a7 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01, cf55383bef6649eb9e78cb2d2512dedc; no voters: 235ad6af04c34c88a5261daa3604fada
I20250109 08:58:20.949148 23580 tablet_bootstrap.cc:492] T 34caa5988a734b65a2eb512747d833b7 P ed1efbce9ee3401e99a5260edeae4e7c: Bootstrap starting.
I20250109 08:58:20.949767 23614 raft_consensus.cc:695] T 1340cf8ee2cb423f875276137da17e8b P 235ad6af04c34c88a5261daa3604fada [term 1 LEADER]: Becoming Leader. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Running, Role: LEADER
I20250109 08:58:20.950026 23271 leader_election.cc:304] T 3208a99957554f6db341654856980c8a P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 235ad6af04c34c88a5261daa3604fada, ed1efbce9ee3401e99a5260edeae4e7c; no voters: cf55383bef6649eb9e78cb2d2512dedc
I20250109 08:58:20.950865 23586 raft_consensus.cc:2798] T 3208a99957554f6db341654856980c8a P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:58:20.951225 23586 raft_consensus.cc:491] T 3208a99957554f6db341654856980c8a P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:58:20.951563 23586 raft_consensus.cc:3054] T 3208a99957554f6db341654856980c8a P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:20.950866 23614 consensus_queue.cc:237] T 1340cf8ee2cb423f875276137da17e8b P 235ad6af04c34c88a5261daa3604fada [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:20.956511 23631 raft_consensus.cc:2798] T 82ea7dc042dd4d05b260d508f4422c3c P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:58:20.957075 23631 raft_consensus.cc:695] T 82ea7dc042dd4d05b260d508f4422c3c P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 LEADER]: Becoming Leader. State: Replica: 05a74e1eb2474b3d8b3a2da1ef127e01, State: Running, Role: LEADER
I20250109 08:58:20.957882 23569 tablet_bootstrap.cc:492] T 41e7f0cc38c4414ab88c92c1cdeb3984 P 05a74e1eb2474b3d8b3a2da1ef127e01: No bootstrap required, opened a new log
I20250109 08:58:20.949836 23627 raft_consensus.cc:2798] T 5ccc0d42d4a148d7819366cea0b754a7 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:58:20.960790 23569 ts_tablet_manager.cc:1397] T 41e7f0cc38c4414ab88c92c1cdeb3984 P 05a74e1eb2474b3d8b3a2da1ef127e01: Time spent bootstrapping tablet: real 0.096s user 0.017s sys 0.022s
I20250109 08:58:20.961025 23627 raft_consensus.cc:491] T 5ccc0d42d4a148d7819366cea0b754a7 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:58:20.961421 23627 raft_consensus.cc:3054] T 5ccc0d42d4a148d7819366cea0b754a7 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:20.970552 23611 raft_consensus.cc:695] T ed79047393d2425f8ecff31ea08b9cad P ed1efbce9ee3401e99a5260edeae4e7c [term 1 LEADER]: Becoming Leader. State: Replica: ed1efbce9ee3401e99a5260edeae4e7c, State: Running, Role: LEADER
I20250109 08:58:20.971736 23569 raft_consensus.cc:357] T 41e7f0cc38c4414ab88c92c1cdeb3984 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:20.971611 23611 consensus_queue.cc:237] T ed79047393d2425f8ecff31ea08b9cad P ed1efbce9ee3401e99a5260edeae4e7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:20.972669 23569 raft_consensus.cc:383] T 41e7f0cc38c4414ab88c92c1cdeb3984 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:20.972996 23569 raft_consensus.cc:738] T 41e7f0cc38c4414ab88c92c1cdeb3984 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05a74e1eb2474b3d8b3a2da1ef127e01, State: Initialized, Role: FOLLOWER
I20250109 08:58:20.973665 23569 consensus_queue.cc:260] T 41e7f0cc38c4414ab88c92c1cdeb3984 P 05a74e1eb2474b3d8b3a2da1ef127e01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:20.977557 23569 ts_tablet_manager.cc:1428] T 41e7f0cc38c4414ab88c92c1cdeb3984 P 05a74e1eb2474b3d8b3a2da1ef127e01: Time spent starting tablet: real 0.015s user 0.006s sys 0.000s
I20250109 08:58:20.977926 23380 raft_consensus.cc:2463] T 1340cf8ee2cb423f875276137da17e8b P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 235ad6af04c34c88a5261daa3604fada in term 1.
I20250109 08:58:20.978490 23569 tablet_bootstrap.cc:492] T 9b245cb3666c421c884cf20161a22831 P 05a74e1eb2474b3d8b3a2da1ef127e01: Bootstrap starting.
I20250109 08:58:20.979719 23586 raft_consensus.cc:513] T 3208a99957554f6db341654856980c8a P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:20.981701 23586 leader_election.cc:290] T 3208a99957554f6db341654856980c8a P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 election: Requested vote from peers ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365), cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551)
I20250109 08:58:20.982707 23199 catalog_manager.cc:5526] T 8c5e3d0155d140c4b32f162d3173f5c9 P cf55383bef6649eb9e78cb2d2512dedc reported cstate change: term changed from 0 to 1, leader changed from <none> to cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195). New cstate: current_term: 1 leader_uuid: "cf55383bef6649eb9e78cb2d2512dedc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:20.985126 23569 tablet_bootstrap.cc:654] T 9b245cb3666c421c884cf20161a22831 P 05a74e1eb2474b3d8b3a2da1ef127e01: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:20.957944 23631 consensus_queue.cc:237] T 82ea7dc042dd4d05b260d508f4422c3c P 05a74e1eb2474b3d8b3a2da1ef127e01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:20.985965 23454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3208a99957554f6db341654856980c8a" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc"
W20250109 08:58:20.987887 23271 leader_election.cc:343] T 3208a99957554f6db341654856980c8a P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:20.988749 23580 tablet_bootstrap.cc:654] T 34caa5988a734b65a2eb512747d833b7 P ed1efbce9ee3401e99a5260edeae4e7c: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:20.998929 23380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3208a99957554f6db341654856980c8a" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c"
I20250109 08:58:21.000062 23380 raft_consensus.cc:3054] T 3208a99957554f6db341654856980c8a P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:21.009416 23380 raft_consensus.cc:2463] T 3208a99957554f6db341654856980c8a P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 235ad6af04c34c88a5261daa3604fada in term 1.
I20250109 08:58:21.012337 23271 leader_election.cc:304] T 3208a99957554f6db341654856980c8a P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 235ad6af04c34c88a5261daa3604fada, ed1efbce9ee3401e99a5260edeae4e7c; no voters: cf55383bef6649eb9e78cb2d2512dedc
I20250109 08:58:21.009276 23201 catalog_manager.cc:5526] T 1340cf8ee2cb423f875276137da17e8b P 235ad6af04c34c88a5261daa3604fada reported cstate change: term changed from 0 to 1, leader changed from <none> to 235ad6af04c34c88a5261daa3604fada (127.19.226.193). New cstate: current_term: 1 leader_uuid: "235ad6af04c34c88a5261daa3604fada" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:21.013572 23586 raft_consensus.cc:2798] T 3208a99957554f6db341654856980c8a P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:58:21.014027 23586 raft_consensus.cc:695] T 3208a99957554f6db341654856980c8a P 235ad6af04c34c88a5261daa3604fada [term 1 LEADER]: Becoming Leader. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Running, Role: LEADER
I20250109 08:58:21.015278 23572 tablet_bootstrap.cc:492] T 82ea7dc042dd4d05b260d508f4422c3c P cf55383bef6649eb9e78cb2d2512dedc: No bootstrap required, opened a new log
I20250109 08:58:21.014988 23586 consensus_queue.cc:237] T 3208a99957554f6db341654856980c8a P 235ad6af04c34c88a5261daa3604fada [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:21.015770 23572 ts_tablet_manager.cc:1397] T 82ea7dc042dd4d05b260d508f4422c3c P cf55383bef6649eb9e78cb2d2512dedc: Time spent bootstrapping tablet: real 0.173s user 0.008s sys 0.009s
I20250109 08:58:21.009315 23627 raft_consensus.cc:513] T 5ccc0d42d4a148d7819366cea0b754a7 P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:21.029776 23606 raft_consensus.cc:491] T 9b245cb3666c421c884cf20161a22831 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:21.033020 23606 raft_consensus.cc:513] T 9b245cb3666c421c884cf20161a22831 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:21.033749 23527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ccc0d42d4a148d7819366cea0b754a7" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01"
I20250109 08:58:21.034456 23527 raft_consensus.cc:3054] T 5ccc0d42d4a148d7819366cea0b754a7 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:21.034688 23606 leader_election.cc:290] T 9b245cb3666c421c884cf20161a22831 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551), 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829)
I20250109 08:58:21.036919 23569 tablet_bootstrap.cc:492] T 9b245cb3666c421c884cf20161a22831 P 05a74e1eb2474b3d8b3a2da1ef127e01: No bootstrap required, opened a new log
I20250109 08:58:21.036818 23454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b245cb3666c421c884cf20161a22831" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc" is_pre_election: true
I20250109 08:58:21.037549 23569 ts_tablet_manager.cc:1397] T 9b245cb3666c421c884cf20161a22831 P 05a74e1eb2474b3d8b3a2da1ef127e01: Time spent bootstrapping tablet: real 0.059s user 0.011s sys 0.004s
I20250109 08:58:21.043005 23658 raft_consensus.cc:491] T 4df64aae69ca41dc91286cae1603000f P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:21.043565 23658 raft_consensus.cc:513] T 4df64aae69ca41dc91286cae1603000f P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
W20250109 08:58:21.044646 23271 leader_election.cc:343] T 9b245cb3666c421c884cf20161a22831 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:21.045884 23580 tablet_bootstrap.cc:492] T 34caa5988a734b65a2eb512747d833b7 P ed1efbce9ee3401e99a5260edeae4e7c: No bootstrap required, opened a new log
I20250109 08:58:21.046396 23580 ts_tablet_manager.cc:1397] T 34caa5988a734b65a2eb512747d833b7 P ed1efbce9ee3401e99a5260edeae4e7c: Time spent bootstrapping tablet: real 0.097s user 0.015s sys 0.005s
I20250109 08:58:21.049623 23528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b245cb3666c421c884cf20161a22831" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" is_pre_election: true
I20250109 08:58:21.045424 23611 raft_consensus.cc:491] T 9b0bbf2e4a424fb5ac13a3b55627b048 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:21.052229 23611 raft_consensus.cc:513] T 9b0bbf2e4a424fb5ac13a3b55627b048 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:21.054720 23611 leader_election.cc:290] T 9b0bbf2e4a424fb5ac13a3b55627b048 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551), 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829)
I20250109 08:58:21.056228 23658 leader_election.cc:290] T 4df64aae69ca41dc91286cae1603000f P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829), 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691)
I20250109 08:58:21.057555 23569 raft_consensus.cc:357] T 9b245cb3666c421c884cf20161a22831 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:21.058307 23569 raft_consensus.cc:383] T 9b245cb3666c421c884cf20161a22831 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:21.058570 23569 raft_consensus.cc:738] T 9b245cb3666c421c884cf20161a22831 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05a74e1eb2474b3d8b3a2da1ef127e01, State: Initialized, Role: FOLLOWER
I20250109 08:58:21.057945 23199 catalog_manager.cc:5526] T 82ea7dc042dd4d05b260d508f4422c3c P 05a74e1eb2474b3d8b3a2da1ef127e01 reported cstate change: term changed from 0 to 1, leader changed from <none> to 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196). New cstate: current_term: 1 leader_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:21.062129 23526 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4df64aae69ca41dc91286cae1603000f" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" is_pre_election: true
W20250109 08:58:21.063485 23344 leader_election.cc:343] T 4df64aae69ca41dc91286cae1603000f P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:21.063537 23454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b0bbf2e4a424fb5ac13a3b55627b048" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc" is_pre_election: true
W20250109 08:58:21.064728 23346 leader_election.cc:343] T 9b0bbf2e4a424fb5ac13a3b55627b048 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:21.059182 23569 consensus_queue.cc:260] T 9b245cb3666c421c884cf20161a22831 P 05a74e1eb2474b3d8b3a2da1ef127e01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:21.065948 23528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b0bbf2e4a424fb5ac13a3b55627b048" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" is_pre_election: true
I20250109 08:58:21.066609 23528 raft_consensus.cc:2463] T 9b0bbf2e4a424fb5ac13a3b55627b048 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed1efbce9ee3401e99a5260edeae4e7c in term 0.
I20250109 08:58:21.067714 23569 ts_tablet_manager.cc:1428] T 9b245cb3666c421c884cf20161a22831 P 05a74e1eb2474b3d8b3a2da1ef127e01: Time spent starting tablet: real 0.030s user 0.006s sys 0.000s
I20250109 08:58:21.047678 23627 leader_election.cc:290] T 5ccc0d42d4a148d7819366cea0b754a7 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 election: Requested vote from peers 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829), 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691)
I20250109 08:58:21.068459 23344 leader_election.cc:304] T 9b0bbf2e4a424fb5ac13a3b55627b048 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01, ed1efbce9ee3401e99a5260edeae4e7c; no voters: cf55383bef6649eb9e78cb2d2512dedc
I20250109 08:58:21.068825 23569 tablet_bootstrap.cc:492] T d5fa255ab5f24750a0afa6d81ec869f2 P 05a74e1eb2474b3d8b3a2da1ef127e01: Bootstrap starting.
I20250109 08:58:21.070422 23574 tablet_bootstrap.cc:492] T ed79047393d2425f8ecff31ea08b9cad P 235ad6af04c34c88a5261daa3604fada: No bootstrap required, opened a new log
I20250109 08:58:21.070884 23574 ts_tablet_manager.cc:1397] T ed79047393d2425f8ecff31ea08b9cad P 235ad6af04c34c88a5261daa3604fada: Time spent bootstrapping tablet: real 0.147s user 0.016s sys 0.000s
I20250109 08:58:21.071970 23658 raft_consensus.cc:2798] T 9b0bbf2e4a424fb5ac13a3b55627b048 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:58:21.072410 23658 raft_consensus.cc:491] T 9b0bbf2e4a424fb5ac13a3b55627b048 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:58:21.073582 23574 raft_consensus.cc:357] T ed79047393d2425f8ecff31ea08b9cad P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:21.074426 23574 raft_consensus.cc:383] T ed79047393d2425f8ecff31ea08b9cad P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:21.074692 23574 raft_consensus.cc:738] T ed79047393d2425f8ecff31ea08b9cad P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Initialized, Role: FOLLOWER
I20250109 08:58:21.075205 23574 consensus_queue.cc:260] T ed79047393d2425f8ecff31ea08b9cad P 235ad6af04c34c88a5261daa3604fada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:21.075486 23580 raft_consensus.cc:357] T 34caa5988a734b65a2eb512747d833b7 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:21.076201 23580 raft_consensus.cc:383] T 34caa5988a734b65a2eb512747d833b7 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:21.076491 23580 raft_consensus.cc:738] T 34caa5988a734b65a2eb512747d833b7 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed1efbce9ee3401e99a5260edeae4e7c, State: Initialized, Role: FOLLOWER
I20250109 08:58:21.077016 23574 ts_tablet_manager.cc:1428] T ed79047393d2425f8ecff31ea08b9cad P 235ad6af04c34c88a5261daa3604fada: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
W20250109 08:58:21.077544 23269 leader_election.cc:343] T 9b245cb3666c421c884cf20161a22831 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:21.077962 23574 tablet_bootstrap.cc:492] T 5c37c58abaa54656b31c72160b4dec9e P 235ad6af04c34c88a5261daa3604fada: Bootstrap starting.
I20250109 08:58:21.077771 23303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4df64aae69ca41dc91286cae1603000f" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada" is_pre_election: true
I20250109 08:58:21.077742 23305 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ccc0d42d4a148d7819366cea0b754a7" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada"
I20250109 08:58:21.080056 23527 raft_consensus.cc:2463] T 5ccc0d42d4a148d7819366cea0b754a7 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf55383bef6649eb9e78cb2d2512dedc in term 1.
I20250109 08:58:21.049849 23572 raft_consensus.cc:357] T 82ea7dc042dd4d05b260d508f4422c3c P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:21.080816 23572 raft_consensus.cc:383] T 82ea7dc042dd4d05b260d508f4422c3c P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:21.081123 23572 raft_consensus.cc:738] T 82ea7dc042dd4d05b260d508f4422c3c P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf55383bef6649eb9e78cb2d2512dedc, State: Initialized, Role: FOLLOWER
I20250109 08:58:21.081197 23418 leader_election.cc:304] T 5ccc0d42d4a148d7819366cea0b754a7 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01, cf55383bef6649eb9e78cb2d2512dedc; no voters:
I20250109 08:58:21.082105 23627 raft_consensus.cc:2798] T 5ccc0d42d4a148d7819366cea0b754a7 P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:58:21.081822 23572 consensus_queue.cc:260] T 82ea7dc042dd4d05b260d508f4422c3c P cf55383bef6649eb9e78cb2d2512dedc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:21.082593 23201 catalog_manager.cc:5526] T ed79047393d2425f8ecff31ea08b9cad P ed1efbce9ee3401e99a5260edeae4e7c reported cstate change: term changed from 0 to 1, leader changed from <none> to ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194). New cstate: current_term: 1 leader_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } health_report { overall_health: HEALTHY } } }
I20250109 08:58:21.084015 23574 tablet_bootstrap.cc:654] T 5c37c58abaa54656b31c72160b4dec9e P 235ad6af04c34c88a5261daa3604fada: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:21.082643 23627 raft_consensus.cc:695] T 5ccc0d42d4a148d7819366cea0b754a7 P cf55383bef6649eb9e78cb2d2512dedc [term 1 LEADER]: Becoming Leader. State: Replica: cf55383bef6649eb9e78cb2d2512dedc, State: Running, Role: LEADER
I20250109 08:58:21.085047 23627 consensus_queue.cc:237] T 5ccc0d42d4a148d7819366cea0b754a7 P cf55383bef6649eb9e78cb2d2512dedc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:21.087287 23572 ts_tablet_manager.cc:1428] T 82ea7dc042dd4d05b260d508f4422c3c P cf55383bef6649eb9e78cb2d2512dedc: Time spent starting tablet: real 0.071s user 0.005s sys 0.003s
I20250109 08:58:21.088150 23580 consensus_queue.cc:260] T 34caa5988a734b65a2eb512747d833b7 P ed1efbce9ee3401e99a5260edeae4e7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:21.086675 23658 raft_consensus.cc:3054] T 9b0bbf2e4a424fb5ac13a3b55627b048 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:21.089771 23269 leader_election.cc:304] T 9b245cb3666c421c884cf20161a22831 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 235ad6af04c34c88a5261daa3604fada; no voters: 05a74e1eb2474b3d8b3a2da1ef127e01, cf55383bef6649eb9e78cb2d2512dedc
I20250109 08:58:21.091539 23572 tablet_bootstrap.cc:492] T 3208a99957554f6db341654856980c8a P cf55383bef6649eb9e78cb2d2512dedc: Bootstrap starting.
I20250109 08:58:21.092150 23666 raft_consensus.cc:2743] T 9b245cb3666c421c884cf20161a22831 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:58:21.094364 23611 raft_consensus.cc:491] T 34caa5988a734b65a2eb512747d833b7 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:21.094866 23611 raft_consensus.cc:513] T 34caa5988a734b65a2eb512747d833b7 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:21.095763 23580 ts_tablet_manager.cc:1428] T 34caa5988a734b65a2eb512747d833b7 P ed1efbce9ee3401e99a5260edeae4e7c: Time spent starting tablet: real 0.049s user 0.006s sys 0.000s
I20250109 08:58:21.096900 23611 leader_election.cc:290] T 34caa5988a734b65a2eb512747d833b7 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551), 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829)
W20250109 08:58:21.097946 23346 leader_election.cc:343] T 4df64aae69ca41dc91286cae1603000f P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Illegal state: must be running to vote when last-logged opid is not known
W20250109 08:58:21.098125 23420 leader_election.cc:343] T 5ccc0d42d4a148d7819366cea0b754a7 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:21.098582 23569 tablet_bootstrap.cc:654] T d5fa255ab5f24750a0afa6d81ec869f2 P 05a74e1eb2474b3d8b3a2da1ef127e01: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:21.098410 23346 leader_election.cc:304] T 4df64aae69ca41dc91286cae1603000f P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ed1efbce9ee3401e99a5260edeae4e7c; no voters: 05a74e1eb2474b3d8b3a2da1ef127e01, 235ad6af04c34c88a5261daa3604fada
I20250109 08:58:21.099573 23635 raft_consensus.cc:2743] T 4df64aae69ca41dc91286cae1603000f P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:58:21.100831 23454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34caa5988a734b65a2eb512747d833b7" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc" is_pre_election: true
I20250109 08:58:21.101465 23454 raft_consensus.cc:2463] T 34caa5988a734b65a2eb512747d833b7 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed1efbce9ee3401e99a5260edeae4e7c in term 0.
I20250109 08:58:21.101387 23583 consensus_queue.cc:1035] T 2e3d34c0d09843d2be54e47f3ccc16d8 P cf55383bef6649eb9e78cb2d2512dedc [LEADER]: Connected to new peer: Peer: permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250109 08:58:21.102860 23580 tablet_bootstrap.cc:492] T ea2b0762940e44dbbf69b9a163ace846 P ed1efbce9ee3401e99a5260edeae4e7c: Bootstrap starting.
I20250109 08:58:21.104209 23346 leader_election.cc:304] T 34caa5988a734b65a2eb512747d833b7 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf55383bef6649eb9e78cb2d2512dedc, ed1efbce9ee3401e99a5260edeae4e7c; no voters:
I20250109 08:58:21.105144 23635 raft_consensus.cc:2798] T 34caa5988a734b65a2eb512747d833b7 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:58:21.105505 23635 raft_consensus.cc:491] T 34caa5988a734b65a2eb512747d833b7 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:58:21.105832 23635 raft_consensus.cc:3054] T 34caa5988a734b65a2eb512747d833b7 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:21.106662 23658 raft_consensus.cc:513] T 9b0bbf2e4a424fb5ac13a3b55627b048 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:21.108932 23658 leader_election.cc:290] T 9b0bbf2e4a424fb5ac13a3b55627b048 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 election: Requested vote from peers cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551), 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829)
I20250109 08:58:21.109493 23527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34caa5988a734b65a2eb512747d833b7" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" is_pre_election: true
I20250109 08:58:21.110198 23527 raft_consensus.cc:2463] T 34caa5988a734b65a2eb512747d833b7 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed1efbce9ee3401e99a5260edeae4e7c in term 0.
I20250109 08:58:21.111503 23572 tablet_bootstrap.cc:654] T 3208a99957554f6db341654856980c8a P cf55383bef6649eb9e78cb2d2512dedc: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:21.112272 23454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b0bbf2e4a424fb5ac13a3b55627b048" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc"
W20250109 08:58:21.113715 23346 leader_election.cc:343] T 9b0bbf2e4a424fb5ac13a3b55627b048 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:21.125794 23200 catalog_manager.cc:5526] T 5ccc0d42d4a148d7819366cea0b754a7 P cf55383bef6649eb9e78cb2d2512dedc reported cstate change: term changed from 0 to 1, leader changed from <none> to cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195). New cstate: current_term: 1 leader_uuid: "cf55383bef6649eb9e78cb2d2512dedc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:21.131119 23527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b0bbf2e4a424fb5ac13a3b55627b048" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01"
I20250109 08:58:21.133009 23527 raft_consensus.cc:3054] T 9b0bbf2e4a424fb5ac13a3b55627b048 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:21.134914 23202 catalog_manager.cc:5526] T 3208a99957554f6db341654856980c8a P 235ad6af04c34c88a5261daa3604fada reported cstate change: term changed from 0 to 1, leader changed from <none> to 235ad6af04c34c88a5261daa3604fada (127.19.226.193). New cstate: current_term: 1 leader_uuid: "235ad6af04c34c88a5261daa3604fada" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:21.139662 23666 raft_consensus.cc:491] T 970a82e754db47b6b1a19551d4ba43d8 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:21.144472 23527 raft_consensus.cc:2463] T 9b0bbf2e4a424fb5ac13a3b55627b048 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed1efbce9ee3401e99a5260edeae4e7c in term 1.
I20250109 08:58:21.144906 23666 raft_consensus.cc:513] T 970a82e754db47b6b1a19551d4ba43d8 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:21.145761 23344 leader_election.cc:304] T 9b0bbf2e4a424fb5ac13a3b55627b048 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01, ed1efbce9ee3401e99a5260edeae4e7c; no voters: cf55383bef6649eb9e78cb2d2512dedc
I20250109 08:58:21.147044 23666 leader_election.cc:290] T 970a82e754db47b6b1a19551d4ba43d8 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829), ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365)
I20250109 08:58:21.147215 23658 raft_consensus.cc:2798] T 9b0bbf2e4a424fb5ac13a3b55627b048 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:58:21.149600 23658 raft_consensus.cc:695] T 9b0bbf2e4a424fb5ac13a3b55627b048 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 LEADER]: Becoming Leader. State: Replica: ed1efbce9ee3401e99a5260edeae4e7c, State: Running, Role: LEADER
I20250109 08:58:21.151638 23380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "970a82e754db47b6b1a19551d4ba43d8" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" is_pre_election: true
I20250109 08:58:21.151770 23658 consensus_queue.cc:237] T 9b0bbf2e4a424fb5ac13a3b55627b048 P ed1efbce9ee3401e99a5260edeae4e7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:21.154920 23574 tablet_bootstrap.cc:492] T 5c37c58abaa54656b31c72160b4dec9e P 235ad6af04c34c88a5261daa3604fada: No bootstrap required, opened a new log
I20250109 08:58:21.155398 23574 ts_tablet_manager.cc:1397] T 5c37c58abaa54656b31c72160b4dec9e P 235ad6af04c34c88a5261daa3604fada: Time spent bootstrapping tablet: real 0.078s user 0.012s sys 0.001s
I20250109 08:58:21.155879 23527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "970a82e754db47b6b1a19551d4ba43d8" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" is_pre_election: true
I20250109 08:58:21.156800 23527 raft_consensus.cc:2388] T 970a82e754db47b6b1a19551d4ba43d8 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 235ad6af04c34c88a5261daa3604fada in current term 1: Already voted for candidate 05a74e1eb2474b3d8b3a2da1ef127e01 in this term.
I20250109 08:58:21.157418 23572 tablet_bootstrap.cc:492] T 3208a99957554f6db341654856980c8a P cf55383bef6649eb9e78cb2d2512dedc: No bootstrap required, opened a new log
I20250109 08:58:21.157888 23572 ts_tablet_manager.cc:1397] T 3208a99957554f6db341654856980c8a P cf55383bef6649eb9e78cb2d2512dedc: Time spent bootstrapping tablet: real 0.067s user 0.022s sys 0.020s
I20250109 08:58:21.158022 23574 raft_consensus.cc:357] T 5c37c58abaa54656b31c72160b4dec9e P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:21.158782 23574 raft_consensus.cc:383] T 5c37c58abaa54656b31c72160b4dec9e P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:21.159042 23574 raft_consensus.cc:738] T 5c37c58abaa54656b31c72160b4dec9e P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Initialized, Role: FOLLOWER
I20250109 08:58:21.159773 23380 raft_consensus.cc:2388] T 970a82e754db47b6b1a19551d4ba43d8 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 235ad6af04c34c88a5261daa3604fada in current term 1: Already voted for candidate ed1efbce9ee3401e99a5260edeae4e7c in this term.
I20250109 08:58:21.159729 23574 consensus_queue.cc:260] T 5c37c58abaa54656b31c72160b4dec9e P 235ad6af04c34c88a5261daa3604fada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:21.160022 23572 raft_consensus.cc:357] T 3208a99957554f6db341654856980c8a P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:21.160730 23572 raft_consensus.cc:383] T 3208a99957554f6db341654856980c8a P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:21.161083 23572 raft_consensus.cc:738] T 3208a99957554f6db341654856980c8a P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf55383bef6649eb9e78cb2d2512dedc, State: Initialized, Role: FOLLOWER
I20250109 08:58:21.160744 23271 leader_election.cc:304] T 970a82e754db47b6b1a19551d4ba43d8 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 235ad6af04c34c88a5261daa3604fada; no voters: 05a74e1eb2474b3d8b3a2da1ef127e01, ed1efbce9ee3401e99a5260edeae4e7c
I20250109 08:58:21.164726 23572 consensus_queue.cc:260] T 3208a99957554f6db341654856980c8a P cf55383bef6649eb9e78cb2d2512dedc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:21.154862 23569 tablet_bootstrap.cc:492] T d5fa255ab5f24750a0afa6d81ec869f2 P 05a74e1eb2474b3d8b3a2da1ef127e01: No bootstrap required, opened a new log
I20250109 08:58:21.165952 23569 ts_tablet_manager.cc:1397] T d5fa255ab5f24750a0afa6d81ec869f2 P 05a74e1eb2474b3d8b3a2da1ef127e01: Time spent bootstrapping tablet: real 0.097s user 0.020s sys 0.017s
I20250109 08:58:21.167397 23572 ts_tablet_manager.cc:1428] T 3208a99957554f6db341654856980c8a P cf55383bef6649eb9e78cb2d2512dedc: Time spent starting tablet: real 0.009s user 0.006s sys 0.004s
I20250109 08:58:21.167546 23574 ts_tablet_manager.cc:1428] T 5c37c58abaa54656b31c72160b4dec9e P 235ad6af04c34c88a5261daa3604fada: Time spent starting tablet: real 0.012s user 0.006s sys 0.000s
I20250109 08:58:21.168399 23574 tablet_bootstrap.cc:492] T 4df64aae69ca41dc91286cae1603000f P 235ad6af04c34c88a5261daa3604fada: Bootstrap starting.
I20250109 08:58:21.170048 23572 tablet_bootstrap.cc:492] T 940653b6d55c46439827697b9aadf16b P cf55383bef6649eb9e78cb2d2512dedc: Bootstrap starting.
I20250109 08:58:21.165385 23666 raft_consensus.cc:3054] T 970a82e754db47b6b1a19551d4ba43d8 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:21.170759 23627 raft_consensus.cc:491] T 3208a99957554f6db341654856980c8a P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:21.171674 23627 raft_consensus.cc:513] T 3208a99957554f6db341654856980c8a P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:21.173588 23627 leader_election.cc:290] T 3208a99957554f6db341654856980c8a P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691), ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365)
I20250109 08:58:21.175058 23379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3208a99957554f6db341654856980c8a" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" is_pre_election: true
I20250109 08:58:21.175058 23303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3208a99957554f6db341654856980c8a" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada" is_pre_election: true
I20250109 08:58:21.175983 23379 raft_consensus.cc:2388] T 3208a99957554f6db341654856980c8a P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf55383bef6649eb9e78cb2d2512dedc in current term 1: Already voted for candidate 235ad6af04c34c88a5261daa3604fada in this term.
I20250109 08:58:21.178175 23420 leader_election.cc:304] T 3208a99957554f6db341654856980c8a P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf55383bef6649eb9e78cb2d2512dedc; no voters: 235ad6af04c34c88a5261daa3604fada, ed1efbce9ee3401e99a5260edeae4e7c
I20250109 08:58:21.179888 23627 raft_consensus.cc:3054] T 3208a99957554f6db341654856980c8a P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:21.183851 23580 tablet_bootstrap.cc:654] T ea2b0762940e44dbbf69b9a163ace846 P ed1efbce9ee3401e99a5260edeae4e7c: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:21.186489 23666 raft_consensus.cc:2743] T 970a82e754db47b6b1a19551d4ba43d8 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:58:21.188457 23572 tablet_bootstrap.cc:654] T 940653b6d55c46439827697b9aadf16b P cf55383bef6649eb9e78cb2d2512dedc: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:21.194977 23569 raft_consensus.cc:357] T d5fa255ab5f24750a0afa6d81ec869f2 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:21.197753 23569 raft_consensus.cc:383] T d5fa255ab5f24750a0afa6d81ec869f2 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:21.198542 23569 raft_consensus.cc:738] T d5fa255ab5f24750a0afa6d81ec869f2 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05a74e1eb2474b3d8b3a2da1ef127e01, State: Initialized, Role: FOLLOWER
I20250109 08:58:21.198307 23635 raft_consensus.cc:513] T 34caa5988a734b65a2eb512747d833b7 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:21.200973 23635 leader_election.cc:290] T 34caa5988a734b65a2eb512747d833b7 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 election: Requested vote from peers cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551), 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829)
I20250109 08:58:21.201805 23454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34caa5988a734b65a2eb512747d833b7" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc"
I20250109 08:58:21.202399 23454 raft_consensus.cc:3054] T 34caa5988a734b65a2eb512747d833b7 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:21.205461 23627 raft_consensus.cc:2743] T 3208a99957554f6db341654856980c8a P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:58:21.209872 23569 consensus_queue.cc:260] T d5fa255ab5f24750a0afa6d81ec869f2 P 05a74e1eb2474b3d8b3a2da1ef127e01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:21.212633 23569 ts_tablet_manager.cc:1428] T d5fa255ab5f24750a0afa6d81ec869f2 P 05a74e1eb2474b3d8b3a2da1ef127e01: Time spent starting tablet: real 0.046s user 0.006s sys 0.001s
I20250109 08:58:21.217856 23574 tablet_bootstrap.cc:654] T 4df64aae69ca41dc91286cae1603000f P 235ad6af04c34c88a5261daa3604fada: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:21.220620 23527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34caa5988a734b65a2eb512747d833b7" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01"
I20250109 08:58:21.221231 23527 raft_consensus.cc:3054] T 34caa5988a734b65a2eb512747d833b7 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Advancing to term 1
W20250109 08:58:21.223433 23251 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20250109 08:58:21.226495 23583 consensus_queue.cc:1035] T 2e3d34c0d09843d2be54e47f3ccc16d8 P cf55383bef6649eb9e78cb2d2512dedc [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:58:21.227757 23583 raft_consensus.cc:491] T 82ea7dc042dd4d05b260d508f4422c3c P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:21.228161 23583 raft_consensus.cc:513] T 82ea7dc042dd4d05b260d508f4422c3c P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:21.230015 23583 leader_election.cc:290] T 82ea7dc042dd4d05b260d508f4422c3c P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829), 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691)
I20250109 08:58:21.231438 23305 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82ea7dc042dd4d05b260d508f4422c3c" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada" is_pre_election: true
I20250109 08:58:21.232266 23305 raft_consensus.cc:2388] T 82ea7dc042dd4d05b260d508f4422c3c P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf55383bef6649eb9e78cb2d2512dedc in current term 1: Already voted for candidate 05a74e1eb2474b3d8b3a2da1ef127e01 in this term.
I20250109 08:58:21.237931 23635 raft_consensus.cc:491] T 6eef9bb3c645428d9f1cb843bae9a264 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:21.238463 23635 raft_consensus.cc:513] T 6eef9bb3c645428d9f1cb843bae9a264 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:21.240634 23528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82ea7dc042dd4d05b260d508f4422c3c" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" is_pre_election: true
I20250109 08:58:21.242228 23418 leader_election.cc:304] T 82ea7dc042dd4d05b260d508f4422c3c P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf55383bef6649eb9e78cb2d2512dedc; no voters: 05a74e1eb2474b3d8b3a2da1ef127e01, 235ad6af04c34c88a5261daa3604fada
I20250109 08:58:21.242477 23454 raft_consensus.cc:2463] T 34caa5988a734b65a2eb512747d833b7 P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed1efbce9ee3401e99a5260edeae4e7c in term 1.
I20250109 08:58:21.243057 23675 raft_consensus.cc:3054] T 82ea7dc042dd4d05b260d508f4422c3c P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:21.250705 23569 tablet_bootstrap.cc:492] T 08214ee9c4c14bda92bdc6fa8231c313 P 05a74e1eb2474b3d8b3a2da1ef127e01: Bootstrap starting.
I20250109 08:58:21.266911 23572 tablet_bootstrap.cc:492] T 940653b6d55c46439827697b9aadf16b P cf55383bef6649eb9e78cb2d2512dedc: No bootstrap required, opened a new log
I20250109 08:58:21.267469 23572 ts_tablet_manager.cc:1397] T 940653b6d55c46439827697b9aadf16b P cf55383bef6649eb9e78cb2d2512dedc: Time spent bootstrapping tablet: real 0.098s user 0.007s sys 0.017s
I20250109 08:58:21.270092 23346 leader_election.cc:304] T 34caa5988a734b65a2eb512747d833b7 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf55383bef6649eb9e78cb2d2512dedc, ed1efbce9ee3401e99a5260edeae4e7c; no voters:
I20250109 08:58:21.271067 23658 raft_consensus.cc:2798] T 34caa5988a734b65a2eb512747d833b7 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:58:21.271222 23527 raft_consensus.cc:2463] T 34caa5988a734b65a2eb512747d833b7 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed1efbce9ee3401e99a5260edeae4e7c in term 1.
I20250109 08:58:21.271574 23658 raft_consensus.cc:695] T 34caa5988a734b65a2eb512747d833b7 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 LEADER]: Becoming Leader. State: Replica: ed1efbce9ee3401e99a5260edeae4e7c, State: Running, Role: LEADER
I20250109 08:58:21.272414 23658 consensus_queue.cc:237] T 34caa5988a734b65a2eb512747d833b7 P ed1efbce9ee3401e99a5260edeae4e7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:21.281490 23635 leader_election.cc:290] T 6eef9bb3c645428d9f1cb843bae9a264 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691), cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551)
I20250109 08:58:21.281778 23305 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6eef9bb3c645428d9f1cb843bae9a264" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada" is_pre_election: true
I20250109 08:58:21.282712 23305 raft_consensus.cc:2463] T 6eef9bb3c645428d9f1cb843bae9a264 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed1efbce9ee3401e99a5260edeae4e7c in term 0.
I20250109 08:58:21.283550 23454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6eef9bb3c645428d9f1cb843bae9a264" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc" is_pre_election: true
I20250109 08:58:21.283836 23346 leader_election.cc:304] T 6eef9bb3c645428d9f1cb843bae9a264 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 235ad6af04c34c88a5261daa3604fada, ed1efbce9ee3401e99a5260edeae4e7c; no voters:
W20250109 08:58:21.285204 23346 leader_election.cc:343] T 6eef9bb3c645428d9f1cb843bae9a264 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:21.288254 23574 tablet_bootstrap.cc:492] T 4df64aae69ca41dc91286cae1603000f P 235ad6af04c34c88a5261daa3604fada: No bootstrap required, opened a new log
I20250109 08:58:21.288012 23572 raft_consensus.cc:357] T 940653b6d55c46439827697b9aadf16b P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:21.288719 23574 ts_tablet_manager.cc:1397] T 4df64aae69ca41dc91286cae1603000f P 235ad6af04c34c88a5261daa3604fada: Time spent bootstrapping tablet: real 0.120s user 0.011s sys 0.000s
I20250109 08:58:21.288779 23572 raft_consensus.cc:383] T 940653b6d55c46439827697b9aadf16b P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:21.289114 23572 raft_consensus.cc:738] T 940653b6d55c46439827697b9aadf16b P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf55383bef6649eb9e78cb2d2512dedc, State: Initialized, Role: FOLLOWER
I20250109 08:58:21.289276 23675 raft_consensus.cc:2743] T 82ea7dc042dd4d05b260d508f4422c3c P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:58:21.290241 23635 raft_consensus.cc:2798] T 6eef9bb3c645428d9f1cb843bae9a264 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:58:21.290616 23635 raft_consensus.cc:491] T 6eef9bb3c645428d9f1cb843bae9a264 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:58:21.290937 23635 raft_consensus.cc:3054] T 6eef9bb3c645428d9f1cb843bae9a264 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:21.292969 23580 tablet_bootstrap.cc:492] T ea2b0762940e44dbbf69b9a163ace846 P ed1efbce9ee3401e99a5260edeae4e7c: No bootstrap required, opened a new log
I20250109 08:58:21.293560 23580 ts_tablet_manager.cc:1397] T ea2b0762940e44dbbf69b9a163ace846 P ed1efbce9ee3401e99a5260edeae4e7c: Time spent bootstrapping tablet: real 0.192s user 0.014s sys 0.020s
I20250109 08:58:21.289856 23572 consensus_queue.cc:260] T 940653b6d55c46439827697b9aadf16b P cf55383bef6649eb9e78cb2d2512dedc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:21.297408 23580 raft_consensus.cc:357] T ea2b0762940e44dbbf69b9a163ace846 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:21.298527 23580 raft_consensus.cc:383] T ea2b0762940e44dbbf69b9a163ace846 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:21.298856 23631 raft_consensus.cc:491] T 9b245cb3666c421c884cf20161a22831 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:21.298852 23580 raft_consensus.cc:738] T ea2b0762940e44dbbf69b9a163ace846 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed1efbce9ee3401e99a5260edeae4e7c, State: Initialized, Role: FOLLOWER
I20250109 08:58:21.299515 23631 raft_consensus.cc:513] T 9b245cb3666c421c884cf20161a22831 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:21.299955 23580 consensus_queue.cc:260] T ea2b0762940e44dbbf69b9a163ace846 P ed1efbce9ee3401e99a5260edeae4e7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:21.301216 23572 ts_tablet_manager.cc:1428] T 940653b6d55c46439827697b9aadf16b P cf55383bef6649eb9e78cb2d2512dedc: Time spent starting tablet: real 0.033s user 0.002s sys 0.005s
I20250109 08:58:21.301774 23631 leader_election.cc:290] T 9b245cb3666c421c884cf20161a22831 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551), 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691)
I20250109 08:58:21.302163 23572 tablet_bootstrap.cc:492] T 6eef9bb3c645428d9f1cb843bae9a264 P cf55383bef6649eb9e78cb2d2512dedc: Bootstrap starting.
I20250109 08:58:21.302963 23580 ts_tablet_manager.cc:1428] T ea2b0762940e44dbbf69b9a163ace846 P ed1efbce9ee3401e99a5260edeae4e7c: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20250109 08:58:21.303910 23580 tablet_bootstrap.cc:492] T 8de6b2fb09bc46ce9e4033af06dffd04 P ed1efbce9ee3401e99a5260edeae4e7c: Bootstrap starting.
I20250109 08:58:21.306452 23305 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b245cb3666c421c884cf20161a22831" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada" is_pre_election: true
I20250109 08:58:21.307144 23305 raft_consensus.cc:2463] T 9b245cb3666c421c884cf20161a22831 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05a74e1eb2474b3d8b3a2da1ef127e01 in term 0.
I20250109 08:58:21.307901 23572 tablet_bootstrap.cc:654] T 6eef9bb3c645428d9f1cb843bae9a264 P cf55383bef6649eb9e78cb2d2512dedc: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:21.308557 23494 leader_election.cc:304] T 9b245cb3666c421c884cf20161a22831 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01, 235ad6af04c34c88a5261daa3604fada; no voters:
I20250109 08:58:21.309348 23634 raft_consensus.cc:491] T 41e7f0cc38c4414ab88c92c1cdeb3984 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:21.309844 23634 raft_consensus.cc:513] T 41e7f0cc38c4414ab88c92c1cdeb3984 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:21.311870 23634 leader_election.cc:290] T 41e7f0cc38c4414ab88c92c1cdeb3984 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365), 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691)
I20250109 08:58:21.312301 23631 raft_consensus.cc:2798] T 9b245cb3666c421c884cf20161a22831 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:58:21.313169 23631 raft_consensus.cc:491] T 9b245cb3666c421c884cf20161a22831 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:58:21.313571 23631 raft_consensus.cc:3054] T 9b245cb3666c421c884cf20161a22831 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:21.314934 23634 raft_consensus.cc:491] T d5fa255ab5f24750a0afa6d81ec869f2 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:21.315389 23634 raft_consensus.cc:513] T d5fa255ab5f24750a0afa6d81ec869f2 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:21.316545 23666 raft_consensus.cc:491] T 5c37c58abaa54656b31c72160b4dec9e P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:21.317173 23634 leader_election.cc:290] T d5fa255ab5f24750a0afa6d81ec869f2 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365), cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551)
I20250109 08:58:21.317005 23666 raft_consensus.cc:513] T 5c37c58abaa54656b31c72160b4dec9e P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:21.311964 23454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b245cb3666c421c884cf20161a22831" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc" is_pre_election: true
I20250109 08:58:21.318622 23379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41e7f0cc38c4414ab88c92c1cdeb3984" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" is_pre_election: true
I20250109 08:58:21.318715 23378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5fa255ab5f24750a0afa6d81ec869f2" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" is_pre_election: true
I20250109 08:58:21.320184 23527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c37c58abaa54656b31c72160b4dec9e" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" is_pre_election: true
I20250109 08:58:21.318910 23666 leader_election.cc:290] T 5c37c58abaa54656b31c72160b4dec9e P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551), 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829)
I20250109 08:58:21.320923 23454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c37c58abaa54656b31c72160b4dec9e" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc" is_pre_election: true
I20250109 08:58:21.321521 23453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5fa255ab5f24750a0afa6d81ec869f2" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc" is_pre_election: true
I20250109 08:58:21.321370 23305 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41e7f0cc38c4414ab88c92c1cdeb3984" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada" is_pre_election: true
W20250109 08:58:21.321460 23269 leader_election.cc:343] T 5c37c58abaa54656b31c72160b4dec9e P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:21.323350 23305 raft_consensus.cc:2463] T 41e7f0cc38c4414ab88c92c1cdeb3984 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05a74e1eb2474b3d8b3a2da1ef127e01 in term 0.
W20250109 08:58:21.323935 23271 leader_election.cc:343] T 5c37c58abaa54656b31c72160b4dec9e P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551): Illegal state: must be running to vote when last-logged opid is not known
W20250109 08:58:21.323305 23494 leader_election.cc:343] T 9b245cb3666c421c884cf20161a22831 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551): Illegal state: must be running to vote when last-logged opid is not known
W20250109 08:58:21.325428 23494 leader_election.cc:343] T d5fa255ab5f24750a0afa6d81ec869f2 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551): Illegal state: must be running to vote when last-logged opid is not known
W20250109 08:58:21.326102 23494 leader_election.cc:343] T 41e7f0cc38c4414ab88c92c1cdeb3984 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365): Illegal state: must be running to vote when last-logged opid is not known
W20250109 08:58:21.326777 23494 leader_election.cc:343] T d5fa255ab5f24750a0afa6d81ec869f2 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:21.326694 23271 leader_election.cc:304] T 5c37c58abaa54656b31c72160b4dec9e P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 235ad6af04c34c88a5261daa3604fada; no voters: 05a74e1eb2474b3d8b3a2da1ef127e01, cf55383bef6649eb9e78cb2d2512dedc
I20250109 08:58:21.327245 23494 leader_election.cc:304] T d5fa255ab5f24750a0afa6d81ec869f2 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01; no voters: cf55383bef6649eb9e78cb2d2512dedc, ed1efbce9ee3401e99a5260edeae4e7c
I20250109 08:58:21.327548 23666 raft_consensus.cc:2743] T 5c37c58abaa54656b31c72160b4dec9e P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:58:21.328428 23634 raft_consensus.cc:2743] T d5fa255ab5f24750a0afa6d81ec869f2 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:58:21.329304 23580 tablet_bootstrap.cc:654] T 8de6b2fb09bc46ce9e4033af06dffd04 P ed1efbce9ee3401e99a5260edeae4e7c: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:21.329319 23574 raft_consensus.cc:357] T 4df64aae69ca41dc91286cae1603000f P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:21.330444 23574 raft_consensus.cc:383] T 4df64aae69ca41dc91286cae1603000f P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:21.330698 23574 raft_consensus.cc:738] T 4df64aae69ca41dc91286cae1603000f P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Initialized, Role: FOLLOWER
I20250109 08:58:21.331182 23572 tablet_bootstrap.cc:492] T 6eef9bb3c645428d9f1cb843bae9a264 P cf55383bef6649eb9e78cb2d2512dedc: No bootstrap required, opened a new log
I20250109 08:58:21.328326 23494 leader_election.cc:304] T 41e7f0cc38c4414ab88c92c1cdeb3984 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01, 235ad6af04c34c88a5261daa3604fada; no voters: ed1efbce9ee3401e99a5260edeae4e7c
I20250109 08:58:21.331794 23572 ts_tablet_manager.cc:1397] T 6eef9bb3c645428d9f1cb843bae9a264 P cf55383bef6649eb9e78cb2d2512dedc: Time spent bootstrapping tablet: real 0.030s user 0.010s sys 0.002s
I20250109 08:58:21.331389 23574 consensus_queue.cc:260] T 4df64aae69ca41dc91286cae1603000f P 235ad6af04c34c88a5261daa3604fada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:21.331609 23635 raft_consensus.cc:513] T 6eef9bb3c645428d9f1cb843bae9a264 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:21.332206 23634 raft_consensus.cc:2798] T 41e7f0cc38c4414ab88c92c1cdeb3984 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:58:21.335470 23634 raft_consensus.cc:491] T 41e7f0cc38c4414ab88c92c1cdeb3984 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:58:21.335865 23634 raft_consensus.cc:3054] T 41e7f0cc38c4414ab88c92c1cdeb3984 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:21.336408 23305 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6eef9bb3c645428d9f1cb843bae9a264" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada"
I20250109 08:58:21.336997 23305 raft_consensus.cc:3054] T 6eef9bb3c645428d9f1cb843bae9a264 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:21.333024 23569 tablet_bootstrap.cc:654] T 08214ee9c4c14bda92bdc6fa8231c313 P 05a74e1eb2474b3d8b3a2da1ef127e01: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:21.342372 23634 raft_consensus.cc:513] T 41e7f0cc38c4414ab88c92c1cdeb3984 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:21.333864 23574 ts_tablet_manager.cc:1428] T 4df64aae69ca41dc91286cae1603000f P 235ad6af04c34c88a5261daa3604fada: Time spent starting tablet: real 0.045s user 0.005s sys 0.000s
I20250109 08:58:21.324337 23631 raft_consensus.cc:513] T 9b245cb3666c421c884cf20161a22831 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:21.334460 23572 raft_consensus.cc:357] T 6eef9bb3c645428d9f1cb843bae9a264 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:21.343935 23634 leader_election.cc:290] T 41e7f0cc38c4414ab88c92c1cdeb3984 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 election: Requested vote from peers ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365), 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691)
I20250109 08:58:21.344694 23380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41e7f0cc38c4414ab88c92c1cdeb3984" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c"
I20250109 08:58:21.345773 23666 raft_consensus.cc:491] T ed79047393d2425f8ecff31ea08b9cad P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:21.346544 23305 raft_consensus.cc:2463] T 6eef9bb3c645428d9f1cb843bae9a264 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed1efbce9ee3401e99a5260edeae4e7c in term 1.
I20250109 08:58:21.347834 23631 leader_election.cc:290] T 9b245cb3666c421c884cf20161a22831 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 election: Requested vote from peers cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551), 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691)
I20250109 08:58:21.348649 23454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b245cb3666c421c884cf20161a22831" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc"
I20250109 08:58:21.352869 23666 raft_consensus.cc:513] T ed79047393d2425f8ecff31ea08b9cad P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:21.353732 23346 leader_election.cc:304] T 6eef9bb3c645428d9f1cb843bae9a264 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 235ad6af04c34c88a5261daa3604fada, ed1efbce9ee3401e99a5260edeae4e7c; no voters:
I20250109 08:58:21.349606 23574 tablet_bootstrap.cc:492] T a54ca9f0d6e844e9a8b1d9cd0bb2a026 P 235ad6af04c34c88a5261daa3604fada: Bootstrap starting.
I20250109 08:58:21.354669 23658 raft_consensus.cc:2798] T 6eef9bb3c645428d9f1cb843bae9a264 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:58:21.353935 23586 consensus_queue.cc:1035] T 1340cf8ee2cb423f875276137da17e8b P 235ad6af04c34c88a5261daa3604fada [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250109 08:58:21.349924 23635 leader_election.cc:290] T 6eef9bb3c645428d9f1cb843bae9a264 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 election: Requested vote from peers 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691), cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551)
I20250109 08:58:21.355119 23658 raft_consensus.cc:695] T 6eef9bb3c645428d9f1cb843bae9a264 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 LEADER]: Becoming Leader. State: Replica: ed1efbce9ee3401e99a5260edeae4e7c, State: Running, Role: LEADER
I20250109 08:58:21.350961 23453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6eef9bb3c645428d9f1cb843bae9a264" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc"
I20250109 08:58:21.345019 23303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41e7f0cc38c4414ab88c92c1cdeb3984" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada"
I20250109 08:58:21.349025 23304 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b245cb3666c421c884cf20161a22831" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada"
I20250109 08:58:21.357331 23303 raft_consensus.cc:3054] T 41e7f0cc38c4414ab88c92c1cdeb3984 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:21.357440 23527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed79047393d2425f8ecff31ea08b9cad" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" is_pre_election: true
I20250109 08:58:21.357705 23380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed79047393d2425f8ecff31ea08b9cad" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" is_pre_election: true
I20250109 08:58:21.358232 23527 raft_consensus.cc:2388] T ed79047393d2425f8ecff31ea08b9cad P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 235ad6af04c34c88a5261daa3604fada in current term 1: Already voted for candidate ed1efbce9ee3401e99a5260edeae4e7c in this term.
I20250109 08:58:21.358855 23304 raft_consensus.cc:3054] T 9b245cb3666c421c884cf20161a22831 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:21.360502 23269 leader_election.cc:304] T ed79047393d2425f8ecff31ea08b9cad P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 235ad6af04c34c88a5261daa3604fada; no voters: 05a74e1eb2474b3d8b3a2da1ef127e01, ed1efbce9ee3401e99a5260edeae4e7c
I20250109 08:58:21.361392 23586 raft_consensus.cc:3054] T ed79047393d2425f8ecff31ea08b9cad P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:21.349050 23569 tablet_bootstrap.cc:492] T 08214ee9c4c14bda92bdc6fa8231c313 P 05a74e1eb2474b3d8b3a2da1ef127e01: No bootstrap required, opened a new log
I20250109 08:58:21.363543 23574 tablet_bootstrap.cc:654] T a54ca9f0d6e844e9a8b1d9cd0bb2a026 P 235ad6af04c34c88a5261daa3604fada: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:21.364146 23569 ts_tablet_manager.cc:1397] T 08214ee9c4c14bda92bdc6fa8231c313 P 05a74e1eb2474b3d8b3a2da1ef127e01: Time spent bootstrapping tablet: real 0.128s user 0.012s sys 0.000s
I20250109 08:58:21.356258 23666 leader_election.cc:290] T ed79047393d2425f8ecff31ea08b9cad P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829), ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365)
I20250109 08:58:21.350361 23202 catalog_manager.cc:5526] T 9b0bbf2e4a424fb5ac13a3b55627b048 P ed1efbce9ee3401e99a5260edeae4e7c reported cstate change: term changed from 0 to 1, leader changed from <none> to ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194). New cstate: current_term: 1 leader_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:21.366741 23569 raft_consensus.cc:357] T 08214ee9c4c14bda92bdc6fa8231c313 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:21.366483 23202 catalog_manager.cc:5526] T 34caa5988a734b65a2eb512747d833b7 P ed1efbce9ee3401e99a5260edeae4e7c reported cstate change: term changed from 0 to 1, leader changed from <none> to ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194). New cstate: current_term: 1 leader_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:21.356328 23658 consensus_queue.cc:237] T 6eef9bb3c645428d9f1cb843bae9a264 P ed1efbce9ee3401e99a5260edeae4e7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:21.368918 23303 raft_consensus.cc:2463] T 41e7f0cc38c4414ab88c92c1cdeb3984 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05a74e1eb2474b3d8b3a2da1ef127e01 in term 1.
I20250109 08:58:21.347190 23572 raft_consensus.cc:383] T 6eef9bb3c645428d9f1cb843bae9a264 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:21.370023 23572 raft_consensus.cc:738] T 6eef9bb3c645428d9f1cb843bae9a264 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf55383bef6649eb9e78cb2d2512dedc, State: Initialized, Role: FOLLOWER
I20250109 08:58:21.370699 23572 consensus_queue.cc:260] T 6eef9bb3c645428d9f1cb843bae9a264 P cf55383bef6649eb9e78cb2d2512dedc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:21.371596 23453 raft_consensus.cc:3054] T 6eef9bb3c645428d9f1cb843bae9a264 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:21.372856 23572 ts_tablet_manager.cc:1428] T 6eef9bb3c645428d9f1cb843bae9a264 P cf55383bef6649eb9e78cb2d2512dedc: Time spent starting tablet: real 0.041s user 0.003s sys 0.003s
I20250109 08:58:21.373764 23572 tablet_bootstrap.cc:492] T d5fa255ab5f24750a0afa6d81ec869f2 P cf55383bef6649eb9e78cb2d2512dedc: Bootstrap starting.
I20250109 08:58:21.374627 23574 tablet_bootstrap.cc:492] T a54ca9f0d6e844e9a8b1d9cd0bb2a026 P 235ad6af04c34c88a5261daa3604fada: No bootstrap required, opened a new log
I20250109 08:58:21.375119 23574 ts_tablet_manager.cc:1397] T a54ca9f0d6e844e9a8b1d9cd0bb2a026 P 235ad6af04c34c88a5261daa3604fada: Time spent bootstrapping tablet: real 0.026s user 0.014s sys 0.002s
I20250109 08:58:21.367461 23569 raft_consensus.cc:383] T 08214ee9c4c14bda92bdc6fa8231c313 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:21.375594 23569 raft_consensus.cc:738] T 08214ee9c4c14bda92bdc6fa8231c313 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05a74e1eb2474b3d8b3a2da1ef127e01, State: Initialized, Role: FOLLOWER
I20250109 08:58:21.376176 23569 consensus_queue.cc:260] T 08214ee9c4c14bda92bdc6fa8231c313 P 05a74e1eb2474b3d8b3a2da1ef127e01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:21.377357 23586 raft_consensus.cc:2743] T ed79047393d2425f8ecff31ea08b9cad P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:58:21.378821 23304 raft_consensus.cc:2463] T 9b245cb3666c421c884cf20161a22831 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05a74e1eb2474b3d8b3a2da1ef127e01 in term 1.
I20250109 08:58:21.351663 23580 tablet_bootstrap.cc:492] T 8de6b2fb09bc46ce9e4033af06dffd04 P ed1efbce9ee3401e99a5260edeae4e7c: No bootstrap required, opened a new log
I20250109 08:58:21.381445 23580 ts_tablet_manager.cc:1397] T 8de6b2fb09bc46ce9e4033af06dffd04 P ed1efbce9ee3401e99a5260edeae4e7c: Time spent bootstrapping tablet: real 0.078s user 0.010s sys 0.000s
I20250109 08:58:21.381909 23635 consensus_queue.cc:1035] T ed79047393d2425f8ecff31ea08b9cad P ed1efbce9ee3401e99a5260edeae4e7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20250109 08:58:21.356316 23494 leader_election.cc:343] T 9b245cb3666c421c884cf20161a22831 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551): Illegal state: must be running to vote when last-logged opid is not known
W20250109 08:58:21.382928 23494 leader_election.cc:343] T 41e7f0cc38c4414ab88c92c1cdeb3984 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:21.380973 23569 ts_tablet_manager.cc:1428] T 08214ee9c4c14bda92bdc6fa8231c313 P 05a74e1eb2474b3d8b3a2da1ef127e01: Time spent starting tablet: real 0.017s user 0.005s sys 0.000s
I20250109 08:58:21.386691 23494 leader_election.cc:304] T 41e7f0cc38c4414ab88c92c1cdeb3984 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01, 235ad6af04c34c88a5261daa3604fada; no voters: ed1efbce9ee3401e99a5260edeae4e7c
I20250109 08:58:21.388854 23631 raft_consensus.cc:2798] T 41e7f0cc38c4414ab88c92c1cdeb3984 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:58:21.389324 23631 raft_consensus.cc:695] T 41e7f0cc38c4414ab88c92c1cdeb3984 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 LEADER]: Becoming Leader. State: Replica: 05a74e1eb2474b3d8b3a2da1ef127e01, State: Running, Role: LEADER
I20250109 08:58:21.377698 23574 raft_consensus.cc:357] T a54ca9f0d6e844e9a8b1d9cd0bb2a026 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:21.390111 23631 consensus_queue.cc:237] T 41e7f0cc38c4414ab88c92c1cdeb3984 P 05a74e1eb2474b3d8b3a2da1ef127e01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:21.391983 23574 raft_consensus.cc:383] T a54ca9f0d6e844e9a8b1d9cd0bb2a026 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:21.392320 23574 raft_consensus.cc:738] T a54ca9f0d6e844e9a8b1d9cd0bb2a026 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Initialized, Role: FOLLOWER
I20250109 08:58:21.393112 23574 consensus_queue.cc:260] T a54ca9f0d6e844e9a8b1d9cd0bb2a026 P 235ad6af04c34c88a5261daa3604fada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:21.399300 23666 consensus_queue.cc:1035] T 3208a99957554f6db341654856980c8a P 235ad6af04c34c88a5261daa3604fada [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:58:21.404261 23614 consensus_queue.cc:1035] T 1340cf8ee2cb423f875276137da17e8b P 235ad6af04c34c88a5261daa3604fada [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250109 08:58:21.405970 23574 ts_tablet_manager.cc:1428] T a54ca9f0d6e844e9a8b1d9cd0bb2a026 P 235ad6af04c34c88a5261daa3604fada: Time spent starting tablet: real 0.030s user 0.005s sys 0.001s
I20250109 08:58:21.406903 23574 tablet_bootstrap.cc:492] T 5ccc0d42d4a148d7819366cea0b754a7 P 235ad6af04c34c88a5261daa3604fada: Bootstrap starting.
I20250109 08:58:21.386914 23569 tablet_bootstrap.cc:492] T 1c5b5beea4a840eb9677097cacee3408 P 05a74e1eb2474b3d8b3a2da1ef127e01: Bootstrap starting.
I20250109 08:58:21.385308 23572 tablet_bootstrap.cc:654] T d5fa255ab5f24750a0afa6d81ec869f2 P cf55383bef6649eb9e78cb2d2512dedc: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:21.408172 23379 raft_consensus.cc:3054] T ea2b0762940e44dbbf69b9a163ace846 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:21.412917 23574 tablet_bootstrap.cc:654] T 5ccc0d42d4a148d7819366cea0b754a7 P 235ad6af04c34c88a5261daa3604fada: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:21.415243 23569 tablet_bootstrap.cc:654] T 1c5b5beea4a840eb9677097cacee3408 P 05a74e1eb2474b3d8b3a2da1ef127e01: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:21.380234 23453 raft_consensus.cc:2463] T 6eef9bb3c645428d9f1cb843bae9a264 P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed1efbce9ee3401e99a5260edeae4e7c in term 1.
I20250109 08:58:21.404325 23611 consensus_queue.cc:1035] T ed79047393d2425f8ecff31ea08b9cad P ed1efbce9ee3401e99a5260edeae4e7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:58:21.384919 23580 raft_consensus.cc:357] T 8de6b2fb09bc46ce9e4033af06dffd04 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:21.427961 23580 raft_consensus.cc:383] T 8de6b2fb09bc46ce9e4033af06dffd04 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:21.428352 23580 raft_consensus.cc:738] T 8de6b2fb09bc46ce9e4033af06dffd04 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed1efbce9ee3401e99a5260edeae4e7c, State: Initialized, Role: FOLLOWER
I20250109 08:58:21.397249 23494 leader_election.cc:304] T 9b245cb3666c421c884cf20161a22831 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01, 235ad6af04c34c88a5261daa3604fada; no voters: cf55383bef6649eb9e78cb2d2512dedc
I20250109 08:58:21.429946 23631 raft_consensus.cc:2798] T 9b245cb3666c421c884cf20161a22831 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:58:21.430673 23631 raft_consensus.cc:695] T 9b245cb3666c421c884cf20161a22831 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 LEADER]: Becoming Leader. State: Replica: 05a74e1eb2474b3d8b3a2da1ef127e01, State: Running, Role: LEADER
I20250109 08:58:21.431556 23631 consensus_queue.cc:237] T 9b245cb3666c421c884cf20161a22831 P 05a74e1eb2474b3d8b3a2da1ef127e01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:21.432864 23580 consensus_queue.cc:260] T 8de6b2fb09bc46ce9e4033af06dffd04 P ed1efbce9ee3401e99a5260edeae4e7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:21.436336 23580 ts_tablet_manager.cc:1428] T 8de6b2fb09bc46ce9e4033af06dffd04 P ed1efbce9ee3401e99a5260edeae4e7c: Time spent starting tablet: real 0.055s user 0.005s sys 0.000s
I20250109 08:58:21.441586 23580 tablet_bootstrap.cc:492] T 41e7f0cc38c4414ab88c92c1cdeb3984 P ed1efbce9ee3401e99a5260edeae4e7c: Bootstrap starting.
I20250109 08:58:21.439242 23569 tablet_bootstrap.cc:492] T 1c5b5beea4a840eb9677097cacee3408 P 05a74e1eb2474b3d8b3a2da1ef127e01: No bootstrap required, opened a new log
I20250109 08:58:21.442221 23569 ts_tablet_manager.cc:1397] T 1c5b5beea4a840eb9677097cacee3408 P 05a74e1eb2474b3d8b3a2da1ef127e01: Time spent bootstrapping tablet: real 0.056s user 0.011s sys 0.000s
I20250109 08:58:21.444962 23569 raft_consensus.cc:357] T 1c5b5beea4a840eb9677097cacee3408 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:21.447824 23569 raft_consensus.cc:383] T 1c5b5beea4a840eb9677097cacee3408 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:21.448164 23569 raft_consensus.cc:738] T 1c5b5beea4a840eb9677097cacee3408 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05a74e1eb2474b3d8b3a2da1ef127e01, State: Initialized, Role: FOLLOWER
I20250109 08:58:21.448966 23569 consensus_queue.cc:260] T 1c5b5beea4a840eb9677097cacee3408 P 05a74e1eb2474b3d8b3a2da1ef127e01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:21.453894 23569 ts_tablet_manager.cc:1428] T 1c5b5beea4a840eb9677097cacee3408 P 05a74e1eb2474b3d8b3a2da1ef127e01: Time spent starting tablet: real 0.011s user 0.005s sys 0.000s
I20250109 08:58:21.455571 23569 tablet_bootstrap.cc:492] T b572a6a3dfc04ea199bc703ddad8af67 P 05a74e1eb2474b3d8b3a2da1ef127e01: Bootstrap starting.
I20250109 08:58:21.470952 23675 consensus_queue.cc:1035] T 8c5e3d0155d140c4b32f162d3173f5c9 P cf55383bef6649eb9e78cb2d2512dedc [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:58:21.473884 23572 tablet_bootstrap.cc:492] T d5fa255ab5f24750a0afa6d81ec869f2 P cf55383bef6649eb9e78cb2d2512dedc: No bootstrap required, opened a new log
I20250109 08:58:21.473819 23634 consensus_queue.cc:1035] T 82ea7dc042dd4d05b260d508f4422c3c P 05a74e1eb2474b3d8b3a2da1ef127e01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:58:21.474426 23572 ts_tablet_manager.cc:1397] T d5fa255ab5f24750a0afa6d81ec869f2 P cf55383bef6649eb9e78cb2d2512dedc: Time spent bootstrapping tablet: real 0.101s user 0.018s sys 0.007s
I20250109 08:58:21.476948 23572 raft_consensus.cc:357] T d5fa255ab5f24750a0afa6d81ec869f2 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:21.478788 23572 raft_consensus.cc:383] T d5fa255ab5f24750a0afa6d81ec869f2 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:21.478704 23675 consensus_queue.cc:1035] T 8c5e3d0155d140c4b32f162d3173f5c9 P cf55383bef6649eb9e78cb2d2512dedc [LEADER]: Connected to new peer: Peer: permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250109 08:58:21.479501 23572 raft_consensus.cc:738] T d5fa255ab5f24750a0afa6d81ec869f2 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf55383bef6649eb9e78cb2d2512dedc, State: Initialized, Role: FOLLOWER
I20250109 08:58:21.480266 23572 consensus_queue.cc:260] T d5fa255ab5f24750a0afa6d81ec869f2 P cf55383bef6649eb9e78cb2d2512dedc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:21.485091 23586 raft_consensus.cc:491] T 4df64aae69ca41dc91286cae1603000f P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:21.489750 23572 ts_tablet_manager.cc:1428] T d5fa255ab5f24750a0afa6d81ec869f2 P cf55383bef6649eb9e78cb2d2512dedc: Time spent starting tablet: real 0.015s user 0.008s sys 0.000s
I20250109 08:58:21.490686 23572 tablet_bootstrap.cc:492] T 9b0bbf2e4a424fb5ac13a3b55627b048 P cf55383bef6649eb9e78cb2d2512dedc: Bootstrap starting.
I20250109 08:58:21.491293 23580 tablet_bootstrap.cc:654] T 41e7f0cc38c4414ab88c92c1cdeb3984 P ed1efbce9ee3401e99a5260edeae4e7c: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:21.487694 23586 raft_consensus.cc:513] T 4df64aae69ca41dc91286cae1603000f P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:21.498515 23572 tablet_bootstrap.cc:654] T 9b0bbf2e4a424fb5ac13a3b55627b048 P cf55383bef6649eb9e78cb2d2512dedc: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:21.515705 23675 raft_consensus.cc:491] T 940653b6d55c46439827697b9aadf16b P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:21.506814 23586 leader_election.cc:290] T 4df64aae69ca41dc91286cae1603000f P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829), ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365)
I20250109 08:58:21.520889 23528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4df64aae69ca41dc91286cae1603000f" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" is_pre_election: true
I20250109 08:58:21.524250 23201 catalog_manager.cc:5526] T 41e7f0cc38c4414ab88c92c1cdeb3984 P 05a74e1eb2474b3d8b3a2da1ef127e01 reported cstate change: term changed from 0 to 1, leader changed from <none> to 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196). New cstate: current_term: 1 leader_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:21.526549 23631 raft_consensus.cc:491] T 1c5b5beea4a840eb9677097cacee3408 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:21.527832 23631 raft_consensus.cc:513] T 1c5b5beea4a840eb9677097cacee3408 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
W20250109 08:58:21.528522 23269 leader_election.cc:343] T 4df64aae69ca41dc91286cae1603000f P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:21.516278 23675 raft_consensus.cc:513] T 940653b6d55c46439827697b9aadf16b P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
W20250109 08:58:21.533349 23420 consensus_peers.cc:487] T 5ccc0d42d4a148d7819366cea0b754a7 P cf55383bef6649eb9e78cb2d2512dedc -> Peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Couldn't send request to peer 235ad6af04c34c88a5261daa3604fada. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:21.526891 23606 consensus_queue.cc:1035] T 3208a99957554f6db341654856980c8a P 235ad6af04c34c88a5261daa3604fada [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:58:21.534344 23675 leader_election.cc:290] T 940653b6d55c46439827697b9aadf16b P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365), 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829)
I20250109 08:58:21.535811 23528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "940653b6d55c46439827697b9aadf16b" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" is_pre_election: true
W20250109 08:58:21.537235 23418 leader_election.cc:343] T 940653b6d55c46439827697b9aadf16b P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:21.538664 23379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5b5beea4a840eb9677097cacee3408" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" is_pre_election: true
I20250109 08:58:21.541292 23378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4df64aae69ca41dc91286cae1603000f" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" is_pre_election: true
W20250109 08:58:21.541566 23494 leader_election.cc:343] T 1c5b5beea4a840eb9677097cacee3408 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:21.532698 23303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5b5beea4a840eb9677097cacee3408" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada" is_pre_election: true
I20250109 08:58:21.534364 23631 leader_election.cc:290] T 1c5b5beea4a840eb9677097cacee3408 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691), ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365)
I20250109 08:58:21.544217 23378 raft_consensus.cc:2463] T 4df64aae69ca41dc91286cae1603000f P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 235ad6af04c34c88a5261daa3604fada in term 0.
I20250109 08:58:21.571391 23574 tablet_bootstrap.cc:492] T 5ccc0d42d4a148d7819366cea0b754a7 P 235ad6af04c34c88a5261daa3604fada: No bootstrap required, opened a new log
I20250109 08:58:21.571931 23574 ts_tablet_manager.cc:1397] T 5ccc0d42d4a148d7819366cea0b754a7 P 235ad6af04c34c88a5261daa3604fada: Time spent bootstrapping tablet: real 0.165s user 0.021s sys 0.014s
I20250109 08:58:21.565762 23634 raft_consensus.cc:491] T 08214ee9c4c14bda92bdc6fa8231c313 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:21.574431 23574 raft_consensus.cc:357] T 5ccc0d42d4a148d7819366cea0b754a7 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:21.574985 23574 raft_consensus.cc:383] T 5ccc0d42d4a148d7819366cea0b754a7 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:21.574635 23634 raft_consensus.cc:513] T 08214ee9c4c14bda92bdc6fa8231c313 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:21.575157 23574 raft_consensus.cc:738] T 5ccc0d42d4a148d7819366cea0b754a7 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Initialized, Role: FOLLOWER
I20250109 08:58:21.575662 23574 consensus_queue.cc:260] T 5ccc0d42d4a148d7819366cea0b754a7 P 235ad6af04c34c88a5261daa3604fada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:21.576601 23634 leader_election.cc:290] T 08214ee9c4c14bda92bdc6fa8231c313 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691), ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365)
I20250109 08:58:21.577296 23574 ts_tablet_manager.cc:1428] T 5ccc0d42d4a148d7819366cea0b754a7 P 235ad6af04c34c88a5261daa3604fada: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20250109 08:58:21.577787 23379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "940653b6d55c46439827697b9aadf16b" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" is_pre_election: true
I20250109 08:58:21.578029 23574 tablet_bootstrap.cc:492] T 24ce131f7bb143bea1581d5b0c2a7109 P 235ad6af04c34c88a5261daa3604fada: Bootstrap starting.
W20250109 08:58:21.579149 23420 leader_election.cc:343] T 940653b6d55c46439827697b9aadf16b P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:21.579578 23420 leader_election.cc:304] T 940653b6d55c46439827697b9aadf16b P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf55383bef6649eb9e78cb2d2512dedc; no voters: 05a74e1eb2474b3d8b3a2da1ef127e01, ed1efbce9ee3401e99a5260edeae4e7c
I20250109 08:58:21.579968 23379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08214ee9c4c14bda92bdc6fa8231c313" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" is_pre_election: true
I20250109 08:58:21.580667 23675 raft_consensus.cc:2743] T 940653b6d55c46439827697b9aadf16b P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20250109 08:58:21.581295 23494 leader_election.cc:343] T 08214ee9c4c14bda92bdc6fa8231c313 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:21.583563 23574 tablet_bootstrap.cc:654] T 24ce131f7bb143bea1581d5b0c2a7109 P 235ad6af04c34c88a5261daa3604fada: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:21.585196 23271 leader_election.cc:304] T 4df64aae69ca41dc91286cae1603000f P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 235ad6af04c34c88a5261daa3604fada, ed1efbce9ee3401e99a5260edeae4e7c; no voters: 05a74e1eb2474b3d8b3a2da1ef127e01
I20250109 08:58:21.588584 23606 raft_consensus.cc:2798] T 4df64aae69ca41dc91286cae1603000f P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:58:21.589498 23606 raft_consensus.cc:491] T 4df64aae69ca41dc91286cae1603000f P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:58:21.589820 23606 raft_consensus.cc:3054] T 4df64aae69ca41dc91286cae1603000f P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:21.591032 23202 catalog_manager.cc:5526] T 6eef9bb3c645428d9f1cb843bae9a264 P ed1efbce9ee3401e99a5260edeae4e7c reported cstate change: term changed from 0 to 1, leader changed from <none> to ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194). New cstate: current_term: 1 leader_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:21.595759 23580 tablet_bootstrap.cc:492] T 41e7f0cc38c4414ab88c92c1cdeb3984 P ed1efbce9ee3401e99a5260edeae4e7c: No bootstrap required, opened a new log
I20250109 08:58:21.596253 23580 ts_tablet_manager.cc:1397] T 41e7f0cc38c4414ab88c92c1cdeb3984 P ed1efbce9ee3401e99a5260edeae4e7c: Time spent bootstrapping tablet: real 0.155s user 0.017s sys 0.039s
I20250109 08:58:21.598198 23675 consensus_queue.cc:1035] T 5ccc0d42d4a148d7819366cea0b754a7 P cf55383bef6649eb9e78cb2d2512dedc [LEADER]: Connected to new peer: Peer: permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:58:21.600426 23580 raft_consensus.cc:357] T 41e7f0cc38c4414ab88c92c1cdeb3984 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:21.601256 23580 raft_consensus.cc:383] T 41e7f0cc38c4414ab88c92c1cdeb3984 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:21.601519 23580 raft_consensus.cc:738] T 41e7f0cc38c4414ab88c92c1cdeb3984 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed1efbce9ee3401e99a5260edeae4e7c, State: Initialized, Role: FOLLOWER
I20250109 08:58:21.602185 23580 consensus_queue.cc:260] T 41e7f0cc38c4414ab88c92c1cdeb3984 P ed1efbce9ee3401e99a5260edeae4e7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:21.606477 23580 ts_tablet_manager.cc:1428] T 41e7f0cc38c4414ab88c92c1cdeb3984 P ed1efbce9ee3401e99a5260edeae4e7c: Time spent starting tablet: real 0.010s user 0.005s sys 0.000s
I20250109 08:58:21.607410 23580 tablet_bootstrap.cc:492] T 5831288451724d4ca1114ec5e5f01b78 P ed1efbce9ee3401e99a5260edeae4e7c: Bootstrap starting.
I20250109 08:58:21.609318 23606 raft_consensus.cc:513] T 4df64aae69ca41dc91286cae1603000f P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:21.616586 23528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4df64aae69ca41dc91286cae1603000f" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01"
I20250109 08:58:21.588594 23569 tablet_bootstrap.cc:654] T b572a6a3dfc04ea199bc703ddad8af67 P 05a74e1eb2474b3d8b3a2da1ef127e01: Neither blocks nor log segments found. Creating new log.
W20250109 08:58:21.620191 23494 leader_election.cc:343] T 1c5b5beea4a840eb9677097cacee3408 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:21.620718 23494 leader_election.cc:304] T 1c5b5beea4a840eb9677097cacee3408 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01; no voters: 235ad6af04c34c88a5261daa3604fada, ed1efbce9ee3401e99a5260edeae4e7c
W20250109 08:58:21.621412 23269 leader_election.cc:343] T 4df64aae69ca41dc91286cae1603000f P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:21.622383 23634 raft_consensus.cc:2743] T 1c5b5beea4a840eb9677097cacee3408 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:58:21.623013 23634 consensus_queue.cc:1035] T 82ea7dc042dd4d05b260d508f4422c3c P 05a74e1eb2474b3d8b3a2da1ef127e01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:58:21.628355 23303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08214ee9c4c14bda92bdc6fa8231c313" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada" is_pre_election: true
I20250109 08:58:21.633661 23572 tablet_bootstrap.cc:492] T 9b0bbf2e4a424fb5ac13a3b55627b048 P cf55383bef6649eb9e78cb2d2512dedc: No bootstrap required, opened a new log
W20250109 08:58:21.634099 23494 leader_election.cc:343] T 08214ee9c4c14bda92bdc6fa8231c313 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:21.634583 23494 leader_election.cc:304] T 08214ee9c4c14bda92bdc6fa8231c313 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01; no voters: 235ad6af04c34c88a5261daa3604fada, ed1efbce9ee3401e99a5260edeae4e7c
I20250109 08:58:21.634145 23572 ts_tablet_manager.cc:1397] T 9b0bbf2e4a424fb5ac13a3b55627b048 P cf55383bef6649eb9e78cb2d2512dedc: Time spent bootstrapping tablet: real 0.144s user 0.034s sys 0.028s
I20250109 08:58:21.636399 23606 leader_election.cc:290] T 4df64aae69ca41dc91286cae1603000f P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 election: Requested vote from peers 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829), ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365)
I20250109 08:58:21.638204 23378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4df64aae69ca41dc91286cae1603000f" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c"
I20250109 08:58:21.639026 23378 raft_consensus.cc:3054] T 4df64aae69ca41dc91286cae1603000f P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:21.636662 23631 raft_consensus.cc:2743] T 08214ee9c4c14bda92bdc6fa8231c313 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:58:21.645697 23580 tablet_bootstrap.cc:654] T 5831288451724d4ca1114ec5e5f01b78 P ed1efbce9ee3401e99a5260edeae4e7c: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:21.649477 23574 tablet_bootstrap.cc:492] T 24ce131f7bb143bea1581d5b0c2a7109 P 235ad6af04c34c88a5261daa3604fada: No bootstrap required, opened a new log
I20250109 08:58:21.650247 23572 raft_consensus.cc:357] T 9b0bbf2e4a424fb5ac13a3b55627b048 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:21.650810 23572 raft_consensus.cc:383] T 9b0bbf2e4a424fb5ac13a3b55627b048 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:21.651022 23572 raft_consensus.cc:738] T 9b0bbf2e4a424fb5ac13a3b55627b048 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf55383bef6649eb9e78cb2d2512dedc, State: Initialized, Role: FOLLOWER
I20250109 08:58:21.651736 23572 consensus_queue.cc:260] T 9b0bbf2e4a424fb5ac13a3b55627b048 P cf55383bef6649eb9e78cb2d2512dedc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:21.652257 23574 ts_tablet_manager.cc:1397] T 24ce131f7bb143bea1581d5b0c2a7109 P 235ad6af04c34c88a5261daa3604fada: Time spent bootstrapping tablet: real 0.074s user 0.021s sys 0.015s
I20250109 08:58:21.656662 23572 ts_tablet_manager.cc:1428] T 9b0bbf2e4a424fb5ac13a3b55627b048 P cf55383bef6649eb9e78cb2d2512dedc: Time spent starting tablet: real 0.021s user 0.005s sys 0.001s
I20250109 08:58:21.659510 23378 raft_consensus.cc:2463] T 4df64aae69ca41dc91286cae1603000f P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 235ad6af04c34c88a5261daa3604fada in term 1.
I20250109 08:58:21.660698 23572 tablet_bootstrap.cc:492] T b572a6a3dfc04ea199bc703ddad8af67 P cf55383bef6649eb9e78cb2d2512dedc: Bootstrap starting.
I20250109 08:58:21.660027 23199 catalog_manager.cc:5526] T 9b245cb3666c421c884cf20161a22831 P 05a74e1eb2474b3d8b3a2da1ef127e01 reported cstate change: term changed from 0 to 1, leader changed from <none> to 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196). New cstate: current_term: 1 leader_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:21.664171 23271 leader_election.cc:304] T 4df64aae69ca41dc91286cae1603000f P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 235ad6af04c34c88a5261daa3604fada, ed1efbce9ee3401e99a5260edeae4e7c; no voters: 05a74e1eb2474b3d8b3a2da1ef127e01
I20250109 08:58:21.665529 23606 raft_consensus.cc:2798] T 4df64aae69ca41dc91286cae1603000f P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:58:21.666213 23606 raft_consensus.cc:695] T 4df64aae69ca41dc91286cae1603000f P 235ad6af04c34c88a5261daa3604fada [term 1 LEADER]: Becoming Leader. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Running, Role: LEADER
I20250109 08:58:21.667219 23606 consensus_queue.cc:237] T 4df64aae69ca41dc91286cae1603000f P 235ad6af04c34c88a5261daa3604fada [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:21.668486 23574 raft_consensus.cc:357] T 24ce131f7bb143bea1581d5b0c2a7109 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:21.669176 23574 raft_consensus.cc:383] T 24ce131f7bb143bea1581d5b0c2a7109 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:21.669461 23574 raft_consensus.cc:738] T 24ce131f7bb143bea1581d5b0c2a7109 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Initialized, Role: FOLLOWER
I20250109 08:58:21.670570 23574 consensus_queue.cc:260] T 24ce131f7bb143bea1581d5b0c2a7109 P 235ad6af04c34c88a5261daa3604fada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:21.672674 23574 ts_tablet_manager.cc:1428] T 24ce131f7bb143bea1581d5b0c2a7109 P 235ad6af04c34c88a5261daa3604fada: Time spent starting tablet: real 0.020s user 0.006s sys 0.000s
I20250109 08:58:21.673527 23574 tablet_bootstrap.cc:492] T d87d3b4755344b82b0aec7ffd1ce785c P 235ad6af04c34c88a5261daa3604fada: Bootstrap starting.
I20250109 08:58:21.676263 23572 tablet_bootstrap.cc:654] T b572a6a3dfc04ea199bc703ddad8af67 P cf55383bef6649eb9e78cb2d2512dedc: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:21.683714 23574 tablet_bootstrap.cc:654] T d87d3b4755344b82b0aec7ffd1ce785c P 235ad6af04c34c88a5261daa3604fada: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:21.687556 23569 tablet_bootstrap.cc:492] T b572a6a3dfc04ea199bc703ddad8af67 P 05a74e1eb2474b3d8b3a2da1ef127e01: No bootstrap required, opened a new log
I20250109 08:58:21.688031 23569 ts_tablet_manager.cc:1397] T b572a6a3dfc04ea199bc703ddad8af67 P 05a74e1eb2474b3d8b3a2da1ef127e01: Time spent bootstrapping tablet: real 0.233s user 0.022s sys 0.007s
I20250109 08:58:21.692103 23569 raft_consensus.cc:357] T b572a6a3dfc04ea199bc703ddad8af67 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:21.694555 23569 raft_consensus.cc:383] T b572a6a3dfc04ea199bc703ddad8af67 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:21.694808 23569 raft_consensus.cc:738] T b572a6a3dfc04ea199bc703ddad8af67 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05a74e1eb2474b3d8b3a2da1ef127e01, State: Initialized, Role: FOLLOWER
I20250109 08:58:21.695451 23569 consensus_queue.cc:260] T b572a6a3dfc04ea199bc703ddad8af67 P 05a74e1eb2474b3d8b3a2da1ef127e01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:21.698014 23569 ts_tablet_manager.cc:1428] T b572a6a3dfc04ea199bc703ddad8af67 P 05a74e1eb2474b3d8b3a2da1ef127e01: Time spent starting tablet: real 0.010s user 0.005s sys 0.001s
I20250109 08:58:21.698693 23606 raft_consensus.cc:491] T 5ccc0d42d4a148d7819366cea0b754a7 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:21.698155 23200 catalog_manager.cc:5526] T 4df64aae69ca41dc91286cae1603000f P 235ad6af04c34c88a5261daa3604fada reported cstate change: term changed from 0 to 1, leader changed from <none> to 235ad6af04c34c88a5261daa3604fada (127.19.226.193). New cstate: current_term: 1 leader_uuid: "235ad6af04c34c88a5261daa3604fada" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:21.699357 23606 raft_consensus.cc:513] T 5ccc0d42d4a148d7819366cea0b754a7 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:21.700752 23569 tablet_bootstrap.cc:492] T 2ab70546dda143b5b5db15c3875aa348 P 05a74e1eb2474b3d8b3a2da1ef127e01: Bootstrap starting.
I20250109 08:58:21.700845 23611 raft_consensus.cc:491] T 8de6b2fb09bc46ce9e4033af06dffd04 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:21.701385 23611 raft_consensus.cc:513] T 8de6b2fb09bc46ce9e4033af06dffd04 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:21.702805 23580 tablet_bootstrap.cc:492] T 5831288451724d4ca1114ec5e5f01b78 P ed1efbce9ee3401e99a5260edeae4e7c: No bootstrap required, opened a new log
I20250109 08:58:21.703303 23580 ts_tablet_manager.cc:1397] T 5831288451724d4ca1114ec5e5f01b78 P ed1efbce9ee3401e99a5260edeae4e7c: Time spent bootstrapping tablet: real 0.096s user 0.018s sys 0.008s
I20250109 08:58:21.704825 23454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8de6b2fb09bc46ce9e4033af06dffd04" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc" is_pre_election: true
I20250109 08:58:21.706491 23569 tablet_bootstrap.cc:654] T 2ab70546dda143b5b5db15c3875aa348 P 05a74e1eb2474b3d8b3a2da1ef127e01: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:21.707664 23527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8de6b2fb09bc46ce9e4033af06dffd04" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" is_pre_election: true
I20250109 08:58:21.705791 23580 raft_consensus.cc:357] T 5831288451724d4ca1114ec5e5f01b78 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:21.708838 23580 raft_consensus.cc:383] T 5831288451724d4ca1114ec5e5f01b78 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:21.709101 23580 raft_consensus.cc:738] T 5831288451724d4ca1114ec5e5f01b78 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed1efbce9ee3401e99a5260edeae4e7c, State: Initialized, Role: FOLLOWER
I20250109 08:58:21.709707 23580 consensus_queue.cc:260] T 5831288451724d4ca1114ec5e5f01b78 P ed1efbce9ee3401e99a5260edeae4e7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:21.712468 23580 ts_tablet_manager.cc:1428] T 5831288451724d4ca1114ec5e5f01b78 P ed1efbce9ee3401e99a5260edeae4e7c: Time spent starting tablet: real 0.009s user 0.006s sys 0.000s
I20250109 08:58:21.716104 23574 tablet_bootstrap.cc:492] T d87d3b4755344b82b0aec7ffd1ce785c P 235ad6af04c34c88a5261daa3604fada: No bootstrap required, opened a new log
I20250109 08:58:21.716653 23574 ts_tablet_manager.cc:1397] T d87d3b4755344b82b0aec7ffd1ce785c P 235ad6af04c34c88a5261daa3604fada: Time spent bootstrapping tablet: real 0.043s user 0.007s sys 0.004s
I20250109 08:58:21.718114 23454 raft_consensus.cc:3054] T 9b0bbf2e4a424fb5ac13a3b55627b048 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:21.719370 23574 raft_consensus.cc:357] T d87d3b4755344b82b0aec7ffd1ce785c P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:21.720103 23574 raft_consensus.cc:383] T d87d3b4755344b82b0aec7ffd1ce785c P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:21.720450 23574 raft_consensus.cc:738] T d87d3b4755344b82b0aec7ffd1ce785c P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Initialized, Role: FOLLOWER
I20250109 08:58:21.703295 23611 leader_election.cc:290] T 8de6b2fb09bc46ce9e4033af06dffd04 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551), 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829)
I20250109 08:58:21.721366 23574 consensus_queue.cc:260] T d87d3b4755344b82b0aec7ffd1ce785c P 235ad6af04c34c88a5261daa3604fada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
W20250109 08:58:21.716233 23346 leader_election.cc:343] T 8de6b2fb09bc46ce9e4033af06dffd04 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551): Illegal state: must be running to vote when last-logged opid is not known
W20250109 08:58:21.722662 23344 leader_election.cc:343] T 8de6b2fb09bc46ce9e4033af06dffd04 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:21.723531 23344 leader_election.cc:304] T 8de6b2fb09bc46ce9e4033af06dffd04 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ed1efbce9ee3401e99a5260edeae4e7c; no voters: 05a74e1eb2474b3d8b3a2da1ef127e01, cf55383bef6649eb9e78cb2d2512dedc
I20250109 08:58:21.723922 23574 ts_tablet_manager.cc:1428] T d87d3b4755344b82b0aec7ffd1ce785c P 235ad6af04c34c88a5261daa3604fada: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20250109 08:58:21.724674 23611 raft_consensus.cc:2743] T 8de6b2fb09bc46ce9e4033af06dffd04 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:58:21.724931 23574 tablet_bootstrap.cc:492] T e5251927614642639bd46ecd5373d2a4 P 235ad6af04c34c88a5261daa3604fada: Bootstrap starting.
I20250109 08:58:21.710901 23572 tablet_bootstrap.cc:492] T b572a6a3dfc04ea199bc703ddad8af67 P cf55383bef6649eb9e78cb2d2512dedc: No bootstrap required, opened a new log
I20250109 08:58:21.730604 23606 leader_election.cc:290] T 5ccc0d42d4a148d7819366cea0b754a7 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551), 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829)
I20250109 08:58:21.730583 23580 tablet_bootstrap.cc:492] T 1c5b5beea4a840eb9677097cacee3408 P ed1efbce9ee3401e99a5260edeae4e7c: Bootstrap starting.
I20250109 08:58:21.731326 23572 ts_tablet_manager.cc:1397] T b572a6a3dfc04ea199bc703ddad8af67 P cf55383bef6649eb9e78cb2d2512dedc: Time spent bootstrapping tablet: real 0.071s user 0.009s sys 0.006s
I20250109 08:58:21.730206 23453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ccc0d42d4a148d7819366cea0b754a7" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc" is_pre_election: true
I20250109 08:58:21.730139 23635 consensus_queue.cc:1035] T 9b0bbf2e4a424fb5ac13a3b55627b048 P ed1efbce9ee3401e99a5260edeae4e7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250109 08:58:21.733963 23572 raft_consensus.cc:357] T b572a6a3dfc04ea199bc703ddad8af67 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:21.734671 23527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ccc0d42d4a148d7819366cea0b754a7" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" is_pre_election: true
I20250109 08:58:21.735404 23606 raft_consensus.cc:491] T a54ca9f0d6e844e9a8b1d9cd0bb2a026 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:21.736526 23269 leader_election.cc:304] T 5ccc0d42d4a148d7819366cea0b754a7 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 235ad6af04c34c88a5261daa3604fada; no voters: 05a74e1eb2474b3d8b3a2da1ef127e01, cf55383bef6649eb9e78cb2d2512dedc
I20250109 08:58:21.737062 23635 consensus_queue.cc:1035] T 9b0bbf2e4a424fb5ac13a3b55627b048 P ed1efbce9ee3401e99a5260edeae4e7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:58:21.737715 23586 raft_consensus.cc:2743] T 5ccc0d42d4a148d7819366cea0b754a7 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:58:21.738170 23580 tablet_bootstrap.cc:654] T 1c5b5beea4a840eb9677097cacee3408 P ed1efbce9ee3401e99a5260edeae4e7c: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:21.736543 23606 raft_consensus.cc:513] T a54ca9f0d6e844e9a8b1d9cd0bb2a026 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:21.734642 23572 raft_consensus.cc:383] T b572a6a3dfc04ea199bc703ddad8af67 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:21.746074 23378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a54ca9f0d6e844e9a8b1d9cd0bb2a026" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" is_pre_election: true
W20250109 08:58:21.747815 23271 leader_election.cc:343] T a54ca9f0d6e844e9a8b1d9cd0bb2a026 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:21.750615 23606 leader_election.cc:290] T a54ca9f0d6e844e9a8b1d9cd0bb2a026 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365), cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551)
I20250109 08:58:21.751339 23454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a54ca9f0d6e844e9a8b1d9cd0bb2a026" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc" is_pre_election: true
W20250109 08:58:21.752856 23271 leader_election.cc:343] T a54ca9f0d6e844e9a8b1d9cd0bb2a026 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:21.753263 23271 leader_election.cc:304] T a54ca9f0d6e844e9a8b1d9cd0bb2a026 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 235ad6af04c34c88a5261daa3604fada; no voters: cf55383bef6649eb9e78cb2d2512dedc, ed1efbce9ee3401e99a5260edeae4e7c
I20250109 08:58:21.754302 23606 raft_consensus.cc:2743] T a54ca9f0d6e844e9a8b1d9cd0bb2a026 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:58:21.755012 23572 raft_consensus.cc:738] T b572a6a3dfc04ea199bc703ddad8af67 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf55383bef6649eb9e78cb2d2512dedc, State: Initialized, Role: FOLLOWER
I20250109 08:58:21.756086 23572 consensus_queue.cc:260] T b572a6a3dfc04ea199bc703ddad8af67 P cf55383bef6649eb9e78cb2d2512dedc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:21.760941 23574 tablet_bootstrap.cc:654] T e5251927614642639bd46ecd5373d2a4 P 235ad6af04c34c88a5261daa3604fada: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:21.761040 23572 ts_tablet_manager.cc:1428] T b572a6a3dfc04ea199bc703ddad8af67 P cf55383bef6649eb9e78cb2d2512dedc: Time spent starting tablet: real 0.029s user 0.005s sys 0.000s
I20250109 08:58:21.763259 23572 tablet_bootstrap.cc:492] T a686bf4d947c4b638049e6b88c59ae09 P cf55383bef6649eb9e78cb2d2512dedc: Bootstrap starting.
I20250109 08:58:21.780282 23572 tablet_bootstrap.cc:654] T a686bf4d947c4b638049e6b88c59ae09 P cf55383bef6649eb9e78cb2d2512dedc: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:21.783035 23569 tablet_bootstrap.cc:492] T 2ab70546dda143b5b5db15c3875aa348 P 05a74e1eb2474b3d8b3a2da1ef127e01: No bootstrap required, opened a new log
I20250109 08:58:21.783520 23569 ts_tablet_manager.cc:1397] T 2ab70546dda143b5b5db15c3875aa348 P 05a74e1eb2474b3d8b3a2da1ef127e01: Time spent bootstrapping tablet: real 0.083s user 0.039s sys 0.040s
I20250109 08:58:21.786203 23569 raft_consensus.cc:357] T 2ab70546dda143b5b5db15c3875aa348 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:21.786900 23569 raft_consensus.cc:383] T 2ab70546dda143b5b5db15c3875aa348 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:21.787180 23569 raft_consensus.cc:738] T 2ab70546dda143b5b5db15c3875aa348 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05a74e1eb2474b3d8b3a2da1ef127e01, State: Initialized, Role: FOLLOWER
I20250109 08:58:21.787814 23569 consensus_queue.cc:260] T 2ab70546dda143b5b5db15c3875aa348 P 05a74e1eb2474b3d8b3a2da1ef127e01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:21.790563 23569 ts_tablet_manager.cc:1428] T 2ab70546dda143b5b5db15c3875aa348 P 05a74e1eb2474b3d8b3a2da1ef127e01: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20250109 08:58:21.791492 23569 tablet_bootstrap.cc:492] T 24ce131f7bb143bea1581d5b0c2a7109 P 05a74e1eb2474b3d8b3a2da1ef127e01: Bootstrap starting.
I20250109 08:58:21.797782 23574 tablet_bootstrap.cc:492] T e5251927614642639bd46ecd5373d2a4 P 235ad6af04c34c88a5261daa3604fada: No bootstrap required, opened a new log
I20250109 08:58:21.792927 23572 tablet_bootstrap.cc:492] T a686bf4d947c4b638049e6b88c59ae09 P cf55383bef6649eb9e78cb2d2512dedc: No bootstrap required, opened a new log
I20250109 08:58:21.798249 23574 ts_tablet_manager.cc:1397] T e5251927614642639bd46ecd5373d2a4 P 235ad6af04c34c88a5261daa3604fada: Time spent bootstrapping tablet: real 0.074s user 0.009s sys 0.002s
I20250109 08:58:21.798537 23572 ts_tablet_manager.cc:1397] T a686bf4d947c4b638049e6b88c59ae09 P cf55383bef6649eb9e78cb2d2512dedc: Time spent bootstrapping tablet: real 0.035s user 0.014s sys 0.002s
I20250109 08:58:21.799492 23580 tablet_bootstrap.cc:492] T 1c5b5beea4a840eb9677097cacee3408 P ed1efbce9ee3401e99a5260edeae4e7c: No bootstrap required, opened a new log
I20250109 08:58:21.799939 23580 ts_tablet_manager.cc:1397] T 1c5b5beea4a840eb9677097cacee3408 P ed1efbce9ee3401e99a5260edeae4e7c: Time spent bootstrapping tablet: real 0.070s user 0.012s sys 0.005s
I20250109 08:58:21.800942 23574 raft_consensus.cc:357] T e5251927614642639bd46ecd5373d2a4 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:21.801716 23574 raft_consensus.cc:383] T e5251927614642639bd46ecd5373d2a4 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:21.801213 23572 raft_consensus.cc:357] T a686bf4d947c4b638049e6b88c59ae09 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:21.801998 23574 raft_consensus.cc:738] T e5251927614642639bd46ecd5373d2a4 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Initialized, Role: FOLLOWER
I20250109 08:58:21.802176 23572 raft_consensus.cc:383] T a686bf4d947c4b638049e6b88c59ae09 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:21.802637 23572 raft_consensus.cc:738] T a686bf4d947c4b638049e6b88c59ae09 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf55383bef6649eb9e78cb2d2512dedc, State: Initialized, Role: FOLLOWER
I20250109 08:58:21.802780 23574 consensus_queue.cc:260] T e5251927614642639bd46ecd5373d2a4 P 235ad6af04c34c88a5261daa3604fada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:21.803601 23569 tablet_bootstrap.cc:654] T 24ce131f7bb143bea1581d5b0c2a7109 P 05a74e1eb2474b3d8b3a2da1ef127e01: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:21.803284 23572 consensus_queue.cc:260] T a686bf4d947c4b638049e6b88c59ae09 P cf55383bef6649eb9e78cb2d2512dedc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:21.806797 23580 raft_consensus.cc:357] T 1c5b5beea4a840eb9677097cacee3408 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:21.807467 23580 raft_consensus.cc:383] T 1c5b5beea4a840eb9677097cacee3408 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:21.807722 23580 raft_consensus.cc:738] T 1c5b5beea4a840eb9677097cacee3408 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed1efbce9ee3401e99a5260edeae4e7c, State: Initialized, Role: FOLLOWER
I20250109 08:58:21.812108 23580 consensus_queue.cc:260] T 1c5b5beea4a840eb9677097cacee3408 P ed1efbce9ee3401e99a5260edeae4e7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:21.809206 23572 ts_tablet_manager.cc:1428] T a686bf4d947c4b638049e6b88c59ae09 P cf55383bef6649eb9e78cb2d2512dedc: Time spent starting tablet: real 0.010s user 0.005s sys 0.000s
I20250109 08:58:21.813768 23572 tablet_bootstrap.cc:492] T 5c37c58abaa54656b31c72160b4dec9e P cf55383bef6649eb9e78cb2d2512dedc: Bootstrap starting.
I20250109 08:58:21.814507 23580 ts_tablet_manager.cc:1428] T 1c5b5beea4a840eb9677097cacee3408 P ed1efbce9ee3401e99a5260edeae4e7c: Time spent starting tablet: real 0.014s user 0.006s sys 0.000s
I20250109 08:58:21.820868 23635 consensus_queue.cc:1035] T 6eef9bb3c645428d9f1cb843bae9a264 P ed1efbce9ee3401e99a5260edeae4e7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:58:21.821652 23580 tablet_bootstrap.cc:492] T d5fa255ab5f24750a0afa6d81ec869f2 P ed1efbce9ee3401e99a5260edeae4e7c: Bootstrap starting.
I20250109 08:58:21.821851 23572 tablet_bootstrap.cc:654] T 5c37c58abaa54656b31c72160b4dec9e P cf55383bef6649eb9e78cb2d2512dedc: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:21.827906 23580 tablet_bootstrap.cc:654] T d5fa255ab5f24750a0afa6d81ec869f2 P ed1efbce9ee3401e99a5260edeae4e7c: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:21.847337 23572 tablet_bootstrap.cc:492] T 5c37c58abaa54656b31c72160b4dec9e P cf55383bef6649eb9e78cb2d2512dedc: No bootstrap required, opened a new log
I20250109 08:58:21.847335 23580 tablet_bootstrap.cc:492] T d5fa255ab5f24750a0afa6d81ec869f2 P ed1efbce9ee3401e99a5260edeae4e7c: No bootstrap required, opened a new log
I20250109 08:58:21.847855 23580 ts_tablet_manager.cc:1397] T d5fa255ab5f24750a0afa6d81ec869f2 P ed1efbce9ee3401e99a5260edeae4e7c: Time spent bootstrapping tablet: real 0.026s user 0.008s sys 0.009s
I20250109 08:58:21.847855 23572 ts_tablet_manager.cc:1397] T 5c37c58abaa54656b31c72160b4dec9e P cf55383bef6649eb9e78cb2d2512dedc: Time spent bootstrapping tablet: real 0.034s user 0.019s sys 0.000s
I20250109 08:58:21.850502 23580 raft_consensus.cc:357] T d5fa255ab5f24750a0afa6d81ec869f2 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:21.851263 23580 raft_consensus.cc:383] T d5fa255ab5f24750a0afa6d81ec869f2 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:21.851537 23580 raft_consensus.cc:738] T d5fa255ab5f24750a0afa6d81ec869f2 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed1efbce9ee3401e99a5260edeae4e7c, State: Initialized, Role: FOLLOWER
I20250109 08:58:21.852128 23580 consensus_queue.cc:260] T d5fa255ab5f24750a0afa6d81ec869f2 P ed1efbce9ee3401e99a5260edeae4e7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:21.854305 23580 ts_tablet_manager.cc:1428] T d5fa255ab5f24750a0afa6d81ec869f2 P ed1efbce9ee3401e99a5260edeae4e7c: Time spent starting tablet: real 0.006s user 0.002s sys 0.003s
I20250109 08:58:21.855069 23580 tablet_bootstrap.cc:492] T a54ca9f0d6e844e9a8b1d9cd0bb2a026 P ed1efbce9ee3401e99a5260edeae4e7c: Bootstrap starting.
I20250109 08:58:21.855437 23572 raft_consensus.cc:357] T 5c37c58abaa54656b31c72160b4dec9e P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:21.856144 23572 raft_consensus.cc:383] T 5c37c58abaa54656b31c72160b4dec9e P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:21.856410 23572 raft_consensus.cc:738] T 5c37c58abaa54656b31c72160b4dec9e P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf55383bef6649eb9e78cb2d2512dedc, State: Initialized, Role: FOLLOWER
I20250109 08:58:21.857069 23572 consensus_queue.cc:260] T 5c37c58abaa54656b31c72160b4dec9e P cf55383bef6649eb9e78cb2d2512dedc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:21.862237 23572 ts_tablet_manager.cc:1428] T 5c37c58abaa54656b31c72160b4dec9e P cf55383bef6649eb9e78cb2d2512dedc: Time spent starting tablet: real 0.014s user 0.002s sys 0.003s
I20250109 08:58:21.863126 23572 tablet_bootstrap.cc:492] T 5831288451724d4ca1114ec5e5f01b78 P cf55383bef6649eb9e78cb2d2512dedc: Bootstrap starting.
I20250109 08:58:21.866051 23580 tablet_bootstrap.cc:654] T a54ca9f0d6e844e9a8b1d9cd0bb2a026 P ed1efbce9ee3401e99a5260edeae4e7c: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:21.867307 23658 consensus_queue.cc:1035] T 6eef9bb3c645428d9f1cb843bae9a264 P ed1efbce9ee3401e99a5260edeae4e7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:58:21.872160 23658 consensus_queue.cc:1035] T 34caa5988a734b65a2eb512747d833b7 P ed1efbce9ee3401e99a5260edeae4e7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:58:21.873344 23627 raft_consensus.cc:491] T d5fa255ab5f24750a0afa6d81ec869f2 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:21.873893 23627 raft_consensus.cc:513] T d5fa255ab5f24750a0afa6d81ec869f2 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:21.875923 23627 leader_election.cc:290] T d5fa255ab5f24750a0afa6d81ec869f2 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365), 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829)
I20250109 08:58:21.877489 23528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5fa255ab5f24750a0afa6d81ec869f2" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" is_pre_election: true
I20250109 08:58:21.878185 23528 raft_consensus.cc:2463] T d5fa255ab5f24750a0afa6d81ec869f2 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf55383bef6649eb9e78cb2d2512dedc in term 0.
I20250109 08:58:21.879602 23418 leader_election.cc:304] T d5fa255ab5f24750a0afa6d81ec869f2 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01, cf55383bef6649eb9e78cb2d2512dedc; no voters:
I20250109 08:58:21.880863 23572 tablet_bootstrap.cc:654] T 5831288451724d4ca1114ec5e5f01b78 P cf55383bef6649eb9e78cb2d2512dedc: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:21.881376 23378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5fa255ab5f24750a0afa6d81ec869f2" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" is_pre_election: true
I20250109 08:58:21.857018 23611 raft_consensus.cc:491] T d5fa255ab5f24750a0afa6d81ec869f2 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:21.882499 23611 raft_consensus.cc:513] T d5fa255ab5f24750a0afa6d81ec869f2 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:21.884547 23611 leader_election.cc:290] T d5fa255ab5f24750a0afa6d81ec869f2 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551), 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829)
I20250109 08:58:21.885432 23580 tablet_bootstrap.cc:492] T a54ca9f0d6e844e9a8b1d9cd0bb2a026 P ed1efbce9ee3401e99a5260edeae4e7c: No bootstrap required, opened a new log
I20250109 08:58:21.885802 23580 ts_tablet_manager.cc:1397] T a54ca9f0d6e844e9a8b1d9cd0bb2a026 P ed1efbce9ee3401e99a5260edeae4e7c: Time spent bootstrapping tablet: real 0.031s user 0.014s sys 0.010s
I20250109 08:58:21.886150 23378 raft_consensus.cc:2463] T d5fa255ab5f24750a0afa6d81ec869f2 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf55383bef6649eb9e78cb2d2512dedc in term 0.
I20250109 08:58:21.886590 23528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5fa255ab5f24750a0afa6d81ec869f2" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" is_pre_election: true
I20250109 08:58:21.887187 23528 raft_consensus.cc:2463] T d5fa255ab5f24750a0afa6d81ec869f2 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed1efbce9ee3401e99a5260edeae4e7c in term 0.
I20250109 08:58:21.888257 23344 leader_election.cc:304] T d5fa255ab5f24750a0afa6d81ec869f2 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01, ed1efbce9ee3401e99a5260edeae4e7c; no voters:
I20250109 08:58:21.889853 23572 tablet_bootstrap.cc:492] T 5831288451724d4ca1114ec5e5f01b78 P cf55383bef6649eb9e78cb2d2512dedc: No bootstrap required, opened a new log
I20250109 08:58:21.890259 23572 ts_tablet_manager.cc:1397] T 5831288451724d4ca1114ec5e5f01b78 P cf55383bef6649eb9e78cb2d2512dedc: Time spent bootstrapping tablet: real 0.027s user 0.008s sys 0.001s
I20250109 08:58:21.892290 23627 raft_consensus.cc:2798] T d5fa255ab5f24750a0afa6d81ec869f2 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:58:21.892796 23453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5fa255ab5f24750a0afa6d81ec869f2" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc" is_pre_election: true
I20250109 08:58:21.888206 23580 raft_consensus.cc:357] T a54ca9f0d6e844e9a8b1d9cd0bb2a026 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:21.893579 23580 raft_consensus.cc:383] T a54ca9f0d6e844e9a8b1d9cd0bb2a026 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:21.893846 23580 raft_consensus.cc:738] T a54ca9f0d6e844e9a8b1d9cd0bb2a026 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed1efbce9ee3401e99a5260edeae4e7c, State: Initialized, Role: FOLLOWER
I20250109 08:58:21.894554 23580 consensus_queue.cc:260] T a54ca9f0d6e844e9a8b1d9cd0bb2a026 P ed1efbce9ee3401e99a5260edeae4e7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:21.896688 23635 raft_consensus.cc:2798] T d5fa255ab5f24750a0afa6d81ec869f2 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:58:21.892822 23572 raft_consensus.cc:357] T 5831288451724d4ca1114ec5e5f01b78 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:21.897177 23635 raft_consensus.cc:491] T d5fa255ab5f24750a0afa6d81ec869f2 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:58:21.897778 23635 raft_consensus.cc:3054] T d5fa255ab5f24750a0afa6d81ec869f2 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:21.892719 23627 raft_consensus.cc:491] T d5fa255ab5f24750a0afa6d81ec869f2 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:58:21.902051 23627 raft_consensus.cc:3054] T d5fa255ab5f24750a0afa6d81ec869f2 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:21.904861 23658 consensus_queue.cc:1035] T 34caa5988a734b65a2eb512747d833b7 P ed1efbce9ee3401e99a5260edeae4e7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:58:21.931371 23666 raft_consensus.cc:491] T 24ce131f7bb143bea1581d5b0c2a7109 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:21.932044 23666 raft_consensus.cc:513] T 24ce131f7bb143bea1581d5b0c2a7109 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:21.934165 23666 leader_election.cc:290] T 24ce131f7bb143bea1581d5b0c2a7109 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551), 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829)
I20250109 08:58:21.934978 23658 raft_consensus.cc:491] T 1c5b5beea4a840eb9677097cacee3408 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:21.935633 23658 raft_consensus.cc:513] T 1c5b5beea4a840eb9677097cacee3408 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:21.936327 23709 raft_consensus.cc:491] T 2ab70546dda143b5b5db15c3875aa348 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:21.936844 23709 raft_consensus.cc:513] T 2ab70546dda143b5b5db15c3875aa348 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:21.937132 23527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24ce131f7bb143bea1581d5b0c2a7109" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" is_pre_election: true
I20250109 08:58:21.938872 23709 leader_election.cc:290] T 2ab70546dda143b5b5db15c3875aa348 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691), ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365)
I20250109 08:58:21.939210 23528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5b5beea4a840eb9677097cacee3408" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" is_pre_election: true
I20250109 08:58:21.939878 23528 raft_consensus.cc:2463] T 1c5b5beea4a840eb9677097cacee3408 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed1efbce9ee3401e99a5260edeae4e7c in term 0.
I20250109 08:58:21.941078 23378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ab70546dda143b5b5db15c3875aa348" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" is_pre_election: true
I20250109 08:58:21.941287 23344 leader_election.cc:304] T 1c5b5beea4a840eb9677097cacee3408 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01, ed1efbce9ee3401e99a5260edeae4e7c; no voters:
I20250109 08:58:21.941857 23303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ab70546dda143b5b5db15c3875aa348" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada" is_pre_election: true
W20250109 08:58:21.942310 23494 leader_election.cc:343] T 2ab70546dda143b5b5db15c3875aa348 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365): Illegal state: must be running to vote when last-logged opid is not known
W20250109 08:58:21.943562 23494 leader_election.cc:343] T 2ab70546dda143b5b5db15c3875aa348 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:21.944435 23304 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5b5beea4a840eb9677097cacee3408" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada" is_pre_election: true
I20250109 08:58:21.945374 23451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24ce131f7bb143bea1581d5b0c2a7109" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc" is_pre_election: true
I20250109 08:58:21.944053 23494 leader_election.cc:304] T 2ab70546dda143b5b5db15c3875aa348 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01; no voters: 235ad6af04c34c88a5261daa3604fada, ed1efbce9ee3401e99a5260edeae4e7c
I20250109 08:58:21.946947 23709 raft_consensus.cc:2743] T 2ab70546dda143b5b5db15c3875aa348 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:58:21.950525 23378 raft_consensus.cc:3054] T 41e7f0cc38c4414ab88c92c1cdeb3984 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Advancing to term 1
W20250109 08:58:21.943986 23269 leader_election.cc:343] T 24ce131f7bb143bea1581d5b0c2a7109 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829): Illegal state: must be running to vote when last-logged opid is not known
W20250109 08:58:21.952924 23271 leader_election.cc:343] T 24ce131f7bb143bea1581d5b0c2a7109 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:21.953372 23271 leader_election.cc:304] T 24ce131f7bb143bea1581d5b0c2a7109 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 235ad6af04c34c88a5261daa3604fada; no voters: 05a74e1eb2474b3d8b3a2da1ef127e01, cf55383bef6649eb9e78cb2d2512dedc
W20250109 08:58:21.955231 23346 leader_election.cc:343] T 1c5b5beea4a840eb9677097cacee3408 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:21.955897 23666 raft_consensus.cc:2743] T 24ce131f7bb143bea1581d5b0c2a7109 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:58:21.956694 23611 raft_consensus.cc:2798] T 1c5b5beea4a840eb9677097cacee3408 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:58:21.957136 23611 raft_consensus.cc:491] T 1c5b5beea4a840eb9677097cacee3408 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:58:21.957470 23611 raft_consensus.cc:3054] T 1c5b5beea4a840eb9677097cacee3408 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:21.959781 23709 consensus_queue.cc:1035] T 41e7f0cc38c4414ab88c92c1cdeb3984 P 05a74e1eb2474b3d8b3a2da1ef127e01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:58:21.939390 23658 leader_election.cc:290] T 1c5b5beea4a840eb9677097cacee3408 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829), 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691)
I20250109 08:58:21.963117 23658 raft_consensus.cc:491] T 5831288451724d4ca1114ec5e5f01b78 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:21.967067 23658 raft_consensus.cc:513] T 5831288451724d4ca1114ec5e5f01b78 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:21.911862 23627 raft_consensus.cc:513] T d5fa255ab5f24750a0afa6d81ec869f2 P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:21.969506 23453 raft_consensus.cc:2388] T d5fa255ab5f24750a0afa6d81ec869f2 P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ed1efbce9ee3401e99a5260edeae4e7c in current term 1: Already voted for candidate cf55383bef6649eb9e78cb2d2512dedc in this term.
I20250109 08:58:21.931380 23574 ts_tablet_manager.cc:1428] T e5251927614642639bd46ecd5373d2a4 P 235ad6af04c34c88a5261daa3604fada: Time spent starting tablet: real 0.133s user 0.005s sys 0.000s
I20250109 08:58:21.973409 23378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5fa255ab5f24750a0afa6d81ec869f2" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c"
I20250109 08:58:21.897428 23572 raft_consensus.cc:383] T 5831288451724d4ca1114ec5e5f01b78 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:21.974376 23572 raft_consensus.cc:738] T 5831288451724d4ca1114ec5e5f01b78 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf55383bef6649eb9e78cb2d2512dedc, State: Initialized, Role: FOLLOWER
I20250109 08:58:21.975229 23572 consensus_queue.cc:260] T 5831288451724d4ca1114ec5e5f01b78 P cf55383bef6649eb9e78cb2d2512dedc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:21.966130 23635 raft_consensus.cc:513] T d5fa255ab5f24750a0afa6d81ec869f2 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:21.978237 23527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5fa255ab5f24750a0afa6d81ec869f2" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01"
I20250109 08:58:21.978849 23527 raft_consensus.cc:3054] T d5fa255ab5f24750a0afa6d81ec869f2 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Advancing to term 1
W20250109 08:58:21.979990 23494 consensus_peers.cc:487] T 9b245cb3666c421c884cf20161a22831 P 05a74e1eb2474b3d8b3a2da1ef127e01 -> Peer cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551): Couldn't send request to peer cf55383bef6649eb9e78cb2d2512dedc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:21.984254 23574 tablet_bootstrap.cc:492] T 1c5b5beea4a840eb9677097cacee3408 P 235ad6af04c34c88a5261daa3604fada: Bootstrap starting.
I20250109 08:58:21.985869 23709 consensus_queue.cc:1035] T 41e7f0cc38c4414ab88c92c1cdeb3984 P 05a74e1eb2474b3d8b3a2da1ef127e01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:58:21.976734 23627 leader_election.cc:290] T d5fa255ab5f24750a0afa6d81ec869f2 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 election: Requested vote from peers ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365), 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829)
I20250109 08:58:22.011636 23452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5831288451724d4ca1114ec5e5f01b78" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc" is_pre_election: true
I20250109 08:58:22.002348 23627 raft_consensus.cc:491] T 5831288451724d4ca1114ec5e5f01b78 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:22.027225 23528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5831288451724d4ca1114ec5e5f01b78" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" is_pre_election: true
I20250109 08:58:22.028810 23709 consensus_queue.cc:1035] T 9b245cb3666c421c884cf20161a22831 P 05a74e1eb2474b3d8b3a2da1ef127e01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20250109 08:58:22.033556 23344 leader_election.cc:343] T 5831288451724d4ca1114ec5e5f01b78 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:22.016732 23378 raft_consensus.cc:2388] T d5fa255ab5f24750a0afa6d81ec869f2 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate cf55383bef6649eb9e78cb2d2512dedc in current term 1: Already voted for candidate ed1efbce9ee3401e99a5260edeae4e7c in this term.
I20250109 08:58:22.036584 23572 ts_tablet_manager.cc:1428] T 5831288451724d4ca1114ec5e5f01b78 P cf55383bef6649eb9e78cb2d2512dedc: Time spent starting tablet: real 0.146s user 0.007s sys 0.001s
I20250109 08:58:22.040091 23572 tablet_bootstrap.cc:492] T a54ca9f0d6e844e9a8b1d9cd0bb2a026 P cf55383bef6649eb9e78cb2d2512dedc: Bootstrap starting.
I20250109 08:58:22.046484 23574 tablet_bootstrap.cc:654] T 1c5b5beea4a840eb9677097cacee3408 P 235ad6af04c34c88a5261daa3604fada: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:22.034876 23658 leader_election.cc:290] T 5831288451724d4ca1114ec5e5f01b78 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551), 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829)
I20250109 08:58:22.044610 23453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5fa255ab5f24750a0afa6d81ec869f2" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc"
I20250109 08:58:22.042136 23527 raft_consensus.cc:2463] T d5fa255ab5f24750a0afa6d81ec869f2 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf55383bef6649eb9e78cb2d2512dedc in term 1.
I20250109 08:58:22.049728 23453 raft_consensus.cc:2388] T d5fa255ab5f24750a0afa6d81ec869f2 P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate ed1efbce9ee3401e99a5260edeae4e7c in current term 1: Already voted for candidate cf55383bef6649eb9e78cb2d2512dedc in this term.
I20250109 08:58:22.050776 23418 leader_election.cc:304] T d5fa255ab5f24750a0afa6d81ec869f2 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01, cf55383bef6649eb9e78cb2d2512dedc; no voters:
I20250109 08:58:22.051568 23583 raft_consensus.cc:2798] T d5fa255ab5f24750a0afa6d81ec869f2 P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:58:22.052003 23583 raft_consensus.cc:695] T d5fa255ab5f24750a0afa6d81ec869f2 P cf55383bef6649eb9e78cb2d2512dedc [term 1 LEADER]: Becoming Leader. State: Replica: cf55383bef6649eb9e78cb2d2512dedc, State: Running, Role: LEADER
I20250109 08:58:22.052793 23583 consensus_queue.cc:237] T d5fa255ab5f24750a0afa6d81ec869f2 P cf55383bef6649eb9e78cb2d2512dedc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:22.045931 23528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5fa255ab5f24750a0afa6d81ec869f2" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01"
I20250109 08:58:22.055918 23528 raft_consensus.cc:2388] T d5fa255ab5f24750a0afa6d81ec869f2 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate ed1efbce9ee3401e99a5260edeae4e7c in current term 1: Already voted for candidate cf55383bef6649eb9e78cb2d2512dedc in this term.
I20250109 08:58:22.056896 23344 leader_election.cc:304] T d5fa255ab5f24750a0afa6d81ec869f2 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ed1efbce9ee3401e99a5260edeae4e7c; no voters: 05a74e1eb2474b3d8b3a2da1ef127e01, cf55383bef6649eb9e78cb2d2512dedc
I20250109 08:58:22.057852 23730 raft_consensus.cc:2743] T d5fa255ab5f24750a0afa6d81ec869f2 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20250109 08:58:22.043390 23635 leader_election.cc:290] T d5fa255ab5f24750a0afa6d81ec869f2 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 election: Requested vote from peers cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551), 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829)
I20250109 08:58:22.057543 23611 raft_consensus.cc:513] T 1c5b5beea4a840eb9677097cacee3408 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:22.035847 23627 raft_consensus.cc:513] T 5831288451724d4ca1114ec5e5f01b78 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:22.062158 23611 leader_election.cc:290] T 1c5b5beea4a840eb9677097cacee3408 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 election: Requested vote from peers 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829), 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691)
I20250109 08:58:22.060937 23580 ts_tablet_manager.cc:1428] T a54ca9f0d6e844e9a8b1d9cd0bb2a026 P ed1efbce9ee3401e99a5260edeae4e7c: Time spent starting tablet: real 0.175s user 0.043s sys 0.062s
I20250109 08:58:22.063220 23627 leader_election.cc:290] T 5831288451724d4ca1114ec5e5f01b78 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365), 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829)
I20250109 08:58:22.063918 23378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5831288451724d4ca1114ec5e5f01b78" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" is_pre_election: true
I20250109 08:58:22.064558 23378 raft_consensus.cc:2463] T 5831288451724d4ca1114ec5e5f01b78 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf55383bef6649eb9e78cb2d2512dedc in term 0.
I20250109 08:58:22.065032 23569 tablet_bootstrap.cc:492] T 24ce131f7bb143bea1581d5b0c2a7109 P 05a74e1eb2474b3d8b3a2da1ef127e01: No bootstrap required, opened a new log
I20250109 08:58:22.065450 23569 ts_tablet_manager.cc:1397] T 24ce131f7bb143bea1581d5b0c2a7109 P 05a74e1eb2474b3d8b3a2da1ef127e01: Time spent bootstrapping tablet: real 0.274s user 0.071s sys 0.136s
I20250109 08:58:22.065587 23528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5831288451724d4ca1114ec5e5f01b78" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" is_pre_election: true
I20250109 08:58:22.065711 23420 leader_election.cc:304] T 5831288451724d4ca1114ec5e5f01b78 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf55383bef6649eb9e78cb2d2512dedc, ed1efbce9ee3401e99a5260edeae4e7c; no voters:
I20250109 08:58:22.066692 23627 raft_consensus.cc:2798] T 5831288451724d4ca1114ec5e5f01b78 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Leader pre-election won for term 1
W20250109 08:58:22.067039 23418 leader_election.cc:343] T 5831288451724d4ca1114ec5e5f01b78 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:22.067031 23627 raft_consensus.cc:491] T 5831288451724d4ca1114ec5e5f01b78 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:58:22.064512 23305 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5b5beea4a840eb9677097cacee3408" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada"
I20250109 08:58:22.068251 23627 raft_consensus.cc:3054] T 5831288451724d4ca1114ec5e5f01b78 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Advancing to term 1
W20250109 08:58:22.069376 23346 leader_election.cc:343] T 1c5b5beea4a840eb9677097cacee3408 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:22.067832 23569 raft_consensus.cc:357] T 24ce131f7bb143bea1581d5b0c2a7109 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:22.070139 23611 raft_consensus.cc:491] T a54ca9f0d6e844e9a8b1d9cd0bb2a026 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:22.070340 23569 raft_consensus.cc:383] T 24ce131f7bb143bea1581d5b0c2a7109 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:22.070649 23611 raft_consensus.cc:513] T a54ca9f0d6e844e9a8b1d9cd0bb2a026 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:22.070672 23569 raft_consensus.cc:738] T 24ce131f7bb143bea1581d5b0c2a7109 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05a74e1eb2474b3d8b3a2da1ef127e01, State: Initialized, Role: FOLLOWER
I20250109 08:58:22.072753 23611 leader_election.cc:290] T a54ca9f0d6e844e9a8b1d9cd0bb2a026 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691), cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551)
I20250109 08:58:22.075368 23305 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a54ca9f0d6e844e9a8b1d9cd0bb2a026" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada" is_pre_election: true
I20250109 08:58:22.075590 23709 raft_consensus.cc:491] T b572a6a3dfc04ea199bc703ddad8af67 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:22.078361 23709 raft_consensus.cc:513] T b572a6a3dfc04ea199bc703ddad8af67 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:22.079785 23627 raft_consensus.cc:513] T 5831288451724d4ca1114ec5e5f01b78 P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:22.080288 23709 leader_election.cc:290] T b572a6a3dfc04ea199bc703ddad8af67 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551), 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691)
I20250109 08:58:22.077903 23675 raft_consensus.cc:491] T b572a6a3dfc04ea199bc703ddad8af67 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:22.072440 23569 consensus_queue.cc:260] T 24ce131f7bb143bea1581d5b0c2a7109 P 05a74e1eb2474b3d8b3a2da1ef127e01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:22.080971 23675 raft_consensus.cc:513] T b572a6a3dfc04ea199bc703ddad8af67 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:22.082844 23675 leader_election.cc:290] T b572a6a3dfc04ea199bc703ddad8af67 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829), 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691)
I20250109 08:58:22.083333 23569 ts_tablet_manager.cc:1428] T 24ce131f7bb143bea1581d5b0c2a7109 P 05a74e1eb2474b3d8b3a2da1ef127e01: Time spent starting tablet: real 0.018s user 0.007s sys 0.000s
I20250109 08:58:22.084017 23302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b572a6a3dfc04ea199bc703ddad8af67" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada" is_pre_election: true
I20250109 08:58:22.084753 23569 tablet_bootstrap.cc:492] T cc9886333c0343798226d59758a1ea5e P 05a74e1eb2474b3d8b3a2da1ef127e01: Bootstrap starting.
W20250109 08:58:22.085696 23420 leader_election.cc:343] T b572a6a3dfc04ea199bc703ddad8af67 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:22.085718 23454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b572a6a3dfc04ea199bc703ddad8af67" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc" is_pre_election: true
I20250109 08:58:22.078591 23453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a54ca9f0d6e844e9a8b1d9cd0bb2a026" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc" is_pre_election: true
I20250109 08:58:22.086932 23454 raft_consensus.cc:2463] T b572a6a3dfc04ea199bc703ddad8af67 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05a74e1eb2474b3d8b3a2da1ef127e01 in term 0.
I20250109 08:58:22.086927 23302 raft_consensus.cc:3054] T 5ccc0d42d4a148d7819366cea0b754a7 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:22.070720 23666 raft_consensus.cc:491] T e5251927614642639bd46ecd5373d2a4 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:22.088788 23666 raft_consensus.cc:513] T e5251927614642639bd46ecd5373d2a4 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:22.089395 23494 leader_election.cc:304] T b572a6a3dfc04ea199bc703ddad8af67 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01, cf55383bef6649eb9e78cb2d2512dedc; no voters:
I20250109 08:58:22.090451 23631 raft_consensus.cc:2798] T b572a6a3dfc04ea199bc703ddad8af67 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:58:22.090606 23666 leader_election.cc:290] T e5251927614642639bd46ecd5373d2a4 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551), 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829)
I20250109 08:58:22.090914 23569 tablet_bootstrap.cc:654] T cc9886333c0343798226d59758a1ea5e P 05a74e1eb2474b3d8b3a2da1ef127e01: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:22.091832 23301 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b572a6a3dfc04ea199bc703ddad8af67" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada" is_pre_election: true
I20250109 08:58:22.076067 23305 raft_consensus.cc:2463] T a54ca9f0d6e844e9a8b1d9cd0bb2a026 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed1efbce9ee3401e99a5260edeae4e7c in term 0.
I20250109 08:58:22.092859 23527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5251927614642639bd46ecd5373d2a4" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" is_pre_election: true
I20250109 08:58:22.093755 23346 leader_election.cc:304] T a54ca9f0d6e844e9a8b1d9cd0bb2a026 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 235ad6af04c34c88a5261daa3604fada, ed1efbce9ee3401e99a5260edeae4e7c; no voters:
I20250109 08:58:22.094491 23627 leader_election.cc:290] T 5831288451724d4ca1114ec5e5f01b78 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 election: Requested vote from peers ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365), 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829)
I20250109 08:58:22.094848 23631 raft_consensus.cc:491] T b572a6a3dfc04ea199bc703ddad8af67 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:58:22.095253 23631 raft_consensus.cc:3054] T b572a6a3dfc04ea199bc703ddad8af67 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:22.095819 23378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5831288451724d4ca1114ec5e5f01b78" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c"
I20250109 08:58:22.096357 23378 raft_consensus.cc:3054] T 5831288451724d4ca1114ec5e5f01b78 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:22.094496 23611 raft_consensus.cc:2798] T a54ca9f0d6e844e9a8b1d9cd0bb2a026 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:58:22.097935 23611 raft_consensus.cc:491] T a54ca9f0d6e844e9a8b1d9cd0bb2a026 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:58:22.098244 23611 raft_consensus.cc:3054] T a54ca9f0d6e844e9a8b1d9cd0bb2a026 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:22.100822 23452 raft_consensus.cc:2388] T 5831288451724d4ca1114ec5e5f01b78 P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ed1efbce9ee3401e99a5260edeae4e7c in current term 1: Already voted for candidate cf55383bef6649eb9e78cb2d2512dedc in this term.
I20250109 08:58:22.102367 23528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b572a6a3dfc04ea199bc703ddad8af67" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" is_pre_election: true
I20250109 08:58:22.104729 23527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5831288451724d4ca1114ec5e5f01b78" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01"
W20250109 08:58:22.105777 23346 leader_election.cc:343] T a54ca9f0d6e844e9a8b1d9cd0bb2a026 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:22.109380 23346 leader_election.cc:304] T 5831288451724d4ca1114ec5e5f01b78 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ed1efbce9ee3401e99a5260edeae4e7c; no voters: 05a74e1eb2474b3d8b3a2da1ef127e01, cf55383bef6649eb9e78cb2d2512dedc
W20250109 08:58:22.102382 23269 leader_election.cc:343] T e5251927614642639bd46ecd5373d2a4 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829): Illegal state: must be running to vote when last-logged opid is not known
W20250109 08:58:22.108973 23494 leader_election.cc:343] T b572a6a3dfc04ea199bc703ddad8af67 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:22.116120 23452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5251927614642639bd46ecd5373d2a4" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc" is_pre_election: true
I20250109 08:58:22.117483 23527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5b5beea4a840eb9677097cacee3408" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01"
I20250109 08:58:22.118144 23527 raft_consensus.cc:3054] T 1c5b5beea4a840eb9677097cacee3408 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Advancing to term 1
W20250109 08:58:22.120905 23418 leader_election.cc:343] T 5831288451724d4ca1114ec5e5f01b78 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829): Illegal state: must be running to vote when last-logged opid is not known
W20250109 08:58:22.122048 23271 leader_election.cc:343] T e5251927614642639bd46ecd5373d2a4 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:22.122498 23271 leader_election.cc:304] T e5251927614642639bd46ecd5373d2a4 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 235ad6af04c34c88a5261daa3604fada; no voters: 05a74e1eb2474b3d8b3a2da1ef127e01, cf55383bef6649eb9e78cb2d2512dedc
I20250109 08:58:22.123554 23666 raft_consensus.cc:2743] T e5251927614642639bd46ecd5373d2a4 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:58:22.126189 23631 raft_consensus.cc:513] T b572a6a3dfc04ea199bc703ddad8af67 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:22.131985 23528 raft_consensus.cc:2388] T b572a6a3dfc04ea199bc703ddad8af67 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf55383bef6649eb9e78cb2d2512dedc in current term 1: Already voted for candidate 05a74e1eb2474b3d8b3a2da1ef127e01 in this term.
I20250109 08:58:22.134541 23580 tablet_bootstrap.cc:492] T e6b9aaeda9a947d4a87fedd774301cc0 P ed1efbce9ee3401e99a5260edeae4e7c: Bootstrap starting.
I20250109 08:58:22.060956 23572 tablet_bootstrap.cc:654] T a54ca9f0d6e844e9a8b1d9cd0bb2a026 P cf55383bef6649eb9e78cb2d2512dedc: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:22.139407 23305 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b572a6a3dfc04ea199bc703ddad8af67" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada"
I20250109 08:58:22.141191 23418 leader_election.cc:304] T b572a6a3dfc04ea199bc703ddad8af67 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf55383bef6649eb9e78cb2d2512dedc; no voters: 05a74e1eb2474b3d8b3a2da1ef127e01, 235ad6af04c34c88a5261daa3604fada
I20250109 08:58:22.128000 23611 raft_consensus.cc:513] T a54ca9f0d6e844e9a8b1d9cd0bb2a026 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
W20250109 08:58:22.142575 23494 leader_election.cc:343] T b572a6a3dfc04ea199bc703ddad8af67 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:22.137656 23452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b572a6a3dfc04ea199bc703ddad8af67" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc"
I20250109 08:58:22.143937 23452 raft_consensus.cc:3054] T b572a6a3dfc04ea199bc703ddad8af67 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:22.145632 23580 tablet_bootstrap.cc:654] T e6b9aaeda9a947d4a87fedd774301cc0 P ed1efbce9ee3401e99a5260edeae4e7c: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:22.147017 23303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a54ca9f0d6e844e9a8b1d9cd0bb2a026" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada"
I20250109 08:58:22.147656 23303 raft_consensus.cc:3054] T a54ca9f0d6e844e9a8b1d9cd0bb2a026 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:22.148417 23611 leader_election.cc:290] T a54ca9f0d6e844e9a8b1d9cd0bb2a026 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 election: Requested vote from peers 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691), cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551)
I20250109 08:58:22.149734 23454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a54ca9f0d6e844e9a8b1d9cd0bb2a026" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc"
W20250109 08:58:22.151494 23346 leader_election.cc:343] T a54ca9f0d6e844e9a8b1d9cd0bb2a026 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:22.152544 23631 leader_election.cc:290] T b572a6a3dfc04ea199bc703ddad8af67 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 election: Requested vote from peers cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551), 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691)
I20250109 08:58:22.127101 23378 raft_consensus.cc:2463] T 5831288451724d4ca1114ec5e5f01b78 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf55383bef6649eb9e78cb2d2512dedc in term 1.
I20250109 08:58:22.156952 23658 raft_consensus.cc:2743] T 5831288451724d4ca1114ec5e5f01b78 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:58:22.161775 23420 leader_election.cc:304] T 5831288451724d4ca1114ec5e5f01b78 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf55383bef6649eb9e78cb2d2512dedc, ed1efbce9ee3401e99a5260edeae4e7c; no voters: 05a74e1eb2474b3d8b3a2da1ef127e01
I20250109 08:58:22.163295 23627 raft_consensus.cc:2798] T 5831288451724d4ca1114ec5e5f01b78 P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:58:22.163750 23627 raft_consensus.cc:695] T 5831288451724d4ca1114ec5e5f01b78 P cf55383bef6649eb9e78cb2d2512dedc [term 1 LEADER]: Becoming Leader. State: Replica: cf55383bef6649eb9e78cb2d2512dedc, State: Running, Role: LEADER
I20250109 08:58:22.164518 23627 consensus_queue.cc:237] T 5831288451724d4ca1114ec5e5f01b78 P cf55383bef6649eb9e78cb2d2512dedc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:22.165755 23527 raft_consensus.cc:2463] T 1c5b5beea4a840eb9677097cacee3408 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed1efbce9ee3401e99a5260edeae4e7c in term 1.
I20250109 08:58:22.170888 23344 leader_election.cc:304] T 1c5b5beea4a840eb9677097cacee3408 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01, ed1efbce9ee3401e99a5260edeae4e7c; no voters: 235ad6af04c34c88a5261daa3604fada
I20250109 08:58:22.171965 23658 raft_consensus.cc:2798] T 1c5b5beea4a840eb9677097cacee3408 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:58:22.172479 23658 raft_consensus.cc:695] T 1c5b5beea4a840eb9677097cacee3408 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 LEADER]: Becoming Leader. State: Replica: ed1efbce9ee3401e99a5260edeae4e7c, State: Running, Role: LEADER
I20250109 08:58:22.173251 23658 consensus_queue.cc:237] T 1c5b5beea4a840eb9677097cacee3408 P ed1efbce9ee3401e99a5260edeae4e7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:22.177250 23303 raft_consensus.cc:2463] T a54ca9f0d6e844e9a8b1d9cd0bb2a026 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed1efbce9ee3401e99a5260edeae4e7c in term 1.
I20250109 08:58:22.178403 23346 leader_election.cc:304] T a54ca9f0d6e844e9a8b1d9cd0bb2a026 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 235ad6af04c34c88a5261daa3604fada, ed1efbce9ee3401e99a5260edeae4e7c; no voters: cf55383bef6649eb9e78cb2d2512dedc
I20250109 08:58:22.178954 23583 raft_consensus.cc:491] T a686bf4d947c4b638049e6b88c59ae09 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:22.179587 23611 raft_consensus.cc:2798] T a54ca9f0d6e844e9a8b1d9cd0bb2a026 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:58:22.179409 23583 raft_consensus.cc:513] T a686bf4d947c4b638049e6b88c59ae09 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:22.180218 23611 raft_consensus.cc:695] T a54ca9f0d6e844e9a8b1d9cd0bb2a026 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 LEADER]: Becoming Leader. State: Replica: ed1efbce9ee3401e99a5260edeae4e7c, State: Running, Role: LEADER
I20250109 08:58:22.181110 23611 consensus_queue.cc:237] T a54ca9f0d6e844e9a8b1d9cd0bb2a026 P ed1efbce9ee3401e99a5260edeae4e7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:22.182226 23378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a686bf4d947c4b638049e6b88c59ae09" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" is_pre_election: true
I20250109 08:58:22.182816 23527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a686bf4d947c4b638049e6b88c59ae09" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" is_pre_election: true
W20250109 08:58:22.183574 23420 leader_election.cc:343] T a686bf4d947c4b638049e6b88c59ae09 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365): Illegal state: must be running to vote when last-logged opid is not known
W20250109 08:58:22.184118 23418 leader_election.cc:343] T a686bf4d947c4b638049e6b88c59ae09 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:22.184597 23418 leader_election.cc:304] T a686bf4d947c4b638049e6b88c59ae09 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf55383bef6649eb9e78cb2d2512dedc; no voters: 05a74e1eb2474b3d8b3a2da1ef127e01, ed1efbce9ee3401e99a5260edeae4e7c
I20250109 08:58:22.185338 23675 raft_consensus.cc:2743] T a686bf4d947c4b638049e6b88c59ae09 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:58:22.186511 23586 raft_consensus.cc:491] T d87d3b4755344b82b0aec7ffd1ce785c P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:22.186972 23586 raft_consensus.cc:513] T d87d3b4755344b82b0aec7ffd1ce785c P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:22.188766 23586 leader_election.cc:290] T d87d3b4755344b82b0aec7ffd1ce785c P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829), ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365)
I20250109 08:58:22.189229 23586 consensus_queue.cc:1035] T 4df64aae69ca41dc91286cae1603000f P 235ad6af04c34c88a5261daa3604fada [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:58:22.189708 23379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d87d3b4755344b82b0aec7ffd1ce785c" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" is_pre_election: true
W20250109 08:58:22.191394 23271 leader_election.cc:343] T d87d3b4755344b82b0aec7ffd1ce785c P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:22.200798 23527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d87d3b4755344b82b0aec7ffd1ce785c" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" is_pre_election: true
W20250109 08:58:22.202183 23269 leader_election.cc:343] T d87d3b4755344b82b0aec7ffd1ce785c P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:22.202641 23269 leader_election.cc:304] T d87d3b4755344b82b0aec7ffd1ce785c P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 235ad6af04c34c88a5261daa3604fada; no voters: 05a74e1eb2474b3d8b3a2da1ef127e01, ed1efbce9ee3401e99a5260edeae4e7c
I20250109 08:58:22.203778 23666 raft_consensus.cc:2743] T d87d3b4755344b82b0aec7ffd1ce785c P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:58:22.199450 23201 catalog_manager.cc:5526] T d5fa255ab5f24750a0afa6d81ec869f2 P cf55383bef6649eb9e78cb2d2512dedc reported cstate change: term changed from 0 to 1, leader changed from <none> to cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195). New cstate: current_term: 1 leader_uuid: "cf55383bef6649eb9e78cb2d2512dedc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:22.205705 23583 leader_election.cc:290] T a686bf4d947c4b638049e6b88c59ae09 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365), 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829)
I20250109 08:58:22.218906 23572 tablet_bootstrap.cc:492] T a54ca9f0d6e844e9a8b1d9cd0bb2a026 P cf55383bef6649eb9e78cb2d2512dedc: No bootstrap required, opened a new log
I20250109 08:58:22.219449 23572 ts_tablet_manager.cc:1397] T a54ca9f0d6e844e9a8b1d9cd0bb2a026 P cf55383bef6649eb9e78cb2d2512dedc: Time spent bootstrapping tablet: real 0.179s user 0.009s sys 0.000s
I20250109 08:58:22.226471 23452 raft_consensus.cc:2463] T b572a6a3dfc04ea199bc703ddad8af67 P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05a74e1eb2474b3d8b3a2da1ef127e01 in term 1.
I20250109 08:58:22.215549 23580 tablet_bootstrap.cc:492] T e6b9aaeda9a947d4a87fedd774301cc0 P ed1efbce9ee3401e99a5260edeae4e7c: No bootstrap required, opened a new log
I20250109 08:58:22.228081 23580 ts_tablet_manager.cc:1397] T e6b9aaeda9a947d4a87fedd774301cc0 P ed1efbce9ee3401e99a5260edeae4e7c: Time spent bootstrapping tablet: real 0.094s user 0.019s sys 0.009s
I20250109 08:58:22.228602 23677 raft_consensus.cc:2743] T b572a6a3dfc04ea199bc703ddad8af67 P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:58:22.229889 23494 leader_election.cc:304] T b572a6a3dfc04ea199bc703ddad8af67 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01, cf55383bef6649eb9e78cb2d2512dedc; no voters: 235ad6af04c34c88a5261daa3604fada
I20250109 08:58:22.230618 23580 raft_consensus.cc:357] T e6b9aaeda9a947d4a87fedd774301cc0 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:22.231277 23580 raft_consensus.cc:383] T e6b9aaeda9a947d4a87fedd774301cc0 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:22.231541 23580 raft_consensus.cc:738] T e6b9aaeda9a947d4a87fedd774301cc0 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed1efbce9ee3401e99a5260edeae4e7c, State: Initialized, Role: FOLLOWER
W20250109 08:58:22.236404 23251 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20250109 08:58:22.236785 23572 raft_consensus.cc:357] T a54ca9f0d6e844e9a8b1d9cd0bb2a026 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:22.241389 23572 raft_consensus.cc:383] T a54ca9f0d6e844e9a8b1d9cd0bb2a026 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:22.232113 23580 consensus_queue.cc:260] T e6b9aaeda9a947d4a87fedd774301cc0 P ed1efbce9ee3401e99a5260edeae4e7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:22.241535 23631 raft_consensus.cc:2798] T b572a6a3dfc04ea199bc703ddad8af67 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:58:22.243275 23611 raft_consensus.cc:491] T e6b9aaeda9a947d4a87fedd774301cc0 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:22.242776 23572 raft_consensus.cc:738] T a54ca9f0d6e844e9a8b1d9cd0bb2a026 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf55383bef6649eb9e78cb2d2512dedc, State: Initialized, Role: FOLLOWER
I20250109 08:58:22.243085 23569 tablet_bootstrap.cc:492] T cc9886333c0343798226d59758a1ea5e P 05a74e1eb2474b3d8b3a2da1ef127e01: No bootstrap required, opened a new log
I20250109 08:58:22.243777 23611 raft_consensus.cc:513] T e6b9aaeda9a947d4a87fedd774301cc0 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:22.244300 23572 consensus_queue.cc:260] T a54ca9f0d6e844e9a8b1d9cd0bb2a026 P cf55383bef6649eb9e78cb2d2512dedc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:22.244282 23569 ts_tablet_manager.cc:1397] T cc9886333c0343798226d59758a1ea5e P 05a74e1eb2474b3d8b3a2da1ef127e01: Time spent bootstrapping tablet: real 0.160s user 0.039s sys 0.041s
I20250109 08:58:22.245455 23631 raft_consensus.cc:695] T b572a6a3dfc04ea199bc703ddad8af67 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 LEADER]: Becoming Leader. State: Replica: 05a74e1eb2474b3d8b3a2da1ef127e01, State: Running, Role: LEADER
I20250109 08:58:22.249136 23572 ts_tablet_manager.cc:1428] T a54ca9f0d6e844e9a8b1d9cd0bb2a026 P cf55383bef6649eb9e78cb2d2512dedc: Time spent starting tablet: real 0.014s user 0.006s sys 0.000s
I20250109 08:58:22.246254 23574 tablet_bootstrap.cc:492] T 1c5b5beea4a840eb9677097cacee3408 P 235ad6af04c34c88a5261daa3604fada: No bootstrap required, opened a new log
W20250109 08:58:22.249662 23269 consensus_peers.cc:487] T 4df64aae69ca41dc91286cae1603000f P 235ad6af04c34c88a5261daa3604fada -> Peer 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829): Couldn't send request to peer 05a74e1eb2474b3d8b3a2da1ef127e01. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:22.250104 23572 tablet_bootstrap.cc:492] T cc9886333c0343798226d59758a1ea5e P cf55383bef6649eb9e78cb2d2512dedc: Bootstrap starting.
I20250109 08:58:22.253445 23580 ts_tablet_manager.cc:1428] T e6b9aaeda9a947d4a87fedd774301cc0 P ed1efbce9ee3401e99a5260edeae4e7c: Time spent starting tablet: real 0.025s user 0.005s sys 0.000s
I20250109 08:58:22.253870 23569 raft_consensus.cc:357] T cc9886333c0343798226d59758a1ea5e P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:22.254345 23580 tablet_bootstrap.cc:492] T 86d4074509744900abe94402467a7d85 P ed1efbce9ee3401e99a5260edeae4e7c: Bootstrap starting.
I20250109 08:58:22.254567 23569 raft_consensus.cc:383] T cc9886333c0343798226d59758a1ea5e P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:22.254886 23569 raft_consensus.cc:738] T cc9886333c0343798226d59758a1ea5e P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05a74e1eb2474b3d8b3a2da1ef127e01, State: Initialized, Role: FOLLOWER
I20250109 08:58:22.255586 23569 consensus_queue.cc:260] T cc9886333c0343798226d59758a1ea5e P 05a74e1eb2474b3d8b3a2da1ef127e01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:22.257643 23569 ts_tablet_manager.cc:1428] T cc9886333c0343798226d59758a1ea5e P 05a74e1eb2474b3d8b3a2da1ef127e01: Time spent starting tablet: real 0.012s user 0.005s sys 0.000s
I20250109 08:58:22.258517 23569 tablet_bootstrap.cc:492] T 940653b6d55c46439827697b9aadf16b P 05a74e1eb2474b3d8b3a2da1ef127e01: Bootstrap starting.
I20250109 08:58:22.259701 23580 tablet_bootstrap.cc:654] T 86d4074509744900abe94402467a7d85 P ed1efbce9ee3401e99a5260edeae4e7c: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:22.249701 23574 ts_tablet_manager.cc:1397] T 1c5b5beea4a840eb9677097cacee3408 P 235ad6af04c34c88a5261daa3604fada: Time spent bootstrapping tablet: real 0.266s user 0.061s sys 0.070s
I20250109 08:58:22.262773 23574 raft_consensus.cc:357] T 1c5b5beea4a840eb9677097cacee3408 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:22.263437 23574 raft_consensus.cc:383] T 1c5b5beea4a840eb9677097cacee3408 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:22.263551 23569 tablet_bootstrap.cc:654] T 940653b6d55c46439827697b9aadf16b P 05a74e1eb2474b3d8b3a2da1ef127e01: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:22.264164 23611 leader_election.cc:290] T e6b9aaeda9a947d4a87fedd774301cc0 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691), cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551)
I20250109 08:58:22.264657 23574 raft_consensus.cc:738] T 1c5b5beea4a840eb9677097cacee3408 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Initialized, Role: FOLLOWER
I20250109 08:58:22.265286 23574 consensus_queue.cc:260] T 1c5b5beea4a840eb9677097cacee3408 P 235ad6af04c34c88a5261daa3604fada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:22.266193 23452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6b9aaeda9a947d4a87fedd774301cc0" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc" is_pre_election: true
I20250109 08:58:22.267410 23574 ts_tablet_manager.cc:1428] T 1c5b5beea4a840eb9677097cacee3408 P 235ad6af04c34c88a5261daa3604fada: Time spent starting tablet: real 0.007s user 0.002s sys 0.001s
W20250109 08:58:22.267920 23346 leader_election.cc:343] T e6b9aaeda9a947d4a87fedd774301cc0 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:22.268252 23574 tablet_bootstrap.cc:492] T 2ab70546dda143b5b5db15c3875aa348 P 235ad6af04c34c88a5261daa3604fada: Bootstrap starting.
I20250109 08:58:22.268774 23303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6b9aaeda9a947d4a87fedd774301cc0" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada" is_pre_election: true
W20250109 08:58:22.270181 23346 leader_election.cc:343] T e6b9aaeda9a947d4a87fedd774301cc0 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:22.270610 23346 leader_election.cc:304] T e6b9aaeda9a947d4a87fedd774301cc0 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ed1efbce9ee3401e99a5260edeae4e7c; no voters: 235ad6af04c34c88a5261daa3604fada, cf55383bef6649eb9e78cb2d2512dedc
I20250109 08:58:22.271569 23611 raft_consensus.cc:2743] T e6b9aaeda9a947d4a87fedd774301cc0 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:58:22.273883 23572 tablet_bootstrap.cc:654] T cc9886333c0343798226d59758a1ea5e P cf55383bef6649eb9e78cb2d2512dedc: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:22.278038 23574 tablet_bootstrap.cc:654] T 2ab70546dda143b5b5db15c3875aa348 P 235ad6af04c34c88a5261daa3604fada: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:22.263715 23202 catalog_manager.cc:5526] T a54ca9f0d6e844e9a8b1d9cd0bb2a026 P ed1efbce9ee3401e99a5260edeae4e7c reported cstate change: term changed from 0 to 1, leader changed from <none> to ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194). New cstate: current_term: 1 leader_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:22.279191 23202 catalog_manager.cc:5526] T 1c5b5beea4a840eb9677097cacee3408 P ed1efbce9ee3401e99a5260edeae4e7c reported cstate change: term changed from 0 to 1, leader changed from <none> to ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194). New cstate: current_term: 1 leader_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } health_report { overall_health: HEALTHY } } }
I20250109 08:58:22.248961 23631 consensus_queue.cc:237] T b572a6a3dfc04ea199bc703ddad8af67 P 05a74e1eb2474b3d8b3a2da1ef127e01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:22.284703 23574 tablet_bootstrap.cc:492] T 2ab70546dda143b5b5db15c3875aa348 P 235ad6af04c34c88a5261daa3604fada: No bootstrap required, opened a new log
I20250109 08:58:22.285158 23574 ts_tablet_manager.cc:1397] T 2ab70546dda143b5b5db15c3875aa348 P 235ad6af04c34c88a5261daa3604fada: Time spent bootstrapping tablet: real 0.017s user 0.006s sys 0.004s
I20250109 08:58:22.287468 23574 raft_consensus.cc:357] T 2ab70546dda143b5b5db15c3875aa348 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:22.288129 23574 raft_consensus.cc:383] T 2ab70546dda143b5b5db15c3875aa348 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:22.291319 23580 tablet_bootstrap.cc:492] T 86d4074509744900abe94402467a7d85 P ed1efbce9ee3401e99a5260edeae4e7c: No bootstrap required, opened a new log
I20250109 08:58:22.290748 23201 catalog_manager.cc:5526] T 5831288451724d4ca1114ec5e5f01b78 P cf55383bef6649eb9e78cb2d2512dedc reported cstate change: term changed from 0 to 1, leader changed from <none> to cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195). New cstate: current_term: 1 leader_uuid: "cf55383bef6649eb9e78cb2d2512dedc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:22.291782 23580 ts_tablet_manager.cc:1397] T 86d4074509744900abe94402467a7d85 P ed1efbce9ee3401e99a5260edeae4e7c: Time spent bootstrapping tablet: real 0.038s user 0.010s sys 0.002s
I20250109 08:58:22.294463 23580 raft_consensus.cc:357] T 86d4074509744900abe94402467a7d85 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:22.295099 23580 raft_consensus.cc:383] T 86d4074509744900abe94402467a7d85 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:22.295357 23580 raft_consensus.cc:738] T 86d4074509744900abe94402467a7d85 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed1efbce9ee3401e99a5260edeae4e7c, State: Initialized, Role: FOLLOWER
I20250109 08:58:22.295905 23580 consensus_queue.cc:260] T 86d4074509744900abe94402467a7d85 P ed1efbce9ee3401e99a5260edeae4e7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:22.301255 23580 ts_tablet_manager.cc:1428] T 86d4074509744900abe94402467a7d85 P ed1efbce9ee3401e99a5260edeae4e7c: Time spent starting tablet: real 0.009s user 0.004s sys 0.003s
I20250109 08:58:22.298853 23569 tablet_bootstrap.cc:492] T 940653b6d55c46439827697b9aadf16b P 05a74e1eb2474b3d8b3a2da1ef127e01: No bootstrap required, opened a new log
I20250109 08:58:22.301937 23569 ts_tablet_manager.cc:1397] T 940653b6d55c46439827697b9aadf16b P 05a74e1eb2474b3d8b3a2da1ef127e01: Time spent bootstrapping tablet: real 0.044s user 0.020s sys 0.002s
I20250109 08:58:22.302134 23580 tablet_bootstrap.cc:492] T 940653b6d55c46439827697b9aadf16b P ed1efbce9ee3401e99a5260edeae4e7c: Bootstrap starting.
I20250109 08:58:22.304386 23569 raft_consensus.cc:357] T 940653b6d55c46439827697b9aadf16b P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:22.305053 23569 raft_consensus.cc:383] T 940653b6d55c46439827697b9aadf16b P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:22.305317 23569 raft_consensus.cc:738] T 940653b6d55c46439827697b9aadf16b P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05a74e1eb2474b3d8b3a2da1ef127e01, State: Initialized, Role: FOLLOWER
I20250109 08:58:22.307631 23574 raft_consensus.cc:738] T 2ab70546dda143b5b5db15c3875aa348 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Initialized, Role: FOLLOWER
I20250109 08:58:22.308502 23574 consensus_queue.cc:260] T 2ab70546dda143b5b5db15c3875aa348 P 235ad6af04c34c88a5261daa3604fada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:22.305527 23572 tablet_bootstrap.cc:492] T cc9886333c0343798226d59758a1ea5e P cf55383bef6649eb9e78cb2d2512dedc: No bootstrap required, opened a new log
I20250109 08:58:22.309520 23572 ts_tablet_manager.cc:1397] T cc9886333c0343798226d59758a1ea5e P cf55383bef6649eb9e78cb2d2512dedc: Time spent bootstrapping tablet: real 0.060s user 0.014s sys 0.002s
I20250109 08:58:22.310676 23580 tablet_bootstrap.cc:654] T 940653b6d55c46439827697b9aadf16b P ed1efbce9ee3401e99a5260edeae4e7c: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:22.311120 23574 ts_tablet_manager.cc:1428] T 2ab70546dda143b5b5db15c3875aa348 P 235ad6af04c34c88a5261daa3604fada: Time spent starting tablet: real 0.026s user 0.007s sys 0.000s
I20250109 08:58:22.312098 23574 tablet_bootstrap.cc:492] T e6b9aaeda9a947d4a87fedd774301cc0 P 235ad6af04c34c88a5261daa3604fada: Bootstrap starting.
I20250109 08:58:22.312810 23569 consensus_queue.cc:260] T 940653b6d55c46439827697b9aadf16b P 05a74e1eb2474b3d8b3a2da1ef127e01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:22.313395 23200 catalog_manager.cc:5526] T b572a6a3dfc04ea199bc703ddad8af67 P 05a74e1eb2474b3d8b3a2da1ef127e01 reported cstate change: term changed from 0 to 1, leader changed from <none> to 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196). New cstate: current_term: 1 leader_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:22.315297 23569 ts_tablet_manager.cc:1428] T 940653b6d55c46439827697b9aadf16b P 05a74e1eb2474b3d8b3a2da1ef127e01: Time spent starting tablet: real 0.013s user 0.005s sys 0.001s
I20250109 08:58:22.316145 23569 tablet_bootstrap.cc:492] T 5831288451724d4ca1114ec5e5f01b78 P 05a74e1eb2474b3d8b3a2da1ef127e01: Bootstrap starting.
I20250109 08:58:22.318604 23574 tablet_bootstrap.cc:654] T e6b9aaeda9a947d4a87fedd774301cc0 P 235ad6af04c34c88a5261daa3604fada: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:22.321255 23569 tablet_bootstrap.cc:654] T 5831288451724d4ca1114ec5e5f01b78 P 05a74e1eb2474b3d8b3a2da1ef127e01: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:22.321463 23580 tablet_bootstrap.cc:492] T 940653b6d55c46439827697b9aadf16b P ed1efbce9ee3401e99a5260edeae4e7c: No bootstrap required, opened a new log
I20250109 08:58:22.321955 23580 ts_tablet_manager.cc:1397] T 940653b6d55c46439827697b9aadf16b P ed1efbce9ee3401e99a5260edeae4e7c: Time spent bootstrapping tablet: real 0.020s user 0.011s sys 0.004s
I20250109 08:58:22.324946 23580 raft_consensus.cc:357] T 940653b6d55c46439827697b9aadf16b P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:22.325630 23580 raft_consensus.cc:383] T 940653b6d55c46439827697b9aadf16b P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:22.325946 23580 raft_consensus.cc:738] T 940653b6d55c46439827697b9aadf16b P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed1efbce9ee3401e99a5260edeae4e7c, State: Initialized, Role: FOLLOWER
I20250109 08:58:22.311951 23572 raft_consensus.cc:357] T cc9886333c0343798226d59758a1ea5e P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:22.326599 23580 consensus_queue.cc:260] T 940653b6d55c46439827697b9aadf16b P ed1efbce9ee3401e99a5260edeae4e7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:22.329854 23572 raft_consensus.cc:383] T cc9886333c0343798226d59758a1ea5e P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:22.329661 23574 tablet_bootstrap.cc:492] T e6b9aaeda9a947d4a87fedd774301cc0 P 235ad6af04c34c88a5261daa3604fada: No bootstrap required, opened a new log
I20250109 08:58:22.330233 23572 raft_consensus.cc:738] T cc9886333c0343798226d59758a1ea5e P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf55383bef6649eb9e78cb2d2512dedc, State: Initialized, Role: FOLLOWER
I20250109 08:58:22.330546 23574 ts_tablet_manager.cc:1397] T e6b9aaeda9a947d4a87fedd774301cc0 P 235ad6af04c34c88a5261daa3604fada: Time spent bootstrapping tablet: real 0.019s user 0.016s sys 0.000s
I20250109 08:58:22.331004 23572 consensus_queue.cc:260] T cc9886333c0343798226d59758a1ea5e P cf55383bef6649eb9e78cb2d2512dedc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:22.331732 23580 ts_tablet_manager.cc:1428] T 940653b6d55c46439827697b9aadf16b P ed1efbce9ee3401e99a5260edeae4e7c: Time spent starting tablet: real 0.010s user 0.007s sys 0.000s
I20250109 08:58:22.332623 23580 tablet_bootstrap.cc:492] T 2ab70546dda143b5b5db15c3875aa348 P ed1efbce9ee3401e99a5260edeae4e7c: Bootstrap starting.
I20250109 08:58:22.333479 23572 ts_tablet_manager.cc:1428] T cc9886333c0343798226d59758a1ea5e P cf55383bef6649eb9e78cb2d2512dedc: Time spent starting tablet: real 0.024s user 0.003s sys 0.003s
I20250109 08:58:22.333338 23574 raft_consensus.cc:357] T e6b9aaeda9a947d4a87fedd774301cc0 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:22.334164 23574 raft_consensus.cc:383] T e6b9aaeda9a947d4a87fedd774301cc0 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:22.334476 23574 raft_consensus.cc:738] T e6b9aaeda9a947d4a87fedd774301cc0 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Initialized, Role: FOLLOWER
I20250109 08:58:22.334494 23572 tablet_bootstrap.cc:492] T 86d4074509744900abe94402467a7d85 P cf55383bef6649eb9e78cb2d2512dedc: Bootstrap starting.
I20250109 08:58:22.337217 23569 tablet_bootstrap.cc:492] T 5831288451724d4ca1114ec5e5f01b78 P 05a74e1eb2474b3d8b3a2da1ef127e01: No bootstrap required, opened a new log
I20250109 08:58:22.337762 23569 ts_tablet_manager.cc:1397] T 5831288451724d4ca1114ec5e5f01b78 P 05a74e1eb2474b3d8b3a2da1ef127e01: Time spent bootstrapping tablet: real 0.022s user 0.017s sys 0.004s
I20250109 08:58:22.338255 23580 tablet_bootstrap.cc:654] T 2ab70546dda143b5b5db15c3875aa348 P ed1efbce9ee3401e99a5260edeae4e7c: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:22.340211 23569 raft_consensus.cc:357] T 5831288451724d4ca1114ec5e5f01b78 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:22.341070 23569 raft_consensus.cc:383] T 5831288451724d4ca1114ec5e5f01b78 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:22.340859 23574 consensus_queue.cc:260] T e6b9aaeda9a947d4a87fedd774301cc0 P 235ad6af04c34c88a5261daa3604fada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:22.341410 23569 raft_consensus.cc:738] T 5831288451724d4ca1114ec5e5f01b78 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05a74e1eb2474b3d8b3a2da1ef127e01, State: Initialized, Role: FOLLOWER
I20250109 08:58:22.343156 23569 consensus_queue.cc:260] T 5831288451724d4ca1114ec5e5f01b78 P 05a74e1eb2474b3d8b3a2da1ef127e01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:22.343124 23574 ts_tablet_manager.cc:1428] T e6b9aaeda9a947d4a87fedd774301cc0 P 235ad6af04c34c88a5261daa3604fada: Time spent starting tablet: real 0.012s user 0.004s sys 0.000s
I20250109 08:58:22.344890 23574 tablet_bootstrap.cc:492] T b572a6a3dfc04ea199bc703ddad8af67 P 235ad6af04c34c88a5261daa3604fada: Bootstrap starting.
I20250109 08:58:22.345901 23569 ts_tablet_manager.cc:1428] T 5831288451724d4ca1114ec5e5f01b78 P 05a74e1eb2474b3d8b3a2da1ef127e01: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20250109 08:58:22.347004 23569 tablet_bootstrap.cc:492] T 4df64aae69ca41dc91286cae1603000f P 05a74e1eb2474b3d8b3a2da1ef127e01: Bootstrap starting.
I20250109 08:58:22.350869 23574 tablet_bootstrap.cc:654] T b572a6a3dfc04ea199bc703ddad8af67 P 235ad6af04c34c88a5261daa3604fada: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:22.343890 23572 tablet_bootstrap.cc:654] T 86d4074509744900abe94402467a7d85 P cf55383bef6649eb9e78cb2d2512dedc: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:22.353461 23569 tablet_bootstrap.cc:654] T 4df64aae69ca41dc91286cae1603000f P 05a74e1eb2474b3d8b3a2da1ef127e01: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:22.354502 23580 tablet_bootstrap.cc:492] T 2ab70546dda143b5b5db15c3875aa348 P ed1efbce9ee3401e99a5260edeae4e7c: No bootstrap required, opened a new log
I20250109 08:58:22.355319 23677 raft_consensus.cc:491] T 5c37c58abaa54656b31c72160b4dec9e P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:22.355819 23677 raft_consensus.cc:513] T 5c37c58abaa54656b31c72160b4dec9e P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:22.357929 23677 leader_election.cc:290] T 5c37c58abaa54656b31c72160b4dec9e P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829), 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691)
I20250109 08:58:22.359712 23527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c37c58abaa54656b31c72160b4dec9e" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" is_pre_election: true
I20250109 08:58:22.362010 23303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c37c58abaa54656b31c72160b4dec9e" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada" is_pre_election: true
I20250109 08:58:22.362735 23303 raft_consensus.cc:2463] T 5c37c58abaa54656b31c72160b4dec9e P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf55383bef6649eb9e78cb2d2512dedc in term 0.
I20250109 08:58:22.368732 23586 raft_consensus.cc:491] T 2ab70546dda143b5b5db15c3875aa348 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:22.369299 23586 raft_consensus.cc:513] T 2ab70546dda143b5b5db15c3875aa348 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:22.371321 23586 leader_election.cc:290] T 2ab70546dda143b5b5db15c3875aa348 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829), ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365)
I20250109 08:58:22.373157 23379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ab70546dda143b5b5db15c3875aa348" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" is_pre_election: true
W20250109 08:58:22.374792 23418 leader_election.cc:343] T 5c37c58abaa54656b31c72160b4dec9e P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:22.375749 23420 leader_election.cc:304] T 5c37c58abaa54656b31c72160b4dec9e P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 235ad6af04c34c88a5261daa3604fada, cf55383bef6649eb9e78cb2d2512dedc; no voters: 05a74e1eb2474b3d8b3a2da1ef127e01
W20250109 08:58:22.377832 23271 leader_election.cc:343] T 2ab70546dda143b5b5db15c3875aa348 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:22.378856 23527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ab70546dda143b5b5db15c3875aa348" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" is_pre_election: true
I20250109 08:58:22.379542 23527 raft_consensus.cc:2463] T 2ab70546dda143b5b5db15c3875aa348 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 235ad6af04c34c88a5261daa3604fada in term 0.
I20250109 08:58:22.380838 23269 leader_election.cc:304] T 2ab70546dda143b5b5db15c3875aa348 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01, 235ad6af04c34c88a5261daa3604fada; no voters: ed1efbce9ee3401e99a5260edeae4e7c
I20250109 08:58:22.382493 23586 raft_consensus.cc:2798] T 2ab70546dda143b5b5db15c3875aa348 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:58:22.383358 23586 raft_consensus.cc:491] T 2ab70546dda143b5b5db15c3875aa348 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:58:22.383809 23586 raft_consensus.cc:3054] T 2ab70546dda143b5b5db15c3875aa348 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:22.382963 23677 raft_consensus.cc:2798] T 5c37c58abaa54656b31c72160b4dec9e P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:58:22.384588 23677 raft_consensus.cc:491] T 5c37c58abaa54656b31c72160b4dec9e P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:58:22.385038 23677 raft_consensus.cc:3054] T 5c37c58abaa54656b31c72160b4dec9e P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:22.391870 23586 raft_consensus.cc:513] T 2ab70546dda143b5b5db15c3875aa348 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:22.392618 23574 tablet_bootstrap.cc:492] T b572a6a3dfc04ea199bc703ddad8af67 P 235ad6af04c34c88a5261daa3604fada: No bootstrap required, opened a new log
I20250109 08:58:22.394436 23572 tablet_bootstrap.cc:492] T 86d4074509744900abe94402467a7d85 P cf55383bef6649eb9e78cb2d2512dedc: No bootstrap required, opened a new log
I20250109 08:58:22.394884 23572 ts_tablet_manager.cc:1397] T 86d4074509744900abe94402467a7d85 P cf55383bef6649eb9e78cb2d2512dedc: Time spent bootstrapping tablet: real 0.061s user 0.006s sys 0.022s
I20250109 08:58:22.394802 23527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ab70546dda143b5b5db15c3875aa348" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01"
I20250109 08:58:22.395550 23527 raft_consensus.cc:3054] T 2ab70546dda143b5b5db15c3875aa348 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:22.397056 23580 ts_tablet_manager.cc:1397] T 2ab70546dda143b5b5db15c3875aa348 P ed1efbce9ee3401e99a5260edeae4e7c: Time spent bootstrapping tablet: real 0.023s user 0.008s sys 0.013s
I20250109 08:58:22.397399 23572 raft_consensus.cc:357] T 86d4074509744900abe94402467a7d85 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:22.398092 23572 raft_consensus.cc:383] T 86d4074509744900abe94402467a7d85 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:22.398334 23572 raft_consensus.cc:738] T 86d4074509744900abe94402467a7d85 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf55383bef6649eb9e78cb2d2512dedc, State: Initialized, Role: FOLLOWER
I20250109 08:58:22.398483 23677 raft_consensus.cc:513] T 5c37c58abaa54656b31c72160b4dec9e P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:22.399127 23572 consensus_queue.cc:260] T 86d4074509744900abe94402467a7d85 P cf55383bef6649eb9e78cb2d2512dedc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:22.399638 23580 raft_consensus.cc:357] T 2ab70546dda143b5b5db15c3875aa348 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:22.400315 23580 raft_consensus.cc:383] T 2ab70546dda143b5b5db15c3875aa348 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:22.400597 23580 raft_consensus.cc:738] T 2ab70546dda143b5b5db15c3875aa348 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed1efbce9ee3401e99a5260edeae4e7c, State: Initialized, Role: FOLLOWER
I20250109 08:58:22.401181 23528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c37c58abaa54656b31c72160b4dec9e" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01"
I20250109 08:58:22.401379 23580 consensus_queue.cc:260] T 2ab70546dda143b5b5db15c3875aa348 P ed1efbce9ee3401e99a5260edeae4e7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:22.403604 23580 ts_tablet_manager.cc:1428] T 2ab70546dda143b5b5db15c3875aa348 P ed1efbce9ee3401e99a5260edeae4e7c: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
W20250109 08:58:22.404385 23418 leader_election.cc:343] T 5c37c58abaa54656b31c72160b4dec9e P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:22.394439 23569 tablet_bootstrap.cc:492] T 4df64aae69ca41dc91286cae1603000f P 05a74e1eb2474b3d8b3a2da1ef127e01: No bootstrap required, opened a new log
I20250109 08:58:22.394809 23574 ts_tablet_manager.cc:1397] T b572a6a3dfc04ea199bc703ddad8af67 P 235ad6af04c34c88a5261daa3604fada: Time spent bootstrapping tablet: real 0.050s user 0.019s sys 0.008s
I20250109 08:58:22.405707 23569 ts_tablet_manager.cc:1397] T 4df64aae69ca41dc91286cae1603000f P 05a74e1eb2474b3d8b3a2da1ef127e01: Time spent bootstrapping tablet: real 0.059s user 0.020s sys 0.026s
I20250109 08:58:22.406397 23303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c37c58abaa54656b31c72160b4dec9e" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada"
I20250109 08:58:22.406956 23303 raft_consensus.cc:3054] T 5c37c58abaa54656b31c72160b4dec9e P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:22.405128 23572 ts_tablet_manager.cc:1428] T 86d4074509744900abe94402467a7d85 P cf55383bef6649eb9e78cb2d2512dedc: Time spent starting tablet: real 0.010s user 0.004s sys 0.001s
I20250109 08:58:22.405202 23586 leader_election.cc:290] T 2ab70546dda143b5b5db15c3875aa348 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 election: Requested vote from peers 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829), ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365)
I20250109 08:58:22.408123 23569 raft_consensus.cc:357] T 4df64aae69ca41dc91286cae1603000f P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:22.408843 23569 raft_consensus.cc:383] T 4df64aae69ca41dc91286cae1603000f P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:22.409049 23631 raft_consensus.cc:491] T cc9886333c0343798226d59758a1ea5e P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:22.403709 23527 raft_consensus.cc:2463] T 2ab70546dda143b5b5db15c3875aa348 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 235ad6af04c34c88a5261daa3604fada in term 1.
I20250109 08:58:22.409709 23631 raft_consensus.cc:513] T cc9886333c0343798226d59758a1ea5e P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:22.407935 23574 raft_consensus.cc:357] T b572a6a3dfc04ea199bc703ddad8af67 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:22.410631 23574 raft_consensus.cc:383] T b572a6a3dfc04ea199bc703ddad8af67 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:22.410898 23574 raft_consensus.cc:738] T b572a6a3dfc04ea199bc703ddad8af67 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Initialized, Role: FOLLOWER
I20250109 08:58:22.411012 23269 leader_election.cc:304] T 2ab70546dda143b5b5db15c3875aa348 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01, 235ad6af04c34c88a5261daa3604fada; no voters:
I20250109 08:58:22.409104 23569 raft_consensus.cc:738] T 4df64aae69ca41dc91286cae1603000f P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05a74e1eb2474b3d8b3a2da1ef127e01, State: Initialized, Role: FOLLOWER
I20250109 08:58:22.411628 23574 consensus_queue.cc:260] T b572a6a3dfc04ea199bc703ddad8af67 P 235ad6af04c34c88a5261daa3604fada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:22.412251 23569 consensus_queue.cc:260] T 4df64aae69ca41dc91286cae1603000f P 05a74e1eb2474b3d8b3a2da1ef127e01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:22.412667 23606 raft_consensus.cc:2798] T 2ab70546dda143b5b5db15c3875aa348 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:58:22.413210 23606 raft_consensus.cc:695] T 2ab70546dda143b5b5db15c3875aa348 P 235ad6af04c34c88a5261daa3604fada [term 1 LEADER]: Becoming Leader. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Running, Role: LEADER
I20250109 08:58:22.412668 23305 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc9886333c0343798226d59758a1ea5e" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada" is_pre_election: true
I20250109 08:58:22.414429 23569 ts_tablet_manager.cc:1428] T 4df64aae69ca41dc91286cae1603000f P 05a74e1eb2474b3d8b3a2da1ef127e01: Time spent starting tablet: real 0.008s user 0.005s sys 0.000s
I20250109 08:58:22.414013 23606 consensus_queue.cc:237] T 2ab70546dda143b5b5db15c3875aa348 P 235ad6af04c34c88a5261daa3604fada [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
W20250109 08:58:22.414978 23494 leader_election.cc:343] T cc9886333c0343798226d59758a1ea5e P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:22.413365 23452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc9886333c0343798226d59758a1ea5e" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc" is_pre_election: true
I20250109 08:58:22.416092 23452 raft_consensus.cc:2463] T cc9886333c0343798226d59758a1ea5e P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05a74e1eb2474b3d8b3a2da1ef127e01 in term 0.
I20250109 08:58:22.417371 23494 leader_election.cc:304] T cc9886333c0343798226d59758a1ea5e P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01, cf55383bef6649eb9e78cb2d2512dedc; no voters: 235ad6af04c34c88a5261daa3604fada
I20250109 08:58:22.418131 23634 raft_consensus.cc:2798] T cc9886333c0343798226d59758a1ea5e P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:58:22.418471 23634 raft_consensus.cc:491] T cc9886333c0343798226d59758a1ea5e P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:58:22.418802 23634 raft_consensus.cc:3054] T cc9886333c0343798226d59758a1ea5e P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:22.420634 23569 tablet_bootstrap.cc:492] T d87d3b4755344b82b0aec7ffd1ce785c P 05a74e1eb2474b3d8b3a2da1ef127e01: Bootstrap starting.
I20250109 08:58:22.415541 23303 raft_consensus.cc:2463] T 5c37c58abaa54656b31c72160b4dec9e P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf55383bef6649eb9e78cb2d2512dedc in term 1.
I20250109 08:58:22.424669 23420 leader_election.cc:304] T 5c37c58abaa54656b31c72160b4dec9e P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 235ad6af04c34c88a5261daa3604fada, cf55383bef6649eb9e78cb2d2512dedc; no voters: 05a74e1eb2474b3d8b3a2da1ef127e01
I20250109 08:58:22.425514 23627 raft_consensus.cc:2798] T 5c37c58abaa54656b31c72160b4dec9e P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:58:22.426018 23627 raft_consensus.cc:695] T 5c37c58abaa54656b31c72160b4dec9e P cf55383bef6649eb9e78cb2d2512dedc [term 1 LEADER]: Becoming Leader. State: Replica: cf55383bef6649eb9e78cb2d2512dedc, State: Running, Role: LEADER
I20250109 08:58:22.426379 23569 tablet_bootstrap.cc:654] T d87d3b4755344b82b0aec7ffd1ce785c P 05a74e1eb2474b3d8b3a2da1ef127e01: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:22.426258 23658 raft_consensus.cc:491] T 86d4074509744900abe94402467a7d85 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:22.427016 23658 raft_consensus.cc:513] T 86d4074509744900abe94402467a7d85 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:22.426868 23627 consensus_queue.cc:237] T 5c37c58abaa54656b31c72160b4dec9e P cf55383bef6649eb9e78cb2d2512dedc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:22.411664 23631 leader_election.cc:290] T cc9886333c0343798226d59758a1ea5e P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551), 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691)
I20250109 08:58:22.430418 23572 tablet_bootstrap.cc:492] T e6b9aaeda9a947d4a87fedd774301cc0 P cf55383bef6649eb9e78cb2d2512dedc: Bootstrap starting.
I20250109 08:58:22.431855 23658 leader_election.cc:290] T 86d4074509744900abe94402467a7d85 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691), cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551)
I20250109 08:58:22.433676 23452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86d4074509744900abe94402467a7d85" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc" is_pre_election: true
I20250109 08:58:22.434314 23452 raft_consensus.cc:2463] T 86d4074509744900abe94402467a7d85 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed1efbce9ee3401e99a5260edeae4e7c in term 0.
I20250109 08:58:22.435460 23346 leader_election.cc:304] T 86d4074509744900abe94402467a7d85 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf55383bef6649eb9e78cb2d2512dedc, ed1efbce9ee3401e99a5260edeae4e7c; no voters:
I20250109 08:58:22.435868 23631 raft_consensus.cc:491] T 970a82e754db47b6b1a19551d4ba43d8 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:22.436245 23572 tablet_bootstrap.cc:654] T e6b9aaeda9a947d4a87fedd774301cc0 P cf55383bef6649eb9e78cb2d2512dedc: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:22.436312 23631 raft_consensus.cc:513] T 970a82e754db47b6b1a19551d4ba43d8 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:22.437294 23658 raft_consensus.cc:2798] T 86d4074509744900abe94402467a7d85 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:58:22.437680 23658 raft_consensus.cc:491] T 86d4074509744900abe94402467a7d85 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:58:22.438001 23658 raft_consensus.cc:3054] T 86d4074509744900abe94402467a7d85 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:22.436951 23305 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86d4074509744900abe94402467a7d85" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada" is_pre_election: true
I20250109 08:58:22.439035 23302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "970a82e754db47b6b1a19551d4ba43d8" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada" is_pre_election: true
I20250109 08:58:22.439642 23302 raft_consensus.cc:2463] T 970a82e754db47b6b1a19551d4ba43d8 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05a74e1eb2474b3d8b3a2da1ef127e01 in term 1.
I20250109 08:58:22.440233 23378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "970a82e754db47b6b1a19551d4ba43d8" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" is_pre_election: true
I20250109 08:58:22.440748 23494 leader_election.cc:304] T 970a82e754db47b6b1a19551d4ba43d8 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01, 235ad6af04c34c88a5261daa3604fada; no voters:
I20250109 08:58:22.440971 23378 raft_consensus.cc:2463] T 970a82e754db47b6b1a19551d4ba43d8 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05a74e1eb2474b3d8b3a2da1ef127e01 in term 1.
I20250109 08:58:22.441601 23709 raft_consensus.cc:2798] T 970a82e754db47b6b1a19551d4ba43d8 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20250109 08:58:22.441702 23346 leader_election.cc:343] T 86d4074509744900abe94402467a7d85 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:22.442008 23709 raft_consensus.cc:491] T 970a82e754db47b6b1a19551d4ba43d8 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:58:22.442373 23709 raft_consensus.cc:3054] T 970a82e754db47b6b1a19551d4ba43d8 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:22.441895 23635 raft_consensus.cc:491] T 2ab70546dda143b5b5db15c3875aa348 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:22.443051 23635 raft_consensus.cc:513] T 2ab70546dda143b5b5db15c3875aa348 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:22.445022 23635 leader_election.cc:290] T 2ab70546dda143b5b5db15c3875aa348 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829), 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691)
I20250109 08:58:22.446667 23631 leader_election.cc:290] T 970a82e754db47b6b1a19551d4ba43d8 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691), ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365)
I20250109 08:58:22.428653 23379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ab70546dda143b5b5db15c3875aa348" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c"
I20250109 08:58:22.448163 23379 raft_consensus.cc:3054] T 2ab70546dda143b5b5db15c3875aa348 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:22.454859 23379 raft_consensus.cc:2463] T 2ab70546dda143b5b5db15c3875aa348 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 235ad6af04c34c88a5261daa3604fada in term 1.
I20250109 08:58:22.425665 23634 raft_consensus.cc:513] T cc9886333c0343798226d59758a1ea5e P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:22.456995 23631 raft_consensus.cc:491] T 24ce131f7bb143bea1581d5b0c2a7109 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:22.457476 23631 raft_consensus.cc:513] T 24ce131f7bb143bea1581d5b0c2a7109 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:22.457918 23452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc9886333c0343798226d59758a1ea5e" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc"
I20250109 08:58:22.446879 23302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ab70546dda143b5b5db15c3875aa348" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada" is_pre_election: true
I20250109 08:58:22.458504 23452 raft_consensus.cc:3054] T cc9886333c0343798226d59758a1ea5e P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:22.460835 23527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ab70546dda143b5b5db15c3875aa348" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" is_pre_election: true
I20250109 08:58:22.461613 23527 raft_consensus.cc:2388] T 2ab70546dda143b5b5db15c3875aa348 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ed1efbce9ee3401e99a5260edeae4e7c in current term 1: Already voted for candidate 235ad6af04c34c88a5261daa3604fada in this term.
I20250109 08:58:22.462594 23344 leader_election.cc:304] T 2ab70546dda143b5b5db15c3875aa348 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ed1efbce9ee3401e99a5260edeae4e7c; no voters: 05a74e1eb2474b3d8b3a2da1ef127e01, 235ad6af04c34c88a5261daa3604fada
I20250109 08:58:22.463768 23635 raft_consensus.cc:2743] T 2ab70546dda143b5b5db15c3875aa348 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:58:22.464872 23631 leader_election.cc:290] T 24ce131f7bb143bea1581d5b0c2a7109 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551), 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691)
I20250109 08:58:22.466610 23454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24ce131f7bb143bea1581d5b0c2a7109" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc" is_pre_election: true
I20250109 08:58:22.467089 23452 raft_consensus.cc:2463] T cc9886333c0343798226d59758a1ea5e P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05a74e1eb2474b3d8b3a2da1ef127e01 in term 1.
W20250109 08:58:22.468112 23494 leader_election.cc:343] T 24ce131f7bb143bea1581d5b0c2a7109 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:22.468798 23494 leader_election.cc:304] T cc9886333c0343798226d59758a1ea5e P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01, cf55383bef6649eb9e78cb2d2512dedc; no voters:
I20250109 08:58:22.470270 23302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24ce131f7bb143bea1581d5b0c2a7109" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada" is_pre_election: true
I20250109 08:58:22.470896 23302 raft_consensus.cc:2463] T 24ce131f7bb143bea1581d5b0c2a7109 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05a74e1eb2474b3d8b3a2da1ef127e01 in term 0.
I20250109 08:58:22.473690 23494 leader_election.cc:304] T 24ce131f7bb143bea1581d5b0c2a7109 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01, 235ad6af04c34c88a5261daa3604fada; no voters: cf55383bef6649eb9e78cb2d2512dedc
I20250109 08:58:22.473342 23631 raft_consensus.cc:2798] T cc9886333c0343798226d59758a1ea5e P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:58:22.473441 23580 tablet_bootstrap.cc:492] T 08214ee9c4c14bda92bdc6fa8231c313 P ed1efbce9ee3401e99a5260edeae4e7c: Bootstrap starting.
I20250109 08:58:22.481057 23631 raft_consensus.cc:695] T cc9886333c0343798226d59758a1ea5e P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 LEADER]: Becoming Leader. State: Replica: 05a74e1eb2474b3d8b3a2da1ef127e01, State: Running, Role: LEADER
I20250109 08:58:22.481473 23766 raft_consensus.cc:2798] T 24ce131f7bb143bea1581d5b0c2a7109 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:58:22.481858 23766 raft_consensus.cc:491] T 24ce131f7bb143bea1581d5b0c2a7109 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:58:22.471163 23658 raft_consensus.cc:513] T 86d4074509744900abe94402467a7d85 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:22.474781 23677 leader_election.cc:290] T 5c37c58abaa54656b31c72160b4dec9e P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 election: Requested vote from peers 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829), 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691)
I20250109 08:58:22.489487 23574 ts_tablet_manager.cc:1428] T b572a6a3dfc04ea199bc703ddad8af67 P 235ad6af04c34c88a5261daa3604fada: Time spent starting tablet: real 0.084s user 0.006s sys 0.000s
I20250109 08:58:22.490736 23574 tablet_bootstrap.cc:492] T a09d8dffc2e446f1acd3c7f6cc1705bb P 235ad6af04c34c88a5261daa3604fada: Bootstrap starting.
I20250109 08:58:22.482174 23766 raft_consensus.cc:3054] T 24ce131f7bb143bea1581d5b0c2a7109 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:22.493696 23634 leader_election.cc:290] T cc9886333c0343798226d59758a1ea5e P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 election: Requested vote from peers cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551), 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691)
I20250109 08:58:22.494089 23606 raft_consensus.cc:491] T b572a6a3dfc04ea199bc703ddad8af67 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:22.482149 23631 consensus_queue.cc:237] T cc9886333c0343798226d59758a1ea5e P 05a74e1eb2474b3d8b3a2da1ef127e01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:22.503423 23634 raft_consensus.cc:491] T 940653b6d55c46439827697b9aadf16b P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:22.504987 23634 raft_consensus.cc:513] T 940653b6d55c46439827697b9aadf16b P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:22.505981 23658 leader_election.cc:290] T 86d4074509744900abe94402467a7d85 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 election: Requested vote from peers 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691), cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551)
I20250109 08:58:22.507061 23201 catalog_manager.cc:5526] T 2ab70546dda143b5b5db15c3875aa348 P 235ad6af04c34c88a5261daa3604fada reported cstate change: term changed from 0 to 1, leader changed from <none> to 235ad6af04c34c88a5261daa3604fada (127.19.226.193). New cstate: current_term: 1 leader_uuid: "235ad6af04c34c88a5261daa3604fada" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:22.516076 23303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc9886333c0343798226d59758a1ea5e" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada"
I20250109 08:58:22.517155 23200 catalog_manager.cc:5526] T 5c37c58abaa54656b31c72160b4dec9e P cf55383bef6649eb9e78cb2d2512dedc reported cstate change: term changed from 0 to 1, leader changed from <none> to cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195). New cstate: current_term: 1 leader_uuid: "cf55383bef6649eb9e78cb2d2512dedc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:22.519060 23580 tablet_bootstrap.cc:654] T 08214ee9c4c14bda92bdc6fa8231c313 P ed1efbce9ee3401e99a5260edeae4e7c: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:22.522348 23709 raft_consensus.cc:513] T 970a82e754db47b6b1a19551d4ba43d8 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:22.526285 23658 raft_consensus.cc:491] T 940653b6d55c46439827697b9aadf16b P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:22.526727 23454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86d4074509744900abe94402467a7d85" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc"
I20250109 08:58:22.527843 23454 raft_consensus.cc:3054] T 86d4074509744900abe94402467a7d85 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:22.528965 23572 tablet_bootstrap.cc:492] T e6b9aaeda9a947d4a87fedd774301cc0 P cf55383bef6649eb9e78cb2d2512dedc: No bootstrap required, opened a new log
I20250109 08:58:22.529417 23572 ts_tablet_manager.cc:1397] T e6b9aaeda9a947d4a87fedd774301cc0 P cf55383bef6649eb9e78cb2d2512dedc: Time spent bootstrapping tablet: real 0.099s user 0.022s sys 0.028s
I20250109 08:58:22.528295 23302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86d4074509744900abe94402467a7d85" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada"
I20250109 08:58:22.530459 23583 consensus_queue.cc:1035] T d5fa255ab5f24750a0afa6d81ec869f2 P cf55383bef6649eb9e78cb2d2512dedc [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20250109 08:58:22.530939 23346 leader_election.cc:343] T 86d4074509744900abe94402467a7d85 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:22.526763 23658 raft_consensus.cc:513] T 940653b6d55c46439827697b9aadf16b P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:22.535063 23658 leader_election.cc:290] T 940653b6d55c46439827697b9aadf16b P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551), 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829)
I20250109 08:58:22.531826 23572 raft_consensus.cc:357] T e6b9aaeda9a947d4a87fedd774301cc0 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:22.560006 23569 tablet_bootstrap.cc:492] T d87d3b4755344b82b0aec7ffd1ce785c P 05a74e1eb2474b3d8b3a2da1ef127e01: No bootstrap required, opened a new log
I20250109 08:58:22.563181 23572 raft_consensus.cc:383] T e6b9aaeda9a947d4a87fedd774301cc0 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:22.563464 23569 ts_tablet_manager.cc:1397] T d87d3b4755344b82b0aec7ffd1ce785c P 05a74e1eb2474b3d8b3a2da1ef127e01: Time spent bootstrapping tablet: real 0.143s user 0.022s sys 0.025s
I20250109 08:58:22.563588 23572 raft_consensus.cc:738] T e6b9aaeda9a947d4a87fedd774301cc0 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf55383bef6649eb9e78cb2d2512dedc, State: Initialized, Role: FOLLOWER
I20250109 08:58:22.504354 23606 raft_consensus.cc:513] T b572a6a3dfc04ea199bc703ddad8af67 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:22.565307 23574 tablet_bootstrap.cc:654] T a09d8dffc2e446f1acd3c7f6cc1705bb P 235ad6af04c34c88a5261daa3604fada: Neither blocks nor log segments found. Creating new log.
W20250109 08:58:22.565873 23494 leader_election.cc:343] T cc9886333c0343798226d59758a1ea5e P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:22.568769 23709 leader_election.cc:290] T 970a82e754db47b6b1a19551d4ba43d8 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 2 election: Requested vote from peers 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691), ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365)
I20250109 08:58:22.570118 23572 consensus_queue.cc:260] T e6b9aaeda9a947d4a87fedd774301cc0 P cf55383bef6649eb9e78cb2d2512dedc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:22.570711 23451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "940653b6d55c46439827697b9aadf16b" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc" is_pre_election: true
I20250109 08:58:22.571333 23451 raft_consensus.cc:2463] T 940653b6d55c46439827697b9aadf16b P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05a74e1eb2474b3d8b3a2da1ef127e01 in term 0.
I20250109 08:58:22.571895 23572 ts_tablet_manager.cc:1428] T e6b9aaeda9a947d4a87fedd774301cc0 P cf55383bef6649eb9e78cb2d2512dedc: Time spent starting tablet: real 0.042s user 0.003s sys 0.006s
I20250109 08:58:22.561352 23634 leader_election.cc:290] T 940653b6d55c46439827697b9aadf16b P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365), cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551)
I20250109 08:58:22.582568 23378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "940653b6d55c46439827697b9aadf16b" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" is_pre_election: true
I20250109 08:58:22.583384 23378 raft_consensus.cc:2463] T 940653b6d55c46439827697b9aadf16b P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05a74e1eb2474b3d8b3a2da1ef127e01 in term 0.
I20250109 08:58:22.585353 23302 raft_consensus.cc:3054] T 1c5b5beea4a840eb9677097cacee3408 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:22.585773 23451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "940653b6d55c46439827697b9aadf16b" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc" is_pre_election: true
I20250109 08:58:22.586419 23451 raft_consensus.cc:2463] T 940653b6d55c46439827697b9aadf16b P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed1efbce9ee3401e99a5260edeae4e7c in term 0.
I20250109 08:58:22.571903 23766 raft_consensus.cc:513] T 24ce131f7bb143bea1581d5b0c2a7109 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:22.593240 23346 leader_election.cc:304] T 940653b6d55c46439827697b9aadf16b P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf55383bef6649eb9e78cb2d2512dedc, ed1efbce9ee3401e99a5260edeae4e7c; no voters:
I20250109 08:58:22.593477 23766 leader_election.cc:290] T 24ce131f7bb143bea1581d5b0c2a7109 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 election: Requested vote from peers cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551), 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691)
I20250109 08:58:22.594303 23611 raft_consensus.cc:2798] T 940653b6d55c46439827697b9aadf16b P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:58:22.594700 23611 raft_consensus.cc:491] T 940653b6d55c46439827697b9aadf16b P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:58:22.595011 23611 raft_consensus.cc:3054] T 940653b6d55c46439827697b9aadf16b P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:22.574698 23606 leader_election.cc:290] T b572a6a3dfc04ea199bc703ddad8af67 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551), 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829)
I20250109 08:58:22.618551 23572 tablet_bootstrap.cc:492] T 8de6b2fb09bc46ce9e4033af06dffd04 P cf55383bef6649eb9e78cb2d2512dedc: Bootstrap starting.
I20250109 08:58:22.623709 23452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b572a6a3dfc04ea199bc703ddad8af67" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc" is_pre_election: true
I20250109 08:58:22.624850 23452 raft_consensus.cc:2388] T b572a6a3dfc04ea199bc703ddad8af67 P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 235ad6af04c34c88a5261daa3604fada in current term 1: Already voted for candidate 05a74e1eb2474b3d8b3a2da1ef127e01 in this term.
I20250109 08:58:22.627074 23611 raft_consensus.cc:513] T 940653b6d55c46439827697b9aadf16b P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:22.629379 23611 leader_election.cc:290] T 940653b6d55c46439827697b9aadf16b P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 election: Requested vote from peers cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551), 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829)
I20250109 08:58:22.629679 23658 consensus_queue.cc:1035] T a54ca9f0d6e844e9a8b1d9cd0bb2a026 P ed1efbce9ee3401e99a5260edeae4e7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:58:22.630096 23452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "940653b6d55c46439827697b9aadf16b" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc"
I20250109 08:58:22.630738 23452 raft_consensus.cc:3054] T 940653b6d55c46439827697b9aadf16b P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:22.594326 23527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "940653b6d55c46439827697b9aadf16b" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" is_pre_election: true
I20250109 08:58:22.632129 23527 raft_consensus.cc:2463] T 940653b6d55c46439827697b9aadf16b P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed1efbce9ee3401e99a5260edeae4e7c in term 0.
I20250109 08:58:22.633051 23303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "970a82e754db47b6b1a19551d4ba43d8" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada"
I20250109 08:58:22.633659 23303 raft_consensus.cc:3054] T 970a82e754db47b6b1a19551d4ba43d8 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:22.638669 23452 raft_consensus.cc:2463] T 940653b6d55c46439827697b9aadf16b P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed1efbce9ee3401e99a5260edeae4e7c in term 1.
I20250109 08:58:22.642370 23569 raft_consensus.cc:357] T d87d3b4755344b82b0aec7ffd1ce785c P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:22.643203 23569 raft_consensus.cc:383] T d87d3b4755344b82b0aec7ffd1ce785c P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:22.643513 23569 raft_consensus.cc:738] T d87d3b4755344b82b0aec7ffd1ce785c P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05a74e1eb2474b3d8b3a2da1ef127e01, State: Initialized, Role: FOLLOWER
I20250109 08:58:22.644050 23569 consensus_queue.cc:260] T d87d3b4755344b82b0aec7ffd1ce785c P 05a74e1eb2474b3d8b3a2da1ef127e01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:22.645680 23569 ts_tablet_manager.cc:1428] T d87d3b4755344b82b0aec7ffd1ce785c P 05a74e1eb2474b3d8b3a2da1ef127e01: Time spent starting tablet: real 0.082s user 0.004s sys 0.000s
I20250109 08:58:22.646385 23569 tablet_bootstrap.cc:492] T a686bf4d947c4b638049e6b88c59ae09 P 05a74e1eb2474b3d8b3a2da1ef127e01: Bootstrap starting.
I20250109 08:58:22.648816 23346 leader_election.cc:304] T 940653b6d55c46439827697b9aadf16b P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf55383bef6649eb9e78cb2d2512dedc, ed1efbce9ee3401e99a5260edeae4e7c; no voters:
I20250109 08:58:22.649691 23611 raft_consensus.cc:2798] T 940653b6d55c46439827697b9aadf16b P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:58:22.650127 23611 raft_consensus.cc:695] T 940653b6d55c46439827697b9aadf16b P ed1efbce9ee3401e99a5260edeae4e7c [term 1 LEADER]: Becoming Leader. State: Replica: ed1efbce9ee3401e99a5260edeae4e7c, State: Running, Role: LEADER
I20250109 08:58:22.650885 23611 consensus_queue.cc:237] T 940653b6d55c46439827697b9aadf16b P ed1efbce9ee3401e99a5260edeae4e7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:22.653354 23454 raft_consensus.cc:2463] T 86d4074509744900abe94402467a7d85 P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed1efbce9ee3401e99a5260edeae4e7c in term 1.
I20250109 08:58:22.654537 23346 leader_election.cc:304] T 86d4074509744900abe94402467a7d85 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf55383bef6649eb9e78cb2d2512dedc, ed1efbce9ee3401e99a5260edeae4e7c; no voters: 235ad6af04c34c88a5261daa3604fada
I20250109 08:58:22.655352 23658 raft_consensus.cc:2798] T 86d4074509744900abe94402467a7d85 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:58:22.655812 23658 raft_consensus.cc:695] T 86d4074509744900abe94402467a7d85 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 LEADER]: Becoming Leader. State: Replica: ed1efbce9ee3401e99a5260edeae4e7c, State: Running, Role: LEADER
I20250109 08:58:22.656636 23658 consensus_queue.cc:237] T 86d4074509744900abe94402467a7d85 P ed1efbce9ee3401e99a5260edeae4e7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:22.658895 23572 tablet_bootstrap.cc:654] T 8de6b2fb09bc46ce9e4033af06dffd04 P cf55383bef6649eb9e78cb2d2512dedc: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:22.659781 23730 consensus_queue.cc:1035] T 1c5b5beea4a840eb9677097cacee3408 P ed1efbce9ee3401e99a5260edeae4e7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:58:22.662952 23303 raft_consensus.cc:2463] T 970a82e754db47b6b1a19551d4ba43d8 P 235ad6af04c34c88a5261daa3604fada [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05a74e1eb2474b3d8b3a2da1ef127e01 in term 2.
I20250109 08:58:22.682818 23378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "970a82e754db47b6b1a19551d4ba43d8" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c"
I20250109 08:58:22.683488 23378 raft_consensus.cc:3054] T 970a82e754db47b6b1a19551d4ba43d8 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:22.684294 23580 tablet_bootstrap.cc:492] T 08214ee9c4c14bda92bdc6fa8231c313 P ed1efbce9ee3401e99a5260edeae4e7c: No bootstrap required, opened a new log
I20250109 08:58:22.684854 23580 ts_tablet_manager.cc:1397] T 08214ee9c4c14bda92bdc6fa8231c313 P ed1efbce9ee3401e99a5260edeae4e7c: Time spent bootstrapping tablet: real 0.212s user 0.015s sys 0.026s
I20250109 08:58:22.686564 23528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b572a6a3dfc04ea199bc703ddad8af67" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" is_pre_election: true
I20250109 08:58:22.687310 23580 raft_consensus.cc:357] T 08214ee9c4c14bda92bdc6fa8231c313 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:22.689251 23580 raft_consensus.cc:383] T 08214ee9c4c14bda92bdc6fa8231c313 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:22.689576 23580 raft_consensus.cc:738] T 08214ee9c4c14bda92bdc6fa8231c313 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed1efbce9ee3401e99a5260edeae4e7c, State: Initialized, Role: FOLLOWER
I20250109 08:58:22.690196 23580 consensus_queue.cc:260] T 08214ee9c4c14bda92bdc6fa8231c313 P ed1efbce9ee3401e99a5260edeae4e7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:22.695847 23580 ts_tablet_manager.cc:1428] T 08214ee9c4c14bda92bdc6fa8231c313 P ed1efbce9ee3401e99a5260edeae4e7c: Time spent starting tablet: real 0.011s user 0.007s sys 0.000s
I20250109 08:58:22.701603 23580 tablet_bootstrap.cc:492] T a686bf4d947c4b638049e6b88c59ae09 P ed1efbce9ee3401e99a5260edeae4e7c: Bootstrap starting.
I20250109 08:58:22.701854 23269 leader_election.cc:304] T b572a6a3dfc04ea199bc703ddad8af67 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 235ad6af04c34c88a5261daa3604fada; no voters: 05a74e1eb2474b3d8b3a2da1ef127e01, cf55383bef6649eb9e78cb2d2512dedc
I20250109 08:58:22.702701 23574 tablet_bootstrap.cc:492] T a09d8dffc2e446f1acd3c7f6cc1705bb P 235ad6af04c34c88a5261daa3604fada: No bootstrap required, opened a new log
I20250109 08:58:22.703218 23666 raft_consensus.cc:3054] T b572a6a3dfc04ea199bc703ddad8af67 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:22.703639 23574 ts_tablet_manager.cc:1397] T a09d8dffc2e446f1acd3c7f6cc1705bb P 235ad6af04c34c88a5261daa3604fada: Time spent bootstrapping tablet: real 0.213s user 0.025s sys 0.046s
I20250109 08:58:22.707526 23452 raft_consensus.cc:3054] T a54ca9f0d6e844e9a8b1d9cd0bb2a026 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:22.709157 23766 raft_consensus.cc:491] T 5831288451724d4ca1114ec5e5f01b78 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:22.709664 23766 raft_consensus.cc:513] T 5831288451724d4ca1114ec5e5f01b78 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:22.707949 23203 catalog_manager.cc:5526] T cc9886333c0343798226d59758a1ea5e P 05a74e1eb2474b3d8b3a2da1ef127e01 reported cstate change: term changed from 0 to 1, leader changed from <none> to 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196). New cstate: current_term: 1 leader_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:22.711542 23580 tablet_bootstrap.cc:654] T a686bf4d947c4b638049e6b88c59ae09 P ed1efbce9ee3401e99a5260edeae4e7c: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:22.706198 23574 raft_consensus.cc:357] T a09d8dffc2e446f1acd3c7f6cc1705bb P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:22.712813 23574 raft_consensus.cc:383] T a09d8dffc2e446f1acd3c7f6cc1705bb P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:22.713030 23574 raft_consensus.cc:738] T a09d8dffc2e446f1acd3c7f6cc1705bb P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Initialized, Role: FOLLOWER
I20250109 08:58:22.714388 23574 consensus_queue.cc:260] T a09d8dffc2e446f1acd3c7f6cc1705bb P 235ad6af04c34c88a5261daa3604fada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:22.713181 23378 raft_consensus.cc:2463] T 970a82e754db47b6b1a19551d4ba43d8 P ed1efbce9ee3401e99a5260edeae4e7c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05a74e1eb2474b3d8b3a2da1ef127e01 in term 2.
I20250109 08:58:22.718286 23494 leader_election.cc:304] T 940653b6d55c46439827697b9aadf16b P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01, ed1efbce9ee3401e99a5260edeae4e7c; no voters:
I20250109 08:58:22.724396 23574 ts_tablet_manager.cc:1428] T a09d8dffc2e446f1acd3c7f6cc1705bb P 235ad6af04c34c88a5261daa3604fada: Time spent starting tablet: real 0.020s user 0.004s sys 0.000s
I20250109 08:58:22.724469 23301 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24ce131f7bb143bea1581d5b0c2a7109" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada"
I20250109 08:58:22.725131 23301 raft_consensus.cc:3054] T 24ce131f7bb143bea1581d5b0c2a7109 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:22.721722 23634 raft_consensus.cc:2798] T 940653b6d55c46439827697b9aadf16b P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:58:22.728238 23634 raft_consensus.cc:491] T 940653b6d55c46439827697b9aadf16b P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:58:22.728614 23634 raft_consensus.cc:3054] T 940653b6d55c46439827697b9aadf16b P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:22.733758 23574 tablet_bootstrap.cc:492] T 08214ee9c4c14bda92bdc6fa8231c313 P 235ad6af04c34c88a5261daa3604fada: Bootstrap starting.
I20250109 08:58:22.711684 23766 leader_election.cc:290] T 5831288451724d4ca1114ec5e5f01b78 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365), cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551)
I20250109 08:58:22.738296 23301 raft_consensus.cc:2463] T 24ce131f7bb143bea1581d5b0c2a7109 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05a74e1eb2474b3d8b3a2da1ef127e01 in term 1.
I20250109 08:58:22.739715 23203 catalog_manager.cc:5526] T 940653b6d55c46439827697b9aadf16b P ed1efbce9ee3401e99a5260edeae4e7c reported cstate change: term changed from 0 to 1, leader changed from <none> to ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194). New cstate: current_term: 1 leader_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:22.743811 23627 raft_consensus.cc:491] T e6b9aaeda9a947d4a87fedd774301cc0 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:22.744544 23627 raft_consensus.cc:513] T e6b9aaeda9a947d4a87fedd774301cc0 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:22.745846 23203 catalog_manager.cc:5526] T 86d4074509744900abe94402467a7d85 P ed1efbce9ee3401e99a5260edeae4e7c reported cstate change: term changed from 0 to 1, leader changed from <none> to ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194). New cstate: current_term: 1 leader_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:22.721020 23454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24ce131f7bb143bea1581d5b0c2a7109" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc"
I20250109 08:58:22.741750 23634 raft_consensus.cc:513] T 940653b6d55c46439827697b9aadf16b P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:22.757287 23634 leader_election.cc:290] T 940653b6d55c46439827697b9aadf16b P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 election: Requested vote from peers ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365), cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551)
I20250109 08:58:22.754585 23574 tablet_bootstrap.cc:654] T 08214ee9c4c14bda92bdc6fa8231c313 P 235ad6af04c34c88a5261daa3604fada: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:22.755179 23627 leader_election.cc:290] T e6b9aaeda9a947d4a87fedd774301cc0 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691), ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365)
I20250109 08:58:22.758774 23377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6b9aaeda9a947d4a87fedd774301cc0" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" is_pre_election: true
I20250109 08:58:22.759461 23377 raft_consensus.cc:2463] T e6b9aaeda9a947d4a87fedd774301cc0 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf55383bef6649eb9e78cb2d2512dedc in term 0.
I20250109 08:58:22.760151 23494 leader_election.cc:304] T 970a82e754db47b6b1a19551d4ba43d8 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01, 235ad6af04c34c88a5261daa3604fada; no voters:
I20250109 08:58:22.760643 23450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5831288451724d4ca1114ec5e5f01b78" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc" is_pre_election: true
I20250109 08:58:22.761466 23766 raft_consensus.cc:2798] T 970a82e754db47b6b1a19551d4ba43d8 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:58:22.761948 23766 raft_consensus.cc:695] T 970a82e754db47b6b1a19551d4ba43d8 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 LEADER]: Becoming Leader. State: Replica: 05a74e1eb2474b3d8b3a2da1ef127e01, State: Running, Role: LEADER
I20250109 08:58:22.762878 23494 leader_election.cc:304] T 24ce131f7bb143bea1581d5b0c2a7109 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01, 235ad6af04c34c88a5261daa3604fada; no voters:
I20250109 08:58:22.762774 23766 consensus_queue.cc:237] T 970a82e754db47b6b1a19551d4ba43d8 P 05a74e1eb2474b3d8b3a2da1ef127e01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:22.763638 23709 raft_consensus.cc:2798] T 24ce131f7bb143bea1581d5b0c2a7109 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:58:22.764096 23709 raft_consensus.cc:695] T 24ce131f7bb143bea1581d5b0c2a7109 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 LEADER]: Becoming Leader. State: Replica: 05a74e1eb2474b3d8b3a2da1ef127e01, State: Running, Role: LEADER
I20250109 08:58:22.764901 23709 consensus_queue.cc:237] T 24ce131f7bb143bea1581d5b0c2a7109 P 05a74e1eb2474b3d8b3a2da1ef127e01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:22.766134 23527 raft_consensus.cc:3054] T 5831288451724d4ca1114ec5e5f01b78 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:22.765910 23635 consensus_queue.cc:1035] T a54ca9f0d6e844e9a8b1d9cd0bb2a026 P ed1efbce9ee3401e99a5260edeae4e7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:58:22.766909 23420 leader_election.cc:304] T e6b9aaeda9a947d4a87fedd774301cc0 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf55383bef6649eb9e78cb2d2512dedc, ed1efbce9ee3401e99a5260edeae4e7c; no voters:
I20250109 08:58:22.767741 23677 raft_consensus.cc:491] T a54ca9f0d6e844e9a8b1d9cd0bb2a026 P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:22.765815 23627 consensus_queue.cc:1035] T 5831288451724d4ca1114ec5e5f01b78 P cf55383bef6649eb9e78cb2d2512dedc [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:58:22.768008 23675 raft_consensus.cc:2798] T e6b9aaeda9a947d4a87fedd774301cc0 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:58:22.769713 23677 raft_consensus.cc:513] T a54ca9f0d6e844e9a8b1d9cd0bb2a026 P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:22.771942 23378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "940653b6d55c46439827697b9aadf16b" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c"
I20250109 08:58:22.760780 23377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5831288451724d4ca1114ec5e5f01b78" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" is_pre_election: true
I20250109 08:58:22.773023 23377 raft_consensus.cc:2388] T 5831288451724d4ca1114ec5e5f01b78 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 05a74e1eb2474b3d8b3a2da1ef127e01 in current term 1: Already voted for candidate cf55383bef6649eb9e78cb2d2512dedc in this term.
I20250109 08:58:22.774017 23452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "940653b6d55c46439827697b9aadf16b" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc"
I20250109 08:58:22.770226 23675 raft_consensus.cc:491] T e6b9aaeda9a947d4a87fedd774301cc0 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:58:22.778487 23675 raft_consensus.cc:3054] T e6b9aaeda9a947d4a87fedd774301cc0 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Advancing to term 1
W20250109 08:58:22.778673 23494 leader_election.cc:343] T 24ce131f7bb143bea1581d5b0c2a7109 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:22.779340 23305 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6b9aaeda9a947d4a87fedd774301cc0" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada" is_pre_election: true
I20250109 08:58:22.780324 23494 leader_election.cc:304] T 5831288451724d4ca1114ec5e5f01b78 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01; no voters: cf55383bef6649eb9e78cb2d2512dedc, ed1efbce9ee3401e99a5260edeae4e7c
I20250109 08:58:22.784621 23677 leader_election.cc:290] T a54ca9f0d6e844e9a8b1d9cd0bb2a026 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691), ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365)
I20250109 08:58:22.786432 23634 raft_consensus.cc:2743] T 5831288451724d4ca1114ec5e5f01b78 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:58:22.793220 23572 tablet_bootstrap.cc:492] T 8de6b2fb09bc46ce9e4033af06dffd04 P cf55383bef6649eb9e78cb2d2512dedc: No bootstrap required, opened a new log
I20250109 08:58:22.793731 23572 ts_tablet_manager.cc:1397] T 8de6b2fb09bc46ce9e4033af06dffd04 P cf55383bef6649eb9e78cb2d2512dedc: Time spent bootstrapping tablet: real 0.175s user 0.025s sys 0.016s
I20250109 08:58:22.786178 23666 raft_consensus.cc:2743] T b572a6a3dfc04ea199bc703ddad8af67 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:58:22.794574 23452 raft_consensus.cc:2388] T 940653b6d55c46439827697b9aadf16b P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 05a74e1eb2474b3d8b3a2da1ef127e01 in current term 1: Already voted for candidate ed1efbce9ee3401e99a5260edeae4e7c in this term.
I20250109 08:58:22.787895 23378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a54ca9f0d6e844e9a8b1d9cd0bb2a026" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" is_pre_election: true
I20250109 08:58:22.822682 23494 leader_election.cc:304] T 940653b6d55c46439827697b9aadf16b P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01; no voters: cf55383bef6649eb9e78cb2d2512dedc, ed1efbce9ee3401e99a5260edeae4e7c
I20250109 08:58:22.787937 23305 raft_consensus.cc:2463] T e6b9aaeda9a947d4a87fedd774301cc0 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf55383bef6649eb9e78cb2d2512dedc in term 0.
I20250109 08:58:22.803694 23572 raft_consensus.cc:357] T 8de6b2fb09bc46ce9e4033af06dffd04 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:22.823743 23572 raft_consensus.cc:383] T 8de6b2fb09bc46ce9e4033af06dffd04 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:22.823786 23709 raft_consensus.cc:2743] T 940653b6d55c46439827697b9aadf16b P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20250109 08:58:22.824108 23572 raft_consensus.cc:738] T 8de6b2fb09bc46ce9e4033af06dffd04 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf55383bef6649eb9e78cb2d2512dedc, State: Initialized, Role: FOLLOWER
I20250109 08:58:22.824792 23572 consensus_queue.cc:260] T 8de6b2fb09bc46ce9e4033af06dffd04 P cf55383bef6649eb9e78cb2d2512dedc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:22.826930 23572 ts_tablet_manager.cc:1428] T 8de6b2fb09bc46ce9e4033af06dffd04 P cf55383bef6649eb9e78cb2d2512dedc: Time spent starting tablet: real 0.033s user 0.005s sys 0.000s
I20250109 08:58:22.807390 23606 consensus_queue.cc:1035] T 2ab70546dda143b5b5db15c3875aa348 P 235ad6af04c34c88a5261daa3604fada [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:58:22.808245 23302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a54ca9f0d6e844e9a8b1d9cd0bb2a026" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada" is_pre_election: true
I20250109 08:58:22.816143 23677 consensus_queue.cc:1035] T d5fa255ab5f24750a0afa6d81ec869f2 P cf55383bef6649eb9e78cb2d2512dedc [LEADER]: Connected to new peer: Peer: permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:58:22.817346 23627 consensus_queue.cc:1035] T 5831288451724d4ca1114ec5e5f01b78 P cf55383bef6649eb9e78cb2d2512dedc [LEADER]: Connected to new peer: Peer: permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:58:22.829442 23420 leader_election.cc:304] T a54ca9f0d6e844e9a8b1d9cd0bb2a026 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf55383bef6649eb9e78cb2d2512dedc; no voters: 235ad6af04c34c88a5261daa3604fada, ed1efbce9ee3401e99a5260edeae4e7c
I20250109 08:58:22.830415 23572 tablet_bootstrap.cc:492] T 9b245cb3666c421c884cf20161a22831 P cf55383bef6649eb9e78cb2d2512dedc: Bootstrap starting.
I20250109 08:58:22.830897 23677 raft_consensus.cc:2743] T a54ca9f0d6e844e9a8b1d9cd0bb2a026 P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250109 08:58:22.818532 23675 raft_consensus.cc:513] T e6b9aaeda9a947d4a87fedd774301cc0 P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:22.833215 23574 tablet_bootstrap.cc:492] T 08214ee9c4c14bda92bdc6fa8231c313 P 235ad6af04c34c88a5261daa3604fada: No bootstrap required, opened a new log
I20250109 08:58:22.833667 23574 ts_tablet_manager.cc:1397] T 08214ee9c4c14bda92bdc6fa8231c313 P 235ad6af04c34c88a5261daa3604fada: Time spent bootstrapping tablet: real 0.100s user 0.015s sys 0.013s
I20250109 08:58:22.834638 23378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6b9aaeda9a947d4a87fedd774301cc0" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c"
I20250109 08:58:22.835248 23378 raft_consensus.cc:3054] T e6b9aaeda9a947d4a87fedd774301cc0 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:22.836215 23574 raft_consensus.cc:357] T 08214ee9c4c14bda92bdc6fa8231c313 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:22.836910 23574 raft_consensus.cc:383] T 08214ee9c4c14bda92bdc6fa8231c313 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:22.837155 23574 raft_consensus.cc:738] T 08214ee9c4c14bda92bdc6fa8231c313 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Initialized, Role: FOLLOWER
I20250109 08:58:22.837237 23572 tablet_bootstrap.cc:654] T 9b245cb3666c421c884cf20161a22831 P cf55383bef6649eb9e78cb2d2512dedc: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:22.837913 23574 consensus_queue.cc:260] T 08214ee9c4c14bda92bdc6fa8231c313 P 235ad6af04c34c88a5261daa3604fada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:22.840193 23574 ts_tablet_manager.cc:1428] T 08214ee9c4c14bda92bdc6fa8231c313 P 235ad6af04c34c88a5261daa3604fada: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
I20250109 08:58:22.841077 23574 tablet_bootstrap.cc:492] T cc9886333c0343798226d59758a1ea5e P 235ad6af04c34c88a5261daa3604fada: Bootstrap starting.
I20250109 08:58:22.841255 23302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6b9aaeda9a947d4a87fedd774301cc0" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada"
I20250109 08:58:22.841946 23302 raft_consensus.cc:3054] T e6b9aaeda9a947d4a87fedd774301cc0 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:22.844353 23378 raft_consensus.cc:2463] T e6b9aaeda9a947d4a87fedd774301cc0 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf55383bef6649eb9e78cb2d2512dedc in term 1.
I20250109 08:58:22.821347 23527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "940653b6d55c46439827697b9aadf16b" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01"
I20250109 08:58:22.854985 23527 raft_consensus.cc:2388] T 940653b6d55c46439827697b9aadf16b P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate ed1efbce9ee3401e99a5260edeae4e7c in current term 1: Already voted for candidate 05a74e1eb2474b3d8b3a2da1ef127e01 in this term.
I20250109 08:58:22.856102 23420 leader_election.cc:304] T e6b9aaeda9a947d4a87fedd774301cc0 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf55383bef6649eb9e78cb2d2512dedc, ed1efbce9ee3401e99a5260edeae4e7c; no voters:
I20250109 08:58:22.847296 23709 consensus_queue.cc:1035] T b572a6a3dfc04ea199bc703ddad8af67 P 05a74e1eb2474b3d8b3a2da1ef127e01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:58:22.878512 23572 tablet_bootstrap.cc:492] T 9b245cb3666c421c884cf20161a22831 P cf55383bef6649eb9e78cb2d2512dedc: No bootstrap required, opened a new log
I20250109 08:58:22.833143 23675 leader_election.cc:290] T e6b9aaeda9a947d4a87fedd774301cc0 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 election: Requested vote from peers 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691), ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365)
I20250109 08:58:22.862536 23658 consensus_queue.cc:1035] T 1c5b5beea4a840eb9677097cacee3408 P ed1efbce9ee3401e99a5260edeae4e7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:58:22.863107 23302 raft_consensus.cc:2463] T e6b9aaeda9a947d4a87fedd774301cc0 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf55383bef6649eb9e78cb2d2512dedc in term 1.
I20250109 08:58:22.863668 23524 raft_consensus.cc:3054] T 4df64aae69ca41dc91286cae1603000f P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:22.881484 23572 ts_tablet_manager.cc:1397] T 9b245cb3666c421c884cf20161a22831 P cf55383bef6649eb9e78cb2d2512dedc: Time spent bootstrapping tablet: real 0.051s user 0.010s sys 0.002s
I20250109 08:58:22.863390 23586 consensus_queue.cc:1035] T 2ab70546dda143b5b5db15c3875aa348 P 235ad6af04c34c88a5261daa3604fada [LEADER]: Connected to new peer: Peer: permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20250109 08:58:22.846407 23574 tablet_bootstrap.cc:654] T cc9886333c0343798226d59758a1ea5e P 235ad6af04c34c88a5261daa3604fada: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:22.878479 23583 raft_consensus.cc:2798] T e6b9aaeda9a947d4a87fedd774301cc0 P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:58:22.885056 23583 raft_consensus.cc:695] T e6b9aaeda9a947d4a87fedd774301cc0 P cf55383bef6649eb9e78cb2d2512dedc [term 1 LEADER]: Becoming Leader. State: Replica: cf55383bef6649eb9e78cb2d2512dedc, State: Running, Role: LEADER
I20250109 08:58:22.884085 23572 raft_consensus.cc:357] T 9b245cb3666c421c884cf20161a22831 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:22.886917 23572 raft_consensus.cc:383] T 9b245cb3666c421c884cf20161a22831 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:22.887207 23572 raft_consensus.cc:738] T 9b245cb3666c421c884cf20161a22831 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf55383bef6649eb9e78cb2d2512dedc, State: Initialized, Role: FOLLOWER
I20250109 08:58:22.819658 23569 tablet_bootstrap.cc:654] T a686bf4d947c4b638049e6b88c59ae09 P 05a74e1eb2474b3d8b3a2da1ef127e01: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:22.887848 23572 consensus_queue.cc:260] T 9b245cb3666c421c884cf20161a22831 P cf55383bef6649eb9e78cb2d2512dedc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:22.889925 23572 ts_tablet_manager.cc:1428] T 9b245cb3666c421c884cf20161a22831 P cf55383bef6649eb9e78cb2d2512dedc: Time spent starting tablet: real 0.008s user 0.005s sys 0.000s
I20250109 08:58:22.891026 23572 tablet_bootstrap.cc:492] T a09d8dffc2e446f1acd3c7f6cc1705bb P cf55383bef6649eb9e78cb2d2512dedc: Bootstrap starting.
I20250109 08:58:22.885885 23583 consensus_queue.cc:237] T e6b9aaeda9a947d4a87fedd774301cc0 P cf55383bef6649eb9e78cb2d2512dedc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:22.893723 23709 consensus_queue.cc:1035] T b572a6a3dfc04ea199bc703ddad8af67 P 05a74e1eb2474b3d8b3a2da1ef127e01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:58:22.897256 23627 raft_consensus.cc:491] T 9b245cb3666c421c884cf20161a22831 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:22.897380 23572 tablet_bootstrap.cc:654] T a09d8dffc2e446f1acd3c7f6cc1705bb P cf55383bef6649eb9e78cb2d2512dedc: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:22.899379 23627 raft_consensus.cc:513] T 9b245cb3666c421c884cf20161a22831 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:22.901332 23627 leader_election.cc:290] T 9b245cb3666c421c884cf20161a22831 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829), 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691)
I20250109 08:58:22.908771 23572 tablet_bootstrap.cc:492] T a09d8dffc2e446f1acd3c7f6cc1705bb P cf55383bef6649eb9e78cb2d2512dedc: No bootstrap required, opened a new log
I20250109 08:58:22.909322 23572 ts_tablet_manager.cc:1397] T a09d8dffc2e446f1acd3c7f6cc1705bb P cf55383bef6649eb9e78cb2d2512dedc: Time spent bootstrapping tablet: real 0.018s user 0.012s sys 0.004s
I20250109 08:58:22.911950 23572 raft_consensus.cc:357] T a09d8dffc2e446f1acd3c7f6cc1705bb P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:22.912708 23572 raft_consensus.cc:383] T a09d8dffc2e446f1acd3c7f6cc1705bb P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:22.913136 23572 raft_consensus.cc:738] T a09d8dffc2e446f1acd3c7f6cc1705bb P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf55383bef6649eb9e78cb2d2512dedc, State: Initialized, Role: FOLLOWER
I20250109 08:58:22.913753 23572 consensus_queue.cc:260] T a09d8dffc2e446f1acd3c7f6cc1705bb P cf55383bef6649eb9e78cb2d2512dedc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:22.916122 23302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b245cb3666c421c884cf20161a22831" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada" is_pre_election: true
I20250109 08:58:22.919986 23572 ts_tablet_manager.cc:1428] T a09d8dffc2e446f1acd3c7f6cc1705bb P cf55383bef6649eb9e78cb2d2512dedc: Time spent starting tablet: real 0.010s user 0.007s sys 0.000s
I20250109 08:58:22.921083 23572 tablet_bootstrap.cc:492] T 24ce131f7bb143bea1581d5b0c2a7109 P cf55383bef6649eb9e78cb2d2512dedc: Bootstrap starting.
I20250109 08:58:22.920439 23203 catalog_manager.cc:5526] T 970a82e754db47b6b1a19551d4ba43d8 P 05a74e1eb2474b3d8b3a2da1ef127e01 reported cstate change: term changed from 0 to 2, leader changed from <none> to 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196). New cstate: current_term: 2 leader_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:22.922104 23203 catalog_manager.cc:5526] T 24ce131f7bb143bea1581d5b0c2a7109 P 05a74e1eb2474b3d8b3a2da1ef127e01 reported cstate change: term changed from 0 to 1, leader changed from <none> to 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196). New cstate: current_term: 1 leader_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:22.924299 23527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b245cb3666c421c884cf20161a22831" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" is_pre_election: true
W20250109 08:58:22.925900 23418 consensus_peers.cc:487] T 5c37c58abaa54656b31c72160b4dec9e P cf55383bef6649eb9e78cb2d2512dedc -> Peer 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829): Couldn't send request to peer 05a74e1eb2474b3d8b3a2da1ef127e01. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:22.927574 23572 tablet_bootstrap.cc:654] T 24ce131f7bb143bea1581d5b0c2a7109 P cf55383bef6649eb9e78cb2d2512dedc: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:22.927886 23418 leader_election.cc:304] T 9b245cb3666c421c884cf20161a22831 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf55383bef6649eb9e78cb2d2512dedc; no voters: 05a74e1eb2474b3d8b3a2da1ef127e01, 235ad6af04c34c88a5261daa3604fada
I20250109 08:58:22.929205 23677 raft_consensus.cc:2743] T 9b245cb3666c421c884cf20161a22831 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:58:22.929873 23580 tablet_bootstrap.cc:492] T a686bf4d947c4b638049e6b88c59ae09 P ed1efbce9ee3401e99a5260edeae4e7c: No bootstrap required, opened a new log
I20250109 08:58:22.930437 23580 ts_tablet_manager.cc:1397] T a686bf4d947c4b638049e6b88c59ae09 P ed1efbce9ee3401e99a5260edeae4e7c: Time spent bootstrapping tablet: real 0.231s user 0.011s sys 0.003s
I20250109 08:58:22.933182 23580 raft_consensus.cc:357] T a686bf4d947c4b638049e6b88c59ae09 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:22.933944 23580 raft_consensus.cc:383] T a686bf4d947c4b638049e6b88c59ae09 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:22.934221 23580 raft_consensus.cc:738] T a686bf4d947c4b638049e6b88c59ae09 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed1efbce9ee3401e99a5260edeae4e7c, State: Initialized, Role: FOLLOWER
I20250109 08:58:22.938306 23580 consensus_queue.cc:260] T a686bf4d947c4b638049e6b88c59ae09 P ed1efbce9ee3401e99a5260edeae4e7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:22.943007 23635 raft_consensus.cc:491] T 08214ee9c4c14bda92bdc6fa8231c313 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:22.943490 23635 raft_consensus.cc:513] T 08214ee9c4c14bda92bdc6fa8231c313 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:22.946215 23580 ts_tablet_manager.cc:1428] T a686bf4d947c4b638049e6b88c59ae09 P ed1efbce9ee3401e99a5260edeae4e7c: Time spent starting tablet: real 0.015s user 0.005s sys 0.001s
I20250109 08:58:22.947309 23580 tablet_bootstrap.cc:492] T 750dd1e1055f4570b0aecdda3ebe8772 P ed1efbce9ee3401e99a5260edeae4e7c: Bootstrap starting.
I20250109 08:58:22.948558 23635 leader_election.cc:290] T 08214ee9c4c14bda92bdc6fa8231c313 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829), 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691)
I20250109 08:58:22.950556 23302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08214ee9c4c14bda92bdc6fa8231c313" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada" is_pre_election: true
I20250109 08:58:22.951279 23302 raft_consensus.cc:2463] T 08214ee9c4c14bda92bdc6fa8231c313 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed1efbce9ee3401e99a5260edeae4e7c in term 0.
I20250109 08:58:22.952535 23572 tablet_bootstrap.cc:492] T 24ce131f7bb143bea1581d5b0c2a7109 P cf55383bef6649eb9e78cb2d2512dedc: No bootstrap required, opened a new log
I20250109 08:58:22.952478 23346 leader_election.cc:304] T 08214ee9c4c14bda92bdc6fa8231c313 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 235ad6af04c34c88a5261daa3604fada, ed1efbce9ee3401e99a5260edeae4e7c; no voters:
I20250109 08:58:22.953092 23572 ts_tablet_manager.cc:1397] T 24ce131f7bb143bea1581d5b0c2a7109 P cf55383bef6649eb9e78cb2d2512dedc: Time spent bootstrapping tablet: real 0.032s user 0.027s sys 0.004s
I20250109 08:58:22.953339 23635 raft_consensus.cc:2798] T 08214ee9c4c14bda92bdc6fa8231c313 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:58:22.953742 23635 raft_consensus.cc:491] T 08214ee9c4c14bda92bdc6fa8231c313 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:58:22.954056 23635 raft_consensus.cc:3054] T 08214ee9c4c14bda92bdc6fa8231c313 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:22.955648 23572 raft_consensus.cc:357] T 24ce131f7bb143bea1581d5b0c2a7109 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:22.956404 23572 raft_consensus.cc:383] T 24ce131f7bb143bea1581d5b0c2a7109 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:22.956652 23572 raft_consensus.cc:738] T 24ce131f7bb143bea1581d5b0c2a7109 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf55383bef6649eb9e78cb2d2512dedc, State: Initialized, Role: FOLLOWER
I20250109 08:58:22.957290 23580 tablet_bootstrap.cc:654] T 750dd1e1055f4570b0aecdda3ebe8772 P ed1efbce9ee3401e99a5260edeae4e7c: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:22.957509 23572 consensus_queue.cc:260] T 24ce131f7bb143bea1581d5b0c2a7109 P cf55383bef6649eb9e78cb2d2512dedc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:22.959321 23528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08214ee9c4c14bda92bdc6fa8231c313" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" is_pre_election: true
I20250109 08:58:22.959532 23572 ts_tablet_manager.cc:1428] T 24ce131f7bb143bea1581d5b0c2a7109 P cf55383bef6649eb9e78cb2d2512dedc: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20250109 08:58:22.959987 23528 raft_consensus.cc:2463] T 08214ee9c4c14bda92bdc6fa8231c313 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed1efbce9ee3401e99a5260edeae4e7c in term 0.
I20250109 08:58:22.960615 23572 tablet_bootstrap.cc:492] T db111e41c1be497e8e673a2c42b9ba66 P cf55383bef6649eb9e78cb2d2512dedc: Bootstrap starting.
I20250109 08:58:22.961267 23635 raft_consensus.cc:513] T 08214ee9c4c14bda92bdc6fa8231c313 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:22.967442 23572 tablet_bootstrap.cc:654] T db111e41c1be497e8e673a2c42b9ba66 P cf55383bef6649eb9e78cb2d2512dedc: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:22.971103 23528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08214ee9c4c14bda92bdc6fa8231c313" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01"
I20250109 08:58:22.971750 23528 raft_consensus.cc:3054] T 08214ee9c4c14bda92bdc6fa8231c313 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:22.972623 23580 tablet_bootstrap.cc:492] T 750dd1e1055f4570b0aecdda3ebe8772 P ed1efbce9ee3401e99a5260edeae4e7c: No bootstrap required, opened a new log
I20250109 08:58:22.973158 23580 ts_tablet_manager.cc:1397] T 750dd1e1055f4570b0aecdda3ebe8772 P ed1efbce9ee3401e99a5260edeae4e7c: Time spent bootstrapping tablet: real 0.026s user 0.011s sys 0.000s
I20250109 08:58:22.976883 23572 tablet_bootstrap.cc:492] T db111e41c1be497e8e673a2c42b9ba66 P cf55383bef6649eb9e78cb2d2512dedc: No bootstrap required, opened a new log
I20250109 08:58:22.977480 23572 ts_tablet_manager.cc:1397] T db111e41c1be497e8e673a2c42b9ba66 P cf55383bef6649eb9e78cb2d2512dedc: Time spent bootstrapping tablet: real 0.017s user 0.011s sys 0.003s
I20250109 08:58:22.978789 23528 raft_consensus.cc:2463] T 08214ee9c4c14bda92bdc6fa8231c313 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed1efbce9ee3401e99a5260edeae4e7c in term 1.
I20250109 08:58:22.979231 23580 raft_consensus.cc:357] T 750dd1e1055f4570b0aecdda3ebe8772 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:22.979955 23580 raft_consensus.cc:383] T 750dd1e1055f4570b0aecdda3ebe8772 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:22.980170 23572 raft_consensus.cc:357] T db111e41c1be497e8e673a2c42b9ba66 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:22.980835 23658 raft_consensus.cc:491] T a686bf4d947c4b638049e6b88c59ae09 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:22.980932 23572 raft_consensus.cc:383] T db111e41c1be497e8e673a2c42b9ba66 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:22.981462 23572 raft_consensus.cc:738] T db111e41c1be497e8e673a2c42b9ba66 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf55383bef6649eb9e78cb2d2512dedc, State: Initialized, Role: FOLLOWER
I20250109 08:58:22.980213 23580 raft_consensus.cc:738] T 750dd1e1055f4570b0aecdda3ebe8772 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed1efbce9ee3401e99a5260edeae4e7c, State: Initialized, Role: FOLLOWER
I20250109 08:58:22.981411 23658 raft_consensus.cc:513] T a686bf4d947c4b638049e6b88c59ae09 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:22.982311 23572 consensus_queue.cc:260] T db111e41c1be497e8e673a2c42b9ba66 P cf55383bef6649eb9e78cb2d2512dedc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:22.982883 23580 consensus_queue.cc:260] T 750dd1e1055f4570b0aecdda3ebe8772 P ed1efbce9ee3401e99a5260edeae4e7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:22.983505 23658 leader_election.cc:290] T a686bf4d947c4b638049e6b88c59ae09 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551), 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829)
I20250109 08:58:22.984737 23344 leader_election.cc:304] T 08214ee9c4c14bda92bdc6fa8231c313 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01, ed1efbce9ee3401e99a5260edeae4e7c; no voters:
I20250109 08:58:22.985009 23572 ts_tablet_manager.cc:1428] T db111e41c1be497e8e673a2c42b9ba66 P cf55383bef6649eb9e78cb2d2512dedc: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20250109 08:58:22.986083 23572 tablet_bootstrap.cc:492] T 5f0d500a91b54851bda52e5ccf741fd6 P cf55383bef6649eb9e78cb2d2512dedc: Bootstrap starting.
I20250109 08:58:22.986387 23523 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a686bf4d947c4b638049e6b88c59ae09" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" is_pre_election: true
I20250109 08:58:22.984963 23580 ts_tablet_manager.cc:1428] T 750dd1e1055f4570b0aecdda3ebe8772 P ed1efbce9ee3401e99a5260edeae4e7c: Time spent starting tablet: real 0.012s user 0.006s sys 0.000s
I20250109 08:58:22.988487 23580 tablet_bootstrap.cc:492] T a09d8dffc2e446f1acd3c7f6cc1705bb P ed1efbce9ee3401e99a5260edeae4e7c: Bootstrap starting.
I20250109 08:58:22.989672 23452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a686bf4d947c4b638049e6b88c59ae09" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc" is_pre_election: true
I20250109 08:58:22.990329 23452 raft_consensus.cc:2463] T a686bf4d947c4b638049e6b88c59ae09 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed1efbce9ee3401e99a5260edeae4e7c in term 0.
I20250109 08:58:22.991552 23572 tablet_bootstrap.cc:654] T 5f0d500a91b54851bda52e5ccf741fd6 P cf55383bef6649eb9e78cb2d2512dedc: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:22.993214 23346 leader_election.cc:304] T a686bf4d947c4b638049e6b88c59ae09 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf55383bef6649eb9e78cb2d2512dedc, ed1efbce9ee3401e99a5260edeae4e7c; no voters:
I20250109 08:58:22.994024 23730 raft_consensus.cc:2798] T a686bf4d947c4b638049e6b88c59ae09 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:58:22.994390 23730 raft_consensus.cc:491] T a686bf4d947c4b638049e6b88c59ae09 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:58:22.994707 23730 raft_consensus.cc:3054] T a686bf4d947c4b638049e6b88c59ae09 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:23.009006 23569 tablet_bootstrap.cc:492] T a686bf4d947c4b638049e6b88c59ae09 P 05a74e1eb2474b3d8b3a2da1ef127e01: No bootstrap required, opened a new log
I20250109 08:58:23.009578 23569 ts_tablet_manager.cc:1397] T a686bf4d947c4b638049e6b88c59ae09 P 05a74e1eb2474b3d8b3a2da1ef127e01: Time spent bootstrapping tablet: real 0.363s user 0.007s sys 0.009s
I20250109 08:58:23.012347 23635 leader_election.cc:290] T 08214ee9c4c14bda92bdc6fa8231c313 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 election: Requested vote from peers 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829), 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691)
I20250109 08:58:23.015125 23302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08214ee9c4c14bda92bdc6fa8231c313" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada"
I20250109 08:58:23.015553 23709 raft_consensus.cc:491] T d87d3b4755344b82b0aec7ffd1ce785c P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:23.015774 23302 raft_consensus.cc:3054] T 08214ee9c4c14bda92bdc6fa8231c313 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Advancing to term 1
W20250109 08:58:23.016021 23344 leader_election.cc:343] T a686bf4d947c4b638049e6b88c59ae09 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:23.002782 23658 raft_consensus.cc:2798] T 08214ee9c4c14bda92bdc6fa8231c313 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:58:23.016103 23709 raft_consensus.cc:513] T d87d3b4755344b82b0aec7ffd1ce785c P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:23.017576 23658 raft_consensus.cc:695] T 08214ee9c4c14bda92bdc6fa8231c313 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 LEADER]: Becoming Leader. State: Replica: ed1efbce9ee3401e99a5260edeae4e7c, State: Running, Role: LEADER
I20250109 08:58:23.018433 23658 consensus_queue.cc:237] T 08214ee9c4c14bda92bdc6fa8231c313 P ed1efbce9ee3401e99a5260edeae4e7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:23.019115 23709 leader_election.cc:290] T d87d3b4755344b82b0aec7ffd1ce785c P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691), ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365)
I20250109 08:58:23.021272 23303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d87d3b4755344b82b0aec7ffd1ce785c" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada" is_pre_election: true
I20250109 08:58:23.021950 23303 raft_consensus.cc:2463] T d87d3b4755344b82b0aec7ffd1ce785c P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05a74e1eb2474b3d8b3a2da1ef127e01 in term 0.
I20250109 08:58:23.022502 23302 raft_consensus.cc:2463] T 08214ee9c4c14bda92bdc6fa8231c313 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed1efbce9ee3401e99a5260edeae4e7c in term 1.
I20250109 08:58:23.025262 23494 leader_election.cc:304] T d87d3b4755344b82b0aec7ffd1ce785c P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01, 235ad6af04c34c88a5261daa3604fada; no voters:
I20250109 08:58:23.026793 23766 raft_consensus.cc:2798] T d87d3b4755344b82b0aec7ffd1ce785c P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:58:23.026862 23677 raft_consensus.cc:491] T a09d8dffc2e446f1acd3c7f6cc1705bb P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:23.027209 23766 raft_consensus.cc:491] T d87d3b4755344b82b0aec7ffd1ce785c P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:58:23.027601 23766 raft_consensus.cc:3054] T d87d3b4755344b82b0aec7ffd1ce785c P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:23.027355 23677 raft_consensus.cc:513] T a09d8dffc2e446f1acd3c7f6cc1705bb P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:23.024104 23730 raft_consensus.cc:513] T a686bf4d947c4b638049e6b88c59ae09 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:23.028772 23627 consensus_queue.cc:1035] T 5c37c58abaa54656b31c72160b4dec9e P cf55383bef6649eb9e78cb2d2512dedc [LEADER]: Connected to new peer: Peer: permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:58:23.029522 23730 leader_election.cc:290] T a686bf4d947c4b638049e6b88c59ae09 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 election: Requested vote from peers cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551), 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829)
I20250109 08:58:23.030061 23677 leader_election.cc:290] T a09d8dffc2e446f1acd3c7f6cc1705bb P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691), ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365)
I20250109 08:58:23.030697 23452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a686bf4d947c4b638049e6b88c59ae09" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc"
I20250109 08:58:23.031267 23452 raft_consensus.cc:3054] T a686bf4d947c4b638049e6b88c59ae09 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:23.012142 23569 raft_consensus.cc:357] T a686bf4d947c4b638049e6b88c59ae09 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:23.034195 23569 raft_consensus.cc:383] T a686bf4d947c4b638049e6b88c59ae09 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:23.034513 23569 raft_consensus.cc:738] T a686bf4d947c4b638049e6b88c59ae09 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05a74e1eb2474b3d8b3a2da1ef127e01, State: Initialized, Role: FOLLOWER
I20250109 08:58:23.034286 23580 tablet_bootstrap.cc:654] T a09d8dffc2e446f1acd3c7f6cc1705bb P ed1efbce9ee3401e99a5260edeae4e7c: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:23.038061 23569 consensus_queue.cc:260] T a686bf4d947c4b638049e6b88c59ae09 P 05a74e1eb2474b3d8b3a2da1ef127e01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:23.039835 23569 ts_tablet_manager.cc:1428] T a686bf4d947c4b638049e6b88c59ae09 P 05a74e1eb2474b3d8b3a2da1ef127e01: Time spent starting tablet: real 0.030s user 0.006s sys 0.000s
I20250109 08:58:23.046063 23569 tablet_bootstrap.cc:492] T e5251927614642639bd46ecd5373d2a4 P 05a74e1eb2474b3d8b3a2da1ef127e01: Bootstrap starting.
I20250109 08:58:23.049500 23574 tablet_bootstrap.cc:492] T cc9886333c0343798226d59758a1ea5e P 235ad6af04c34c88a5261daa3604fada: No bootstrap required, opened a new log
I20250109 08:58:23.050180 23574 ts_tablet_manager.cc:1397] T cc9886333c0343798226d59758a1ea5e P 235ad6af04c34c88a5261daa3604fada: Time spent bootstrapping tablet: real 0.209s user 0.009s sys 0.011s
I20250109 08:58:23.058506 23378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a09d8dffc2e446f1acd3c7f6cc1705bb" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" is_pre_election: true
I20250109 08:58:23.058692 23574 raft_consensus.cc:357] T cc9886333c0343798226d59758a1ea5e P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:23.059535 23574 raft_consensus.cc:383] T cc9886333c0343798226d59758a1ea5e P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:23.059921 23574 raft_consensus.cc:738] T cc9886333c0343798226d59758a1ea5e P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Initialized, Role: FOLLOWER
I20250109 08:58:23.060716 23574 consensus_queue.cc:260] T cc9886333c0343798226d59758a1ea5e P 235ad6af04c34c88a5261daa3604fada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:23.061283 23302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a09d8dffc2e446f1acd3c7f6cc1705bb" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada" is_pre_election: true
I20250109 08:58:23.066124 23302 raft_consensus.cc:2463] T a09d8dffc2e446f1acd3c7f6cc1705bb P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf55383bef6649eb9e78cb2d2512dedc in term 0.
I20250109 08:58:23.067548 23377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d87d3b4755344b82b0aec7ffd1ce785c" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" is_pre_election: true
I20250109 08:58:23.061724 23200 catalog_manager.cc:5526] T e6b9aaeda9a947d4a87fedd774301cc0 P cf55383bef6649eb9e78cb2d2512dedc reported cstate change: term changed from 0 to 1, leader changed from <none> to cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195). New cstate: current_term: 1 leader_uuid: "cf55383bef6649eb9e78cb2d2512dedc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } health_report { overall_health: HEALTHY } } }
I20250109 08:58:23.070413 23451 raft_consensus.cc:3054] T 9b245cb3666c421c884cf20161a22831 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:23.072002 23574 ts_tablet_manager.cc:1428] T cc9886333c0343798226d59758a1ea5e P 235ad6af04c34c88a5261daa3604fada: Time spent starting tablet: real 0.021s user 0.005s sys 0.001s
I20250109 08:58:23.073020 23574 tablet_bootstrap.cc:492] T 86d4074509744900abe94402467a7d85 P 235ad6af04c34c88a5261daa3604fada: Bootstrap starting.
W20250109 08:58:23.068079 23420 leader_election.cc:343] T a09d8dffc2e446f1acd3c7f6cc1705bb P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:23.099916 23627 raft_consensus.cc:491] T 24ce131f7bb143bea1581d5b0c2a7109 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:23.114305 23627 raft_consensus.cc:513] T 24ce131f7bb143bea1581d5b0c2a7109 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:23.111588 23522 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a686bf4d947c4b638049e6b88c59ae09" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01"
I20250109 08:58:23.133713 23203 catalog_manager.cc:5526] T 08214ee9c4c14bda92bdc6fa8231c313 P ed1efbce9ee3401e99a5260edeae4e7c reported cstate change: term changed from 0 to 1, leader changed from <none> to ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194). New cstate: current_term: 1 leader_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } health_report { overall_health: HEALTHY } } }
W20250109 08:58:23.138325 23494 leader_election.cc:343] T d87d3b4755344b82b0aec7ffd1ce785c P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:23.116240 23627 leader_election.cc:290] T 24ce131f7bb143bea1581d5b0c2a7109 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829), 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691)
I20250109 08:58:23.132542 23522 raft_consensus.cc:3054] T a686bf4d947c4b638049e6b88c59ae09 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:23.154211 23580 tablet_bootstrap.cc:492] T a09d8dffc2e446f1acd3c7f6cc1705bb P ed1efbce9ee3401e99a5260edeae4e7c: No bootstrap required, opened a new log
I20250109 08:58:23.154745 23580 ts_tablet_manager.cc:1397] T a09d8dffc2e446f1acd3c7f6cc1705bb P ed1efbce9ee3401e99a5260edeae4e7c: Time spent bootstrapping tablet: real 0.166s user 0.014s sys 0.012s
I20250109 08:58:23.156723 23301 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24ce131f7bb143bea1581d5b0c2a7109" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada" is_pre_election: true
I20250109 08:58:23.157536 23301 raft_consensus.cc:2388] T 24ce131f7bb143bea1581d5b0c2a7109 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf55383bef6649eb9e78cb2d2512dedc in current term 1: Already voted for candidate 05a74e1eb2474b3d8b3a2da1ef127e01 in this term.
W20250109 08:58:23.158814 23346 consensus_peers.cc:487] T 86d4074509744900abe94402467a7d85 P ed1efbce9ee3401e99a5260edeae4e7c -> Peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Couldn't send request to peer 235ad6af04c34c88a5261daa3604fada. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:23.148536 23766 raft_consensus.cc:513] T d87d3b4755344b82b0aec7ffd1ce785c P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
W20250109 08:58:23.167472 23797 log.cc:926] Time spent T 4df64aae69ca41dc91286cae1603000f P 05a74e1eb2474b3d8b3a2da1ef127e01: Append to log took a long time: real 0.062s user 0.000s sys 0.000s
I20250109 08:58:23.168838 23634 raft_consensus.cc:491] T 4df64aae69ca41dc91286cae1603000f P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 235ad6af04c34c88a5261daa3604fada)
I20250109 08:58:23.170818 23634 raft_consensus.cc:513] T 4df64aae69ca41dc91286cae1603000f P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:23.172350 23522 raft_consensus.cc:2463] T a686bf4d947c4b638049e6b88c59ae09 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed1efbce9ee3401e99a5260edeae4e7c in term 1.
I20250109 08:58:23.174461 23580 raft_consensus.cc:357] T a09d8dffc2e446f1acd3c7f6cc1705bb P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:23.175312 23580 raft_consensus.cc:383] T a09d8dffc2e446f1acd3c7f6cc1705bb P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:23.175577 23580 raft_consensus.cc:738] T a09d8dffc2e446f1acd3c7f6cc1705bb P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed1efbce9ee3401e99a5260edeae4e7c, State: Initialized, Role: FOLLOWER
I20250109 08:58:23.177326 23580 consensus_queue.cc:260] T a09d8dffc2e446f1acd3c7f6cc1705bb P ed1efbce9ee3401e99a5260edeae4e7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:23.184805 23634 leader_election.cc:290] T 4df64aae69ca41dc91286cae1603000f P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691), ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365)
W20250109 08:58:23.194213 23789 log.cc:926] Time spent T b572a6a3dfc04ea199bc703ddad8af67 P 235ad6af04c34c88a5261daa3604fada: Append to log took a long time: real 0.075s user 0.000s sys 0.000s
I20250109 08:58:23.185038 23572 tablet_bootstrap.cc:492] T 5f0d500a91b54851bda52e5ccf741fd6 P cf55383bef6649eb9e78cb2d2512dedc: No bootstrap required, opened a new log
I20250109 08:58:23.185420 23528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24ce131f7bb143bea1581d5b0c2a7109" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" is_pre_election: true
I20250109 08:58:23.183135 23452 raft_consensus.cc:2463] T a686bf4d947c4b638049e6b88c59ae09 P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed1efbce9ee3401e99a5260edeae4e7c in term 1.
I20250109 08:58:23.204602 23572 ts_tablet_manager.cc:1397] T 5f0d500a91b54851bda52e5ccf741fd6 P cf55383bef6649eb9e78cb2d2512dedc: Time spent bootstrapping tablet: real 0.219s user 0.010s sys 0.006s
I20250109 08:58:23.207142 23420 leader_election.cc:304] T a09d8dffc2e446f1acd3c7f6cc1705bb P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 235ad6af04c34c88a5261daa3604fada, cf55383bef6649eb9e78cb2d2512dedc; no voters: ed1efbce9ee3401e99a5260edeae4e7c
I20250109 08:58:23.207983 23583 raft_consensus.cc:2798] T a09d8dffc2e446f1acd3c7f6cc1705bb P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:58:23.208423 23583 raft_consensus.cc:491] T a09d8dffc2e446f1acd3c7f6cc1705bb P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:58:23.208737 23583 raft_consensus.cc:3054] T a09d8dffc2e446f1acd3c7f6cc1705bb P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:23.207872 23572 raft_consensus.cc:357] T 5f0d500a91b54851bda52e5ccf741fd6 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:23.212473 23572 raft_consensus.cc:383] T 5f0d500a91b54851bda52e5ccf741fd6 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:23.212728 23572 raft_consensus.cc:738] T 5f0d500a91b54851bda52e5ccf741fd6 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf55383bef6649eb9e78cb2d2512dedc, State: Initialized, Role: FOLLOWER
I20250109 08:58:23.187302 23574 tablet_bootstrap.cc:654] T 86d4074509744900abe94402467a7d85 P 235ad6af04c34c88a5261daa3604fada: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:23.188858 23301 raft_consensus.cc:3054] T cc9886333c0343798226d59758a1ea5e P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:23.193403 23569 tablet_bootstrap.cc:654] T e5251927614642639bd46ecd5373d2a4 P 05a74e1eb2474b3d8b3a2da1ef127e01: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:23.215075 23634 consensus_queue.cc:1035] T cc9886333c0343798226d59758a1ea5e P 05a74e1eb2474b3d8b3a2da1ef127e01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20250109 08:58:23.215829 23572 consensus_queue.cc:260] T 5f0d500a91b54851bda52e5ccf741fd6 P cf55383bef6649eb9e78cb2d2512dedc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:23.219837 23580 ts_tablet_manager.cc:1428] T a09d8dffc2e446f1acd3c7f6cc1705bb P ed1efbce9ee3401e99a5260edeae4e7c: Time spent starting tablet: real 0.065s user 0.005s sys 0.001s
I20250109 08:58:23.220868 23580 tablet_bootstrap.cc:492] T fc1038255dd64cb5ba2425037d22f1f7 P ed1efbce9ee3401e99a5260edeae4e7c: Bootstrap starting.
I20250109 08:58:23.222182 23635 consensus_queue.cc:1035] T 940653b6d55c46439827697b9aadf16b P ed1efbce9ee3401e99a5260edeae4e7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:58:23.225441 23346 leader_election.cc:304] T a686bf4d947c4b638049e6b88c59ae09 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf55383bef6649eb9e78cb2d2512dedc, ed1efbce9ee3401e99a5260edeae4e7c; no voters:
I20250109 08:58:23.227303 23611 consensus_queue.cc:1035] T 940653b6d55c46439827697b9aadf16b P ed1efbce9ee3401e99a5260edeae4e7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:58:23.227864 23580 tablet_bootstrap.cc:654] T fc1038255dd64cb5ba2425037d22f1f7 P ed1efbce9ee3401e99a5260edeae4e7c: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:23.228503 23611 raft_consensus.cc:2798] T a686bf4d947c4b638049e6b88c59ae09 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:58:23.217780 23658 raft_consensus.cc:491] T 750dd1e1055f4570b0aecdda3ebe8772 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:23.219246 23627 raft_consensus.cc:491] T 8de6b2fb09bc46ce9e4033af06dffd04 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:23.229629 23418 leader_election.cc:304] T 24ce131f7bb143bea1581d5b0c2a7109 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf55383bef6649eb9e78cb2d2512dedc; no voters: 05a74e1eb2474b3d8b3a2da1ef127e01, 235ad6af04c34c88a5261daa3604fada
I20250109 08:58:23.228957 23611 raft_consensus.cc:695] T a686bf4d947c4b638049e6b88c59ae09 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 LEADER]: Becoming Leader. State: Replica: ed1efbce9ee3401e99a5260edeae4e7c, State: Running, Role: LEADER
I20250109 08:58:23.236315 23784 raft_consensus.cc:3054] T 24ce131f7bb143bea1581d5b0c2a7109 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:23.233377 23611 consensus_queue.cc:237] T a686bf4d947c4b638049e6b88c59ae09 P ed1efbce9ee3401e99a5260edeae4e7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:23.230141 23658 raft_consensus.cc:513] T 750dd1e1055f4570b0aecdda3ebe8772 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:23.218590 23766 leader_election.cc:290] T d87d3b4755344b82b0aec7ffd1ce785c P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 election: Requested vote from peers 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691), ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365)
I20250109 08:58:23.230585 23627 raft_consensus.cc:513] T 8de6b2fb09bc46ce9e4033af06dffd04 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:23.244632 23627 leader_election.cc:290] T 8de6b2fb09bc46ce9e4033af06dffd04 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365), 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829)
I20250109 08:58:23.252849 23303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4df64aae69ca41dc91286cae1603000f" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada" is_pre_election: true
I20250109 08:58:23.255218 23377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8de6b2fb09bc46ce9e4033af06dffd04" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" is_pre_election: true
I20250109 08:58:23.230216 23730 consensus_queue.cc:1035] T 86d4074509744900abe94402467a7d85 P ed1efbce9ee3401e99a5260edeae4e7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.026s
I20250109 08:58:23.255869 23377 raft_consensus.cc:2463] T 8de6b2fb09bc46ce9e4033af06dffd04 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf55383bef6649eb9e78cb2d2512dedc in term 0.
I20250109 08:58:23.255244 23572 ts_tablet_manager.cc:1428] T 5f0d500a91b54851bda52e5ccf741fd6 P cf55383bef6649eb9e78cb2d2512dedc: Time spent starting tablet: real 0.050s user 0.006s sys 0.000s
I20250109 08:58:23.258699 23730 raft_consensus.cc:491] T a09d8dffc2e446f1acd3c7f6cc1705bb P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:23.259166 23730 raft_consensus.cc:513] T a09d8dffc2e446f1acd3c7f6cc1705bb P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:23.259740 23580 tablet_bootstrap.cc:492] T fc1038255dd64cb5ba2425037d22f1f7 P ed1efbce9ee3401e99a5260edeae4e7c: No bootstrap required, opened a new log
I20250109 08:58:23.260402 23580 ts_tablet_manager.cc:1397] T fc1038255dd64cb5ba2425037d22f1f7 P ed1efbce9ee3401e99a5260edeae4e7c: Time spent bootstrapping tablet: real 0.040s user 0.025s sys 0.010s
I20250109 08:58:23.240803 23658 leader_election.cc:290] T 750dd1e1055f4570b0aecdda3ebe8772 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551), 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829)
I20250109 08:58:23.262176 23572 tablet_bootstrap.cc:492] T e5251927614642639bd46ecd5373d2a4 P cf55383bef6649eb9e78cb2d2512dedc: Bootstrap starting.
I20250109 08:58:23.265064 23586 raft_consensus.cc:491] T cc9886333c0343798226d59758a1ea5e P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:23.265551 23586 raft_consensus.cc:513] T cc9886333c0343798226d59758a1ea5e P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:23.266705 23730 leader_election.cc:290] T a09d8dffc2e446f1acd3c7f6cc1705bb P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691), cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551)
I20250109 08:58:23.265673 23420 leader_election.cc:304] T 8de6b2fb09bc46ce9e4033af06dffd04 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf55383bef6649eb9e78cb2d2512dedc, ed1efbce9ee3401e99a5260edeae4e7c; no voters:
I20250109 08:58:23.268970 23675 raft_consensus.cc:2798] T 8de6b2fb09bc46ce9e4033af06dffd04 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:58:23.269205 23586 leader_election.cc:290] T cc9886333c0343798226d59758a1ea5e P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551), 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829)
I20250109 08:58:23.269376 23675 raft_consensus.cc:491] T 8de6b2fb09bc46ce9e4033af06dffd04 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:58:23.269762 23675 raft_consensus.cc:3054] T 8de6b2fb09bc46ce9e4033af06dffd04 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:23.255486 23784 raft_consensus.cc:2743] T 24ce131f7bb143bea1581d5b0c2a7109 P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:58:23.270044 23627 raft_consensus.cc:491] T db111e41c1be497e8e673a2c42b9ba66 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:23.272614 23627 raft_consensus.cc:513] T db111e41c1be497e8e673a2c42b9ba66 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:23.269762 23452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc9886333c0343798226d59758a1ea5e" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc" is_pre_election: true
I20250109 08:58:23.273741 23452 raft_consensus.cc:2463] T cc9886333c0343798226d59758a1ea5e P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 235ad6af04c34c88a5261daa3604fada in term 1.
I20250109 08:58:23.274585 23627 leader_election.cc:290] T db111e41c1be497e8e673a2c42b9ba66 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691), ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365)
I20250109 08:58:23.275391 23301 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db111e41c1be497e8e673a2c42b9ba66" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada" is_pre_election: true
I20250109 08:58:23.276209 23271 leader_election.cc:304] T cc9886333c0343798226d59758a1ea5e P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 235ad6af04c34c88a5261daa3604fada, cf55383bef6649eb9e78cb2d2512dedc; no voters:
W20250109 08:58:23.278007 23420 leader_election.cc:343] T db111e41c1be497e8e673a2c42b9ba66 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:23.288654 23377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db111e41c1be497e8e673a2c42b9ba66" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" is_pre_election: true
I20250109 08:58:23.282632 23586 raft_consensus.cc:2798] T cc9886333c0343798226d59758a1ea5e P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Leader pre-election won for term 2
W20250109 08:58:23.283603 23344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.226.196:41829, user_credentials={real_user=slave}} blocked reactor thread for 102786us
I20250109 08:58:23.299513 23586 raft_consensus.cc:491] T cc9886333c0343798226d59758a1ea5e P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:58:23.307240 23586 raft_consensus.cc:3054] T cc9886333c0343798226d59758a1ea5e P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:23.309741 23454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "750dd1e1055f4570b0aecdda3ebe8772" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc" is_pre_election: true
I20250109 08:58:23.315304 23580 raft_consensus.cc:357] T fc1038255dd64cb5ba2425037d22f1f7 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:23.316049 23580 raft_consensus.cc:383] T fc1038255dd64cb5ba2425037d22f1f7 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:23.316308 23580 raft_consensus.cc:738] T fc1038255dd64cb5ba2425037d22f1f7 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed1efbce9ee3401e99a5260edeae4e7c, State: Initialized, Role: FOLLOWER
I20250109 08:58:23.278995 23583 raft_consensus.cc:513] T a09d8dffc2e446f1acd3c7f6cc1705bb P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:23.319262 23583 leader_election.cc:290] T a09d8dffc2e446f1acd3c7f6cc1705bb P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 election: Requested vote from peers 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691), ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365)
I20250109 08:58:23.300601 23528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc9886333c0343798226d59758a1ea5e" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" is_pre_election: true
I20250109 08:58:23.304704 23525 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8de6b2fb09bc46ce9e4033af06dffd04" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" is_pre_election: true
I20250109 08:58:23.303371 23522 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "750dd1e1055f4570b0aecdda3ebe8772" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" is_pre_election: true
I20250109 08:58:23.322499 23634 consensus_queue.cc:1035] T cc9886333c0343798226d59758a1ea5e P 05a74e1eb2474b3d8b3a2da1ef127e01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20250109 08:58:23.314914 23420 leader_election.cc:343] T db111e41c1be497e8e673a2c42b9ba66 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365): Illegal state: must be running to vote when last-logged opid is not known
W20250109 08:58:23.342099 23346 leader_election.cc:343] T 750dd1e1055f4570b0aecdda3ebe8772 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:23.327869 23301 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d87d3b4755344b82b0aec7ffd1ce785c" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada"
I20250109 08:58:23.342449 23420 leader_election.cc:304] T db111e41c1be497e8e673a2c42b9ba66 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf55383bef6649eb9e78cb2d2512dedc; no voters: 235ad6af04c34c88a5261daa3604fada, ed1efbce9ee3401e99a5260edeae4e7c
I20250109 08:58:23.327901 23377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4df64aae69ca41dc91286cae1603000f" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" is_pre_election: true
I20250109 08:58:23.342844 23301 raft_consensus.cc:3054] T d87d3b4755344b82b0aec7ffd1ce785c P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:23.345086 23304 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a09d8dffc2e446f1acd3c7f6cc1705bb" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada"
I20250109 08:58:23.345338 23302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a09d8dffc2e446f1acd3c7f6cc1705bb" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada" is_pre_election: true
I20250109 08:58:23.345954 23304 raft_consensus.cc:3054] T a09d8dffc2e446f1acd3c7f6cc1705bb P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:23.345909 23379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a09d8dffc2e446f1acd3c7f6cc1705bb" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c"
I20250109 08:58:23.346649 23379 raft_consensus.cc:3054] T a09d8dffc2e446f1acd3c7f6cc1705bb P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:23.347795 23583 raft_consensus.cc:2743] T db111e41c1be497e8e673a2c42b9ba66 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:58:23.330682 23675 raft_consensus.cc:513] T 8de6b2fb09bc46ce9e4033af06dffd04 P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:23.317121 23580 consensus_queue.cc:260] T fc1038255dd64cb5ba2425037d22f1f7 P ed1efbce9ee3401e99a5260edeae4e7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:23.351281 23377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8de6b2fb09bc46ce9e4033af06dffd04" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c"
I20250109 08:58:23.345078 23200 catalog_manager.cc:5526] T a686bf4d947c4b638049e6b88c59ae09 P ed1efbce9ee3401e99a5260edeae4e7c reported cstate change: term changed from 0 to 1, leader changed from <none> to ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194). New cstate: current_term: 1 leader_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:23.339756 23303 raft_consensus.cc:1270] T 970a82e754db47b6b1a19551d4ba43d8 P 235ad6af04c34c88a5261daa3604fada [term 2 FOLLOWER]: Refusing update from remote peer 05a74e1eb2474b3d8b3a2da1ef127e01: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20250109 08:58:23.350903 23634 consensus_queue.cc:1035] T 24ce131f7bb143bea1581d5b0c2a7109 P 05a74e1eb2474b3d8b3a2da1ef127e01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20250109 08:58:23.354107 23784 consensus_queue.cc:1035] T e6b9aaeda9a947d4a87fedd774301cc0 P cf55383bef6649eb9e78cb2d2512dedc [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:58:23.354817 23580 ts_tablet_manager.cc:1428] T fc1038255dd64cb5ba2425037d22f1f7 P ed1efbce9ee3401e99a5260edeae4e7c: Time spent starting tablet: real 0.094s user 0.006s sys 0.000s
I20250109 08:58:23.351845 23377 raft_consensus.cc:3054] T 8de6b2fb09bc46ce9e4033af06dffd04 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:23.355736 23301 raft_consensus.cc:2463] T d87d3b4755344b82b0aec7ffd1ce785c P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05a74e1eb2474b3d8b3a2da1ef127e01 in term 1.
I20250109 08:58:23.355757 23580 tablet_bootstrap.cc:492] T f1f9274aeda748f493f5752814fb262d P ed1efbce9ee3401e99a5260edeae4e7c: Bootstrap starting.
I20250109 08:58:23.336309 23586 raft_consensus.cc:513] T cc9886333c0343798226d59758a1ea5e P 235ad6af04c34c88a5261daa3604fada [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:23.361696 23572 tablet_bootstrap.cc:654] T e5251927614642639bd46ecd5373d2a4 P cf55383bef6649eb9e78cb2d2512dedc: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:23.371760 23494 leader_election.cc:304] T d87d3b4755344b82b0aec7ffd1ce785c P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01, 235ad6af04c34c88a5261daa3604fada; no voters:
I20250109 08:58:23.373948 23766 consensus_queue.cc:1035] T 970a82e754db47b6b1a19551d4ba43d8 P 05a74e1eb2474b3d8b3a2da1ef127e01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:58:23.375329 23766 raft_consensus.cc:2798] T d87d3b4755344b82b0aec7ffd1ce785c P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:58:23.375821 23766 raft_consensus.cc:695] T d87d3b4755344b82b0aec7ffd1ce785c P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 LEADER]: Becoming Leader. State: Replica: 05a74e1eb2474b3d8b3a2da1ef127e01, State: Running, Role: LEADER
W20250109 08:58:23.368969 23418 leader_election.cc:343] T 8de6b2fb09bc46ce9e4033af06dffd04 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:23.381307 23675 leader_election.cc:290] T 8de6b2fb09bc46ce9e4033af06dffd04 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 election: Requested vote from peers ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365), 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829)
I20250109 08:58:23.381842 23574 tablet_bootstrap.cc:492] T 86d4074509744900abe94402467a7d85 P 235ad6af04c34c88a5261daa3604fada: No bootstrap required, opened a new log
I20250109 08:58:23.383139 23305 raft_consensus.cc:1235] T cc9886333c0343798226d59758a1ea5e P 235ad6af04c34c88a5261daa3604fada [term 2 FOLLOWER]: Rejecting Update request from peer 05a74e1eb2474b3d8b3a2da1ef127e01 for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20250109 08:58:23.383545 23574 ts_tablet_manager.cc:1397] T 86d4074509744900abe94402467a7d85 P 235ad6af04c34c88a5261daa3604fada: Time spent bootstrapping tablet: real 0.311s user 0.013s sys 0.005s
I20250109 08:58:23.384614 23766 consensus_queue.cc:237] T d87d3b4755344b82b0aec7ffd1ce785c P 05a74e1eb2474b3d8b3a2da1ef127e01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:23.386585 23528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8de6b2fb09bc46ce9e4033af06dffd04" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01"
I20250109 08:58:23.387805 23586 leader_election.cc:290] T cc9886333c0343798226d59758a1ea5e P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 2 election: Requested vote from peers cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551), 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829)
W20250109 08:58:23.377422 23251 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20250109 08:58:23.369517 23379 raft_consensus.cc:2463] T a09d8dffc2e446f1acd3c7f6cc1705bb P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf55383bef6649eb9e78cb2d2512dedc in term 1.
I20250109 08:58:23.406473 23810 consensus_queue.cc:1046] T cc9886333c0343798226d59758a1ea5e P 05a74e1eb2474b3d8b3a2da1ef127e01 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:58:23.406982 23452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a09d8dffc2e446f1acd3c7f6cc1705bb" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc" is_pre_election: true
I20250109 08:58:23.410646 23810 raft_consensus.cc:3049] T cc9886333c0343798226d59758a1ea5e P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 LEADER]: Stepping down as leader of term 1
I20250109 08:58:23.411046 23810 raft_consensus.cc:738] T cc9886333c0343798226d59758a1ea5e P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 05a74e1eb2474b3d8b3a2da1ef127e01, State: Running, Role: LEADER
I20250109 08:58:23.411312 23452 raft_consensus.cc:2388] T a09d8dffc2e446f1acd3c7f6cc1705bb P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ed1efbce9ee3401e99a5260edeae4e7c in current term 1: Already voted for candidate cf55383bef6649eb9e78cb2d2512dedc in this term.
I20250109 08:58:23.407876 23813 consensus_queue.cc:1035] T 24ce131f7bb143bea1581d5b0c2a7109 P 05a74e1eb2474b3d8b3a2da1ef127e01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250109 08:58:23.412353 23420 leader_election.cc:304] T a09d8dffc2e446f1acd3c7f6cc1705bb P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf55383bef6649eb9e78cb2d2512dedc, ed1efbce9ee3401e99a5260edeae4e7c; no voters:
I20250109 08:58:23.413156 23675 raft_consensus.cc:2798] T a09d8dffc2e446f1acd3c7f6cc1705bb P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:58:23.411828 23810 consensus_queue.cc:260] T cc9886333c0343798226d59758a1ea5e P 05a74e1eb2474b3d8b3a2da1ef127e01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:23.413592 23675 raft_consensus.cc:695] T a09d8dffc2e446f1acd3c7f6cc1705bb P cf55383bef6649eb9e78cb2d2512dedc [term 1 LEADER]: Becoming Leader. State: Replica: cf55383bef6649eb9e78cb2d2512dedc, State: Running, Role: LEADER
I20250109 08:58:23.414438 23810 raft_consensus.cc:3054] T cc9886333c0343798226d59758a1ea5e P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:23.388538 23450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc9886333c0343798226d59758a1ea5e" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc"
W20250109 08:58:23.405121 23418 leader_election.cc:343] T 8de6b2fb09bc46ce9e4033af06dffd04 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:23.419682 23377 raft_consensus.cc:2463] T 8de6b2fb09bc46ce9e4033af06dffd04 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf55383bef6649eb9e78cb2d2512dedc in term 1.
I20250109 08:58:23.388324 23528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc9886333c0343798226d59758a1ea5e" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01"
I20250109 08:58:23.379518 23304 raft_consensus.cc:2463] T a09d8dffc2e446f1acd3c7f6cc1705bb P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf55383bef6649eb9e78cb2d2512dedc in term 1.
I20250109 08:58:23.414578 23675 consensus_queue.cc:237] T a09d8dffc2e446f1acd3c7f6cc1705bb P cf55383bef6649eb9e78cb2d2512dedc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:23.414777 23574 raft_consensus.cc:357] T 86d4074509744900abe94402467a7d85 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:23.424412 23574 raft_consensus.cc:383] T 86d4074509744900abe94402467a7d85 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:23.424682 23574 raft_consensus.cc:738] T 86d4074509744900abe94402467a7d85 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Initialized, Role: FOLLOWER
I20250109 08:58:23.425768 23574 consensus_queue.cc:260] T 86d4074509744900abe94402467a7d85 P 235ad6af04c34c88a5261daa3604fada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:23.427592 23201 catalog_manager.cc:5526] T d87d3b4755344b82b0aec7ffd1ce785c P 05a74e1eb2474b3d8b3a2da1ef127e01 reported cstate change: term changed from 0 to 1, leader changed from <none> to 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196). New cstate: current_term: 1 leader_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } health_report { overall_health: UNKNOWN } } }
W20250109 08:58:23.391645 23344 leader_election.cc:343] T 750dd1e1055f4570b0aecdda3ebe8772 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:23.438489 23574 ts_tablet_manager.cc:1428] T 86d4074509744900abe94402467a7d85 P 235ad6af04c34c88a5261daa3604fada: Time spent starting tablet: real 0.055s user 0.007s sys 0.000s
I20250109 08:58:23.439688 23574 tablet_bootstrap.cc:492] T 732e4aa69aae44e9b1153ac330a01a96 P 235ad6af04c34c88a5261daa3604fada: Bootstrap starting.
I20250109 08:58:23.433995 23784 raft_consensus.cc:491] T 5f0d500a91b54851bda52e5ccf741fd6 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:23.434125 23344 leader_election.cc:304] T 750dd1e1055f4570b0aecdda3ebe8772 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ed1efbce9ee3401e99a5260edeae4e7c; no voters: 05a74e1eb2474b3d8b3a2da1ef127e01, cf55383bef6649eb9e78cb2d2512dedc
W20250109 08:58:23.446738 23344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.226.196:41829, user_credentials={real_user=slave}} blocked reactor thread for 56557.8us
I20250109 08:58:23.446882 23580 tablet_bootstrap.cc:654] T f1f9274aeda748f493f5752814fb262d P ed1efbce9ee3401e99a5260edeae4e7c: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:23.457666 23569 tablet_bootstrap.cc:492] T e5251927614642639bd46ecd5373d2a4 P 05a74e1eb2474b3d8b3a2da1ef127e01: No bootstrap required, opened a new log
I20250109 08:58:23.460794 23569 ts_tablet_manager.cc:1397] T e5251927614642639bd46ecd5373d2a4 P 05a74e1eb2474b3d8b3a2da1ef127e01: Time spent bootstrapping tablet: real 0.415s user 0.014s sys 0.022s
I20250109 08:58:23.434684 23346 leader_election.cc:304] T a09d8dffc2e446f1acd3c7f6cc1705bb P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ed1efbce9ee3401e99a5260edeae4e7c; no voters: 235ad6af04c34c88a5261daa3604fada, cf55383bef6649eb9e78cb2d2512dedc
I20250109 08:58:23.465327 23816 raft_consensus.cc:2743] T 750dd1e1055f4570b0aecdda3ebe8772 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:58:23.468945 23379 raft_consensus.cc:1270] T 970a82e754db47b6b1a19551d4ba43d8 P ed1efbce9ee3401e99a5260edeae4e7c [term 2 FOLLOWER]: Refusing update from remote peer 05a74e1eb2474b3d8b3a2da1ef127e01: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20250109 08:58:23.471434 23816 raft_consensus.cc:2743] T a09d8dffc2e446f1acd3c7f6cc1705bb P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:58:23.469045 23572 tablet_bootstrap.cc:492] T e5251927614642639bd46ecd5373d2a4 P cf55383bef6649eb9e78cb2d2512dedc: No bootstrap required, opened a new log
I20250109 08:58:23.473274 23572 ts_tablet_manager.cc:1397] T e5251927614642639bd46ecd5373d2a4 P cf55383bef6649eb9e78cb2d2512dedc: Time spent bootstrapping tablet: real 0.215s user 0.015s sys 0.034s
I20250109 08:58:23.474267 23377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d87d3b4755344b82b0aec7ffd1ce785c" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c"
I20250109 08:58:23.448309 23494 leader_election.cc:304] T 4df64aae69ca41dc91286cae1603000f P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01; no voters: 235ad6af04c34c88a5261daa3604fada, ed1efbce9ee3401e99a5260edeae4e7c
I20250109 08:58:23.477142 23813 raft_consensus.cc:2743] T 4df64aae69ca41dc91286cae1603000f P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250109 08:58:23.437865 23420 leader_election.cc:304] T 8de6b2fb09bc46ce9e4033af06dffd04 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf55383bef6649eb9e78cb2d2512dedc, ed1efbce9ee3401e99a5260edeae4e7c; no voters: 05a74e1eb2474b3d8b3a2da1ef127e01
I20250109 08:58:23.485013 23677 raft_consensus.cc:2798] T 8de6b2fb09bc46ce9e4033af06dffd04 P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:58:23.485484 23677 raft_consensus.cc:695] T 8de6b2fb09bc46ce9e4033af06dffd04 P cf55383bef6649eb9e78cb2d2512dedc [term 1 LEADER]: Becoming Leader. State: Replica: cf55383bef6649eb9e78cb2d2512dedc, State: Running, Role: LEADER
I20250109 08:58:23.449090 23784 raft_consensus.cc:513] T 5f0d500a91b54851bda52e5ccf741fd6 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:23.486240 23677 consensus_queue.cc:237] T 8de6b2fb09bc46ce9e4033af06dffd04 P cf55383bef6649eb9e78cb2d2512dedc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
W20250109 08:58:23.489066 23420 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.226.194:41365, user_credentials={real_user=slave}} blocked reactor thread for 55176.1us
I20250109 08:58:23.493047 23813 consensus_queue.cc:1035] T 970a82e754db47b6b1a19551d4ba43d8 P 05a74e1eb2474b3d8b3a2da1ef127e01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:58:23.478045 23572 raft_consensus.cc:357] T e5251927614642639bd46ecd5373d2a4 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:23.494328 23572 raft_consensus.cc:383] T e5251927614642639bd46ecd5373d2a4 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:23.494626 23572 raft_consensus.cc:738] T e5251927614642639bd46ecd5373d2a4 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf55383bef6649eb9e78cb2d2512dedc, State: Initialized, Role: FOLLOWER
W20250109 08:58:23.495286 23494 leader_election.cc:343] T d87d3b4755344b82b0aec7ffd1ce785c P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:23.497464 23572 consensus_queue.cc:260] T e5251927614642639bd46ecd5373d2a4 P cf55383bef6649eb9e78cb2d2512dedc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:23.521102 23611 raft_consensus.cc:491] T fc1038255dd64cb5ba2425037d22f1f7 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:23.521672 23611 raft_consensus.cc:513] T fc1038255dd64cb5ba2425037d22f1f7 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:23.524557 23572 ts_tablet_manager.cc:1428] T e5251927614642639bd46ecd5373d2a4 P cf55383bef6649eb9e78cb2d2512dedc: Time spent starting tablet: real 0.051s user 0.004s sys 0.002s
I20250109 08:58:23.525517 23572 tablet_bootstrap.cc:492] T 732e4aa69aae44e9b1153ac330a01a96 P cf55383bef6649eb9e78cb2d2512dedc: Bootstrap starting.
I20250109 08:58:23.522589 23816 consensus_queue.cc:1035] T 08214ee9c4c14bda92bdc6fa8231c313 P ed1efbce9ee3401e99a5260edeae4e7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:58:23.527772 23611 leader_election.cc:290] T fc1038255dd64cb5ba2425037d22f1f7 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829), 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691)
I20250109 08:58:23.530035 23303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc1038255dd64cb5ba2425037d22f1f7" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada" is_pre_election: true
W20250109 08:58:23.534261 23346 leader_election.cc:343] T fc1038255dd64cb5ba2425037d22f1f7 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:23.535022 23784 leader_election.cc:290] T 5f0d500a91b54851bda52e5ccf741fd6 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829), 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691)
I20250109 08:58:23.536913 23627 consensus_queue.cc:1035] T e6b9aaeda9a947d4a87fedd774301cc0 P cf55383bef6649eb9e78cb2d2512dedc [LEADER]: Connected to new peer: Peer: permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:58:23.551470 23574 tablet_bootstrap.cc:654] T 732e4aa69aae44e9b1153ac330a01a96 P 235ad6af04c34c88a5261daa3604fada: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:23.549691 23522 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc1038255dd64cb5ba2425037d22f1f7" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" is_pre_election: true
W20250109 08:58:23.595444 23824 log.cc:926] Time spent T cc9886333c0343798226d59758a1ea5e P cf55383bef6649eb9e78cb2d2512dedc: Append to log took a long time: real 0.090s user 0.000s sys 0.001s
I20250109 08:58:23.560871 23304 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f0d500a91b54851bda52e5ccf741fd6" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada" is_pre_election: true
W20250109 08:58:23.603509 23344 leader_election.cc:343] T fc1038255dd64cb5ba2425037d22f1f7 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:23.604105 23527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f0d500a91b54851bda52e5ccf741fd6" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" is_pre_election: true
I20250109 08:58:23.604744 23528 raft_consensus.cc:2405] T cc9886333c0343798226d59758a1ea5e P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 235ad6af04c34c88a5261daa3604fada for term 2 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20250109 08:58:23.603999 23569 raft_consensus.cc:357] T e5251927614642639bd46ecd5373d2a4 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:23.607064 23569 raft_consensus.cc:383] T e5251927614642639bd46ecd5373d2a4 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:23.607339 23569 raft_consensus.cc:738] T e5251927614642639bd46ecd5373d2a4 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05a74e1eb2474b3d8b3a2da1ef127e01, State: Initialized, Role: FOLLOWER
I20250109 08:58:23.609338 23569 consensus_queue.cc:260] T e5251927614642639bd46ecd5373d2a4 P 05a74e1eb2474b3d8b3a2da1ef127e01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:23.601473 23583 raft_consensus.cc:491] T e5251927614642639bd46ecd5373d2a4 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:23.614274 23572 tablet_bootstrap.cc:654] T 732e4aa69aae44e9b1153ac330a01a96 P cf55383bef6649eb9e78cb2d2512dedc: Neither blocks nor log segments found. Creating new log.
W20250109 08:58:23.614609 23418 leader_election.cc:343] T 5f0d500a91b54851bda52e5ccf741fd6 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:23.604001 23344 leader_election.cc:304] T fc1038255dd64cb5ba2425037d22f1f7 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ed1efbce9ee3401e99a5260edeae4e7c; no voters: 05a74e1eb2474b3d8b3a2da1ef127e01, 235ad6af04c34c88a5261daa3604fada
I20250109 08:58:23.624828 23569 ts_tablet_manager.cc:1428] T e5251927614642639bd46ecd5373d2a4 P 05a74e1eb2474b3d8b3a2da1ef127e01: Time spent starting tablet: real 0.164s user 0.004s sys 0.003s
I20250109 08:58:23.624758 23817 raft_consensus.cc:2743] T fc1038255dd64cb5ba2425037d22f1f7 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20250109 08:58:23.625486 23420 leader_election.cc:343] T 5f0d500a91b54851bda52e5ccf741fd6 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:23.633579 23420 leader_election.cc:304] T 5f0d500a91b54851bda52e5ccf741fd6 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf55383bef6649eb9e78cb2d2512dedc; no voters: 05a74e1eb2474b3d8b3a2da1ef127e01, 235ad6af04c34c88a5261daa3604fada
I20250109 08:58:23.634676 23677 raft_consensus.cc:2743] T 5f0d500a91b54851bda52e5ccf741fd6 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:58:23.623170 23583 raft_consensus.cc:513] T e5251927614642639bd46ecd5373d2a4 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:23.630344 23269 leader_election.cc:304] T cc9886333c0343798226d59758a1ea5e P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 235ad6af04c34c88a5261daa3604fada; no voters: 05a74e1eb2474b3d8b3a2da1ef127e01, cf55383bef6649eb9e78cb2d2512dedc
I20250109 08:58:23.645715 23586 raft_consensus.cc:2743] T cc9886333c0343798226d59758a1ea5e P 235ad6af04c34c88a5261daa3604fada [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20250109 08:58:23.649652 23583 leader_election.cc:290] T e5251927614642639bd46ecd5373d2a4 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829), 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691)
I20250109 08:58:23.651101 23816 consensus_queue.cc:1035] T 08214ee9c4c14bda92bdc6fa8231c313 P ed1efbce9ee3401e99a5260edeae4e7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:58:23.652154 23569 tablet_bootstrap.cc:492] T 94f2be307f704c75bceb213e52307cfb P 05a74e1eb2474b3d8b3a2da1ef127e01: Bootstrap starting.
I20250109 08:58:23.652582 23525 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5251927614642639bd46ecd5373d2a4" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" is_pre_election: true
I20250109 08:58:23.653244 23525 raft_consensus.cc:2463] T e5251927614642639bd46ecd5373d2a4 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf55383bef6649eb9e78cb2d2512dedc in term 0.
I20250109 08:58:23.658301 23569 tablet_bootstrap.cc:654] T 94f2be307f704c75bceb213e52307cfb P 05a74e1eb2474b3d8b3a2da1ef127e01: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:23.664218 23418 leader_election.cc:304] T e5251927614642639bd46ecd5373d2a4 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01, cf55383bef6649eb9e78cb2d2512dedc; no voters:
I20250109 08:58:23.666714 23305 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5251927614642639bd46ecd5373d2a4" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada" is_pre_election: true
I20250109 08:58:23.667402 23305 raft_consensus.cc:2463] T e5251927614642639bd46ecd5373d2a4 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf55383bef6649eb9e78cb2d2512dedc in term 0.
I20250109 08:58:23.669741 23583 raft_consensus.cc:2798] T e5251927614642639bd46ecd5373d2a4 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:58:23.685649 23583 raft_consensus.cc:491] T e5251927614642639bd46ecd5373d2a4 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:58:23.686014 23583 raft_consensus.cc:3054] T e5251927614642639bd46ecd5373d2a4 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:23.712682 23569 tablet_bootstrap.cc:492] T 94f2be307f704c75bceb213e52307cfb P 05a74e1eb2474b3d8b3a2da1ef127e01: No bootstrap required, opened a new log
I20250109 08:58:23.713163 23569 ts_tablet_manager.cc:1397] T 94f2be307f704c75bceb213e52307cfb P 05a74e1eb2474b3d8b3a2da1ef127e01: Time spent bootstrapping tablet: real 0.072s user 0.017s sys 0.012s
I20250109 08:58:23.717065 23569 raft_consensus.cc:357] T 94f2be307f704c75bceb213e52307cfb P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:23.717767 23569 raft_consensus.cc:383] T 94f2be307f704c75bceb213e52307cfb P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:23.718030 23569 raft_consensus.cc:738] T 94f2be307f704c75bceb213e52307cfb P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05a74e1eb2474b3d8b3a2da1ef127e01, State: Initialized, Role: FOLLOWER
I20250109 08:58:23.718592 23569 consensus_queue.cc:260] T 94f2be307f704c75bceb213e52307cfb P 05a74e1eb2474b3d8b3a2da1ef127e01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:23.720850 23569 ts_tablet_manager.cc:1428] T 94f2be307f704c75bceb213e52307cfb P 05a74e1eb2474b3d8b3a2da1ef127e01: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20250109 08:58:23.721808 23569 tablet_bootstrap.cc:492] T 86b82e19759c4fd0b471a0dbff839aa5 P 05a74e1eb2474b3d8b3a2da1ef127e01: Bootstrap starting.
I20250109 08:58:23.725816 23583 raft_consensus.cc:513] T e5251927614642639bd46ecd5373d2a4 P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:23.727043 23569 tablet_bootstrap.cc:654] T 86b82e19759c4fd0b471a0dbff839aa5 P 05a74e1eb2474b3d8b3a2da1ef127e01: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:23.728081 23580 tablet_bootstrap.cc:492] T f1f9274aeda748f493f5752814fb262d P ed1efbce9ee3401e99a5260edeae4e7c: No bootstrap required, opened a new log
I20250109 08:58:23.730034 23580 ts_tablet_manager.cc:1397] T f1f9274aeda748f493f5752814fb262d P ed1efbce9ee3401e99a5260edeae4e7c: Time spent bootstrapping tablet: real 0.374s user 0.034s sys 0.021s
I20250109 08:58:23.728962 23574 tablet_bootstrap.cc:492] T 732e4aa69aae44e9b1153ac330a01a96 P 235ad6af04c34c88a5261daa3604fada: No bootstrap required, opened a new log
I20250109 08:58:23.730693 23574 ts_tablet_manager.cc:1397] T 732e4aa69aae44e9b1153ac330a01a96 P 235ad6af04c34c88a5261daa3604fada: Time spent bootstrapping tablet: real 0.291s user 0.013s sys 0.009s
I20250109 08:58:23.731550 23200 catalog_manager.cc:5526] T a09d8dffc2e446f1acd3c7f6cc1705bb P cf55383bef6649eb9e78cb2d2512dedc reported cstate change: term changed from 0 to 1, leader changed from <none> to cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195). New cstate: current_term: 1 leader_uuid: "cf55383bef6649eb9e78cb2d2512dedc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } health_report { overall_health: HEALTHY } } }
I20250109 08:58:23.732805 23200 catalog_manager.cc:5526] T 8de6b2fb09bc46ce9e4033af06dffd04 P cf55383bef6649eb9e78cb2d2512dedc reported cstate change: term changed from 0 to 1, leader changed from <none> to cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195). New cstate: current_term: 1 leader_uuid: "cf55383bef6649eb9e78cb2d2512dedc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:23.739467 23305 raft_consensus.cc:3054] T 86d4074509744900abe94402467a7d85 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:23.754685 23574 raft_consensus.cc:357] T 732e4aa69aae44e9b1153ac330a01a96 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:23.755486 23574 raft_consensus.cc:383] T 732e4aa69aae44e9b1153ac330a01a96 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:23.755764 23574 raft_consensus.cc:738] T 732e4aa69aae44e9b1153ac330a01a96 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Initialized, Role: FOLLOWER
I20250109 08:58:23.759505 23574 consensus_queue.cc:260] T 732e4aa69aae44e9b1153ac330a01a96 P 235ad6af04c34c88a5261daa3604fada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:23.764516 23583 leader_election.cc:290] T e5251927614642639bd46ecd5373d2a4 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 election: Requested vote from peers 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829), 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691)
I20250109 08:58:23.768680 23524 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5251927614642639bd46ecd5373d2a4" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01"
I20250109 08:58:23.769325 23524 raft_consensus.cc:3054] T e5251927614642639bd46ecd5373d2a4 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:23.772150 23574 ts_tablet_manager.cc:1428] T 732e4aa69aae44e9b1153ac330a01a96 P 235ad6af04c34c88a5261daa3604fada: Time spent starting tablet: real 0.041s user 0.006s sys 0.000s
I20250109 08:58:23.774303 23574 tablet_bootstrap.cc:492] T a7d2ab61d238471c9f287ac1d5db4ed0 P 235ad6af04c34c88a5261daa3604fada: Bootstrap starting.
I20250109 08:58:23.775200 23580 raft_consensus.cc:357] T f1f9274aeda748f493f5752814fb262d P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:23.776033 23580 raft_consensus.cc:383] T f1f9274aeda748f493f5752814fb262d P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:23.777146 23303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5251927614642639bd46ecd5373d2a4" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada"
I20250109 08:58:23.777769 23303 raft_consensus.cc:3054] T e5251927614642639bd46ecd5373d2a4 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:23.776290 23580 raft_consensus.cc:738] T f1f9274aeda748f493f5752814fb262d P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed1efbce9ee3401e99a5260edeae4e7c, State: Initialized, Role: FOLLOWER
I20250109 08:58:23.778002 23666 raft_consensus.cc:491] T 732e4aa69aae44e9b1153ac330a01a96 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:23.778568 23666 raft_consensus.cc:513] T 732e4aa69aae44e9b1153ac330a01a96 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:23.778738 23580 consensus_queue.cc:260] T f1f9274aeda748f493f5752814fb262d P ed1efbce9ee3401e99a5260edeae4e7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:23.780511 23666 leader_election.cc:290] T 732e4aa69aae44e9b1153ac330a01a96 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365), cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551)
I20250109 08:58:23.782480 23378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "732e4aa69aae44e9b1153ac330a01a96" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" is_pre_election: true
I20250109 08:58:23.783850 23524 raft_consensus.cc:2463] T e5251927614642639bd46ecd5373d2a4 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf55383bef6649eb9e78cb2d2512dedc in term 1.
W20250109 08:58:23.784164 23271 leader_election.cc:343] T 732e4aa69aae44e9b1153ac330a01a96 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:23.785115 23303 raft_consensus.cc:2463] T e5251927614642639bd46ecd5373d2a4 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf55383bef6649eb9e78cb2d2512dedc in term 1.
I20250109 08:58:23.786309 23420 leader_election.cc:304] T e5251927614642639bd46ecd5373d2a4 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 235ad6af04c34c88a5261daa3604fada, cf55383bef6649eb9e78cb2d2512dedc; no voters:
I20250109 08:58:23.787220 23675 raft_consensus.cc:2798] T e5251927614642639bd46ecd5373d2a4 P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:58:23.785336 23580 ts_tablet_manager.cc:1428] T f1f9274aeda748f493f5752814fb262d P ed1efbce9ee3401e99a5260edeae4e7c: Time spent starting tablet: real 0.055s user 0.007s sys 0.000s
I20250109 08:58:23.787688 23675 raft_consensus.cc:695] T e5251927614642639bd46ecd5373d2a4 P cf55383bef6649eb9e78cb2d2512dedc [term 1 LEADER]: Becoming Leader. State: Replica: cf55383bef6649eb9e78cb2d2512dedc, State: Running, Role: LEADER
I20250109 08:58:23.788529 23580 tablet_bootstrap.cc:492] T 86b82e19759c4fd0b471a0dbff839aa5 P ed1efbce9ee3401e99a5260edeae4e7c: Bootstrap starting.
I20250109 08:58:23.788827 23450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "732e4aa69aae44e9b1153ac330a01a96" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc" is_pre_election: true
I20250109 08:58:23.788563 23675 consensus_queue.cc:237] T e5251927614642639bd46ecd5373d2a4 P cf55383bef6649eb9e78cb2d2512dedc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:23.792076 23658 consensus_queue.cc:1035] T a686bf4d947c4b638049e6b88c59ae09 P ed1efbce9ee3401e99a5260edeae4e7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20250109 08:58:23.793954 23271 leader_election.cc:343] T 732e4aa69aae44e9b1153ac330a01a96 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:23.794377 23271 leader_election.cc:304] T 732e4aa69aae44e9b1153ac330a01a96 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 235ad6af04c34c88a5261daa3604fada; no voters: cf55383bef6649eb9e78cb2d2512dedc, ed1efbce9ee3401e99a5260edeae4e7c
I20250109 08:58:23.794729 23572 tablet_bootstrap.cc:492] T 732e4aa69aae44e9b1153ac330a01a96 P cf55383bef6649eb9e78cb2d2512dedc: No bootstrap required, opened a new log
I20250109 08:58:23.795006 23580 tablet_bootstrap.cc:654] T 86b82e19759c4fd0b471a0dbff839aa5 P ed1efbce9ee3401e99a5260edeae4e7c: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:23.795147 23572 ts_tablet_manager.cc:1397] T 732e4aa69aae44e9b1153ac330a01a96 P cf55383bef6649eb9e78cb2d2512dedc: Time spent bootstrapping tablet: real 0.270s user 0.031s sys 0.066s
I20250109 08:58:23.797695 23572 raft_consensus.cc:357] T 732e4aa69aae44e9b1153ac330a01a96 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:23.798338 23572 raft_consensus.cc:383] T 732e4aa69aae44e9b1153ac330a01a96 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:23.798600 23572 raft_consensus.cc:738] T 732e4aa69aae44e9b1153ac330a01a96 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf55383bef6649eb9e78cb2d2512dedc, State: Initialized, Role: FOLLOWER
I20250109 08:58:23.799217 23572 consensus_queue.cc:260] T 732e4aa69aae44e9b1153ac330a01a96 P cf55383bef6649eb9e78cb2d2512dedc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:23.800774 23666 raft_consensus.cc:2743] T 732e4aa69aae44e9b1153ac330a01a96 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:58:23.803269 23569 tablet_bootstrap.cc:492] T 86b82e19759c4fd0b471a0dbff839aa5 P 05a74e1eb2474b3d8b3a2da1ef127e01: No bootstrap required, opened a new log
I20250109 08:58:23.803788 23569 ts_tablet_manager.cc:1397] T 86b82e19759c4fd0b471a0dbff839aa5 P 05a74e1eb2474b3d8b3a2da1ef127e01: Time spent bootstrapping tablet: real 0.082s user 0.013s sys 0.030s
I20250109 08:58:23.806169 23569 raft_consensus.cc:357] T 86b82e19759c4fd0b471a0dbff839aa5 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:23.806823 23569 raft_consensus.cc:383] T 86b82e19759c4fd0b471a0dbff839aa5 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:23.807075 23569 raft_consensus.cc:738] T 86b82e19759c4fd0b471a0dbff839aa5 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05a74e1eb2474b3d8b3a2da1ef127e01, State: Initialized, Role: FOLLOWER
I20250109 08:58:23.807106 23580 tablet_bootstrap.cc:492] T 86b82e19759c4fd0b471a0dbff839aa5 P ed1efbce9ee3401e99a5260edeae4e7c: No bootstrap required, opened a new log
I20250109 08:58:23.807802 23569 consensus_queue.cc:260] T 86b82e19759c4fd0b471a0dbff839aa5 P 05a74e1eb2474b3d8b3a2da1ef127e01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:23.810177 23569 ts_tablet_manager.cc:1428] T 86b82e19759c4fd0b471a0dbff839aa5 P 05a74e1eb2474b3d8b3a2da1ef127e01: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20250109 08:58:23.811028 23569 tablet_bootstrap.cc:492] T 5f0d500a91b54851bda52e5ccf741fd6 P 05a74e1eb2474b3d8b3a2da1ef127e01: Bootstrap starting.
I20250109 08:58:23.811298 23572 ts_tablet_manager.cc:1428] T 732e4aa69aae44e9b1153ac330a01a96 P cf55383bef6649eb9e78cb2d2512dedc: Time spent starting tablet: real 0.016s user 0.005s sys 0.000s
I20250109 08:58:23.812206 23572 tablet_bootstrap.cc:492] T a7d2ab61d238471c9f287ac1d5db4ed0 P cf55383bef6649eb9e78cb2d2512dedc: Bootstrap starting.
I20250109 08:58:23.813308 23658 raft_consensus.cc:491] T f1f9274aeda748f493f5752814fb262d P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:23.813786 23658 raft_consensus.cc:513] T f1f9274aeda748f493f5752814fb262d P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:23.815775 23658 leader_election.cc:290] T f1f9274aeda748f493f5752814fb262d P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691), cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551)
I20250109 08:58:23.816617 23569 tablet_bootstrap.cc:654] T 5f0d500a91b54851bda52e5ccf741fd6 P 05a74e1eb2474b3d8b3a2da1ef127e01: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:23.817122 23580 ts_tablet_manager.cc:1397] T 86b82e19759c4fd0b471a0dbff839aa5 P ed1efbce9ee3401e99a5260edeae4e7c: Time spent bootstrapping tablet: real 0.029s user 0.013s sys 0.001s
I20250109 08:58:23.817318 23572 tablet_bootstrap.cc:654] T a7d2ab61d238471c9f287ac1d5db4ed0 P cf55383bef6649eb9e78cb2d2512dedc: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:23.820166 23574 tablet_bootstrap.cc:654] T a7d2ab61d238471c9f287ac1d5db4ed0 P 235ad6af04c34c88a5261daa3604fada: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:23.819721 23580 raft_consensus.cc:357] T 86b82e19759c4fd0b471a0dbff839aa5 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:23.820806 23580 raft_consensus.cc:383] T 86b82e19759c4fd0b471a0dbff839aa5 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:23.821190 23580 raft_consensus.cc:738] T 86b82e19759c4fd0b471a0dbff839aa5 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed1efbce9ee3401e99a5260edeae4e7c, State: Initialized, Role: FOLLOWER
I20250109 08:58:23.821872 23580 consensus_queue.cc:260] T 86b82e19759c4fd0b471a0dbff839aa5 P ed1efbce9ee3401e99a5260edeae4e7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:23.824255 23451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1f9274aeda748f493f5752814fb262d" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc" is_pre_election: true
W20250109 08:58:23.826395 23346 leader_election.cc:343] T f1f9274aeda748f493f5752814fb262d P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:23.828235 23569 tablet_bootstrap.cc:492] T 5f0d500a91b54851bda52e5ccf741fd6 P 05a74e1eb2474b3d8b3a2da1ef127e01: No bootstrap required, opened a new log
I20250109 08:58:23.828706 23574 tablet_bootstrap.cc:492] T a7d2ab61d238471c9f287ac1d5db4ed0 P 235ad6af04c34c88a5261daa3604fada: No bootstrap required, opened a new log
I20250109 08:58:23.828750 23569 ts_tablet_manager.cc:1397] T 5f0d500a91b54851bda52e5ccf741fd6 P 05a74e1eb2474b3d8b3a2da1ef127e01: Time spent bootstrapping tablet: real 0.018s user 0.009s sys 0.007s
I20250109 08:58:23.829337 23574 ts_tablet_manager.cc:1397] T a7d2ab61d238471c9f287ac1d5db4ed0 P 235ad6af04c34c88a5261daa3604fada: Time spent bootstrapping tablet: real 0.056s user 0.012s sys 0.001s
I20250109 08:58:23.829891 23303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1f9274aeda748f493f5752814fb262d" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada" is_pre_election: true
I20250109 08:58:23.831279 23580 ts_tablet_manager.cc:1428] T 86b82e19759c4fd0b471a0dbff839aa5 P ed1efbce9ee3401e99a5260edeae4e7c: Time spent starting tablet: real 0.014s user 0.003s sys 0.002s
I20250109 08:58:23.831704 23569 raft_consensus.cc:357] T 5f0d500a91b54851bda52e5ccf741fd6 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
W20250109 08:58:23.832161 23346 leader_election.cc:343] T f1f9274aeda748f493f5752814fb262d P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:23.832021 23574 raft_consensus.cc:357] T a7d2ab61d238471c9f287ac1d5db4ed0 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:23.832438 23569 raft_consensus.cc:383] T 5f0d500a91b54851bda52e5ccf741fd6 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:23.832756 23574 raft_consensus.cc:383] T a7d2ab61d238471c9f287ac1d5db4ed0 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:23.832336 23580 tablet_bootstrap.cc:492] T 732e4aa69aae44e9b1153ac330a01a96 P ed1efbce9ee3401e99a5260edeae4e7c: Bootstrap starting.
I20250109 08:58:23.833122 23569 raft_consensus.cc:738] T 5f0d500a91b54851bda52e5ccf741fd6 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05a74e1eb2474b3d8b3a2da1ef127e01, State: Initialized, Role: FOLLOWER
I20250109 08:58:23.832657 23346 leader_election.cc:304] T f1f9274aeda748f493f5752814fb262d P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ed1efbce9ee3401e99a5260edeae4e7c; no voters: 235ad6af04c34c88a5261daa3604fada, cf55383bef6649eb9e78cb2d2512dedc
I20250109 08:58:23.835191 23572 tablet_bootstrap.cc:492] T a7d2ab61d238471c9f287ac1d5db4ed0 P cf55383bef6649eb9e78cb2d2512dedc: No bootstrap required, opened a new log
I20250109 08:58:23.835889 23572 ts_tablet_manager.cc:1397] T a7d2ab61d238471c9f287ac1d5db4ed0 P cf55383bef6649eb9e78cb2d2512dedc: Time spent bootstrapping tablet: real 0.024s user 0.002s sys 0.008s
I20250109 08:58:23.833209 23574 raft_consensus.cc:738] T a7d2ab61d238471c9f287ac1d5db4ed0 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Initialized, Role: FOLLOWER
I20250109 08:58:23.836256 23658 raft_consensus.cc:2743] T f1f9274aeda748f493f5752814fb262d P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:58:23.837036 23574 consensus_queue.cc:260] T a7d2ab61d238471c9f287ac1d5db4ed0 P 235ad6af04c34c88a5261daa3604fada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:23.828662 23203 catalog_manager.cc:5526] T e5251927614642639bd46ecd5373d2a4 P cf55383bef6649eb9e78cb2d2512dedc reported cstate change: term changed from 0 to 1, leader changed from <none> to cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195). New cstate: current_term: 1 leader_uuid: "cf55383bef6649eb9e78cb2d2512dedc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:23.833987 23569 consensus_queue.cc:260] T 5f0d500a91b54851bda52e5ccf741fd6 P 05a74e1eb2474b3d8b3a2da1ef127e01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:23.838604 23572 raft_consensus.cc:357] T a7d2ab61d238471c9f287ac1d5db4ed0 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:23.839630 23572 raft_consensus.cc:383] T a7d2ab61d238471c9f287ac1d5db4ed0 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:23.839915 23572 raft_consensus.cc:738] T a7d2ab61d238471c9f287ac1d5db4ed0 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf55383bef6649eb9e78cb2d2512dedc, State: Initialized, Role: FOLLOWER
I20250109 08:58:23.840269 23666 raft_consensus.cc:491] T 86d4074509744900abe94402467a7d85 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ed1efbce9ee3401e99a5260edeae4e7c)
I20250109 08:58:23.840793 23666 raft_consensus.cc:513] T 86d4074509744900abe94402467a7d85 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:23.839182 23574 ts_tablet_manager.cc:1428] T a7d2ab61d238471c9f287ac1d5db4ed0 P 235ad6af04c34c88a5261daa3604fada: Time spent starting tablet: real 0.010s user 0.004s sys 0.000s
I20250109 08:58:23.841989 23569 ts_tablet_manager.cc:1428] T 5f0d500a91b54851bda52e5ccf741fd6 P 05a74e1eb2474b3d8b3a2da1ef127e01: Time spent starting tablet: real 0.013s user 0.006s sys 0.000s
I20250109 08:58:23.841737 23817 consensus_queue.cc:1035] T a686bf4d947c4b638049e6b88c59ae09 P ed1efbce9ee3401e99a5260edeae4e7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:58:23.842386 23574 tablet_bootstrap.cc:492] T 94f2be307f704c75bceb213e52307cfb P 235ad6af04c34c88a5261daa3604fada: Bootstrap starting.
I20250109 08:58:23.842892 23569 tablet_bootstrap.cc:492] T 8de6b2fb09bc46ce9e4033af06dffd04 P 05a74e1eb2474b3d8b3a2da1ef127e01: Bootstrap starting.
I20250109 08:58:23.844283 23572 consensus_queue.cc:260] T a7d2ab61d238471c9f287ac1d5db4ed0 P cf55383bef6649eb9e78cb2d2512dedc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:23.846302 23378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86d4074509744900abe94402467a7d85" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" is_pre_election: true
I20250109 08:58:23.846767 23572 ts_tablet_manager.cc:1428] T a7d2ab61d238471c9f287ac1d5db4ed0 P cf55383bef6649eb9e78cb2d2512dedc: Time spent starting tablet: real 0.011s user 0.000s sys 0.005s
I20250109 08:58:23.847707 23572 tablet_bootstrap.cc:492] T 0dab6c23ce444dda9427901a98f07b15 P cf55383bef6649eb9e78cb2d2512dedc: Bootstrap starting.
I20250109 08:58:23.848296 23574 tablet_bootstrap.cc:654] T 94f2be307f704c75bceb213e52307cfb P 235ad6af04c34c88a5261daa3604fada: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:23.848136 23451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86d4074509744900abe94402467a7d85" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc" is_pre_election: true
I20250109 08:58:23.854703 23569 tablet_bootstrap.cc:654] T 8de6b2fb09bc46ce9e4033af06dffd04 P 05a74e1eb2474b3d8b3a2da1ef127e01: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:23.844151 23666 leader_election.cc:290] T 86d4074509744900abe94402467a7d85 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365), cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551)
I20250109 08:58:23.855808 23271 leader_election.cc:304] T 86d4074509744900abe94402467a7d85 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 235ad6af04c34c88a5261daa3604fada; no voters: cf55383bef6649eb9e78cb2d2512dedc, ed1efbce9ee3401e99a5260edeae4e7c
I20250109 08:58:23.856665 23666 raft_consensus.cc:2743] T 86d4074509744900abe94402467a7d85 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250109 08:58:23.859632 23574 tablet_bootstrap.cc:492] T 94f2be307f704c75bceb213e52307cfb P 235ad6af04c34c88a5261daa3604fada: No bootstrap required, opened a new log
I20250109 08:58:23.860131 23574 ts_tablet_manager.cc:1397] T 94f2be307f704c75bceb213e52307cfb P 235ad6af04c34c88a5261daa3604fada: Time spent bootstrapping tablet: real 0.018s user 0.008s sys 0.007s
I20250109 08:58:23.860991 23572 tablet_bootstrap.cc:654] T 0dab6c23ce444dda9427901a98f07b15 P cf55383bef6649eb9e78cb2d2512dedc: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:23.862514 23574 raft_consensus.cc:357] T 94f2be307f704c75bceb213e52307cfb P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:23.862959 23580 tablet_bootstrap.cc:654] T 732e4aa69aae44e9b1153ac330a01a96 P ed1efbce9ee3401e99a5260edeae4e7c: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:23.863216 23574 raft_consensus.cc:383] T 94f2be307f704c75bceb213e52307cfb P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:23.863685 23574 raft_consensus.cc:738] T 94f2be307f704c75bceb213e52307cfb P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Initialized, Role: FOLLOWER
I20250109 08:58:23.864562 23574 consensus_queue.cc:260] T 94f2be307f704c75bceb213e52307cfb P 235ad6af04c34c88a5261daa3604fada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:23.866900 23574 ts_tablet_manager.cc:1428] T 94f2be307f704c75bceb213e52307cfb P 235ad6af04c34c88a5261daa3604fada: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20250109 08:58:23.877162 23574 tablet_bootstrap.cc:492] T db111e41c1be497e8e673a2c42b9ba66 P 235ad6af04c34c88a5261daa3604fada: Bootstrap starting.
I20250109 08:58:23.881960 23583 raft_consensus.cc:491] T a7d2ab61d238471c9f287ac1d5db4ed0 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:23.883492 23574 tablet_bootstrap.cc:654] T db111e41c1be497e8e673a2c42b9ba66 P 235ad6af04c34c88a5261daa3604fada: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:23.882493 23583 raft_consensus.cc:513] T a7d2ab61d238471c9f287ac1d5db4ed0 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:23.887367 23766 raft_consensus.cc:491] T 5f0d500a91b54851bda52e5ccf741fd6 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:23.888278 23580 tablet_bootstrap.cc:492] T 732e4aa69aae44e9b1153ac330a01a96 P ed1efbce9ee3401e99a5260edeae4e7c: No bootstrap required, opened a new log
I20250109 08:58:23.888760 23580 ts_tablet_manager.cc:1397] T 732e4aa69aae44e9b1153ac330a01a96 P ed1efbce9ee3401e99a5260edeae4e7c: Time spent bootstrapping tablet: real 0.057s user 0.006s sys 0.011s
I20250109 08:58:23.887573 23583 leader_election.cc:290] T a7d2ab61d238471c9f287ac1d5db4ed0 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829), 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691)
I20250109 08:58:23.893474 23658 raft_consensus.cc:491] T 86b82e19759c4fd0b471a0dbff839aa5 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:23.895139 23658 raft_consensus.cc:513] T 86b82e19759c4fd0b471a0dbff839aa5 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:23.896838 23305 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a7d2ab61d238471c9f287ac1d5db4ed0" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada" is_pre_election: true
I20250109 08:58:23.897504 23305 raft_consensus.cc:2463] T a7d2ab61d238471c9f287ac1d5db4ed0 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf55383bef6649eb9e78cb2d2512dedc in term 0.
I20250109 08:58:23.898530 23420 leader_election.cc:304] T a7d2ab61d238471c9f287ac1d5db4ed0 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 235ad6af04c34c88a5261daa3604fada, cf55383bef6649eb9e78cb2d2512dedc; no voters:
I20250109 08:58:23.899700 23583 raft_consensus.cc:2798] T a7d2ab61d238471c9f287ac1d5db4ed0 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:58:23.887912 23766 raft_consensus.cc:513] T 5f0d500a91b54851bda52e5ccf741fd6 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:23.900031 23583 raft_consensus.cc:491] T a7d2ab61d238471c9f287ac1d5db4ed0 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:58:23.900410 23583 raft_consensus.cc:3054] T a7d2ab61d238471c9f287ac1d5db4ed0 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:23.900607 23527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a7d2ab61d238471c9f287ac1d5db4ed0" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" is_pre_election: true
I20250109 08:58:23.903012 23572 tablet_bootstrap.cc:492] T 0dab6c23ce444dda9427901a98f07b15 P cf55383bef6649eb9e78cb2d2512dedc: No bootstrap required, opened a new log
I20250109 08:58:23.903502 23572 ts_tablet_manager.cc:1397] T 0dab6c23ce444dda9427901a98f07b15 P cf55383bef6649eb9e78cb2d2512dedc: Time spent bootstrapping tablet: real 0.056s user 0.015s sys 0.006s
I20250109 08:58:23.904673 23305 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f0d500a91b54851bda52e5ccf741fd6" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada" is_pre_election: true
I20250109 08:58:23.904856 23451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f0d500a91b54851bda52e5ccf741fd6" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc" is_pre_election: true
I20250109 08:58:23.905550 23451 raft_consensus.cc:2463] T 5f0d500a91b54851bda52e5ccf741fd6 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05a74e1eb2474b3d8b3a2da1ef127e01 in term 0.
I20250109 08:58:23.905858 23572 raft_consensus.cc:357] T 0dab6c23ce444dda9427901a98f07b15 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:23.906605 23572 raft_consensus.cc:383] T 0dab6c23ce444dda9427901a98f07b15 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:23.906859 23572 raft_consensus.cc:738] T 0dab6c23ce444dda9427901a98f07b15 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf55383bef6649eb9e78cb2d2512dedc, State: Initialized, Role: FOLLOWER
W20250109 08:58:23.907169 23418 leader_election.cc:343] T a7d2ab61d238471c9f287ac1d5db4ed0 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:23.907452 23572 consensus_queue.cc:260] T 0dab6c23ce444dda9427901a98f07b15 P cf55383bef6649eb9e78cb2d2512dedc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:23.907693 23580 raft_consensus.cc:357] T 732e4aa69aae44e9b1153ac330a01a96 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:23.902961 23766 leader_election.cc:290] T 5f0d500a91b54851bda52e5ccf741fd6 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551), 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691)
I20250109 08:58:23.909618 23572 ts_tablet_manager.cc:1428] T 0dab6c23ce444dda9427901a98f07b15 P cf55383bef6649eb9e78cb2d2512dedc: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
W20250109 08:58:23.905962 23494 leader_election.cc:343] T 5f0d500a91b54851bda52e5ccf741fd6 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:23.910523 23572 tablet_bootstrap.cc:492] T de98c70e25494d2a8e581443be836bd3 P cf55383bef6649eb9e78cb2d2512dedc: Bootstrap starting.
I20250109 08:58:23.910550 23494 leader_election.cc:304] T 5f0d500a91b54851bda52e5ccf741fd6 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01, cf55383bef6649eb9e78cb2d2512dedc; no voters: 235ad6af04c34c88a5261daa3604fada
I20250109 08:58:23.912948 23634 raft_consensus.cc:2798] T 5f0d500a91b54851bda52e5ccf741fd6 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:58:23.913302 23634 raft_consensus.cc:491] T 5f0d500a91b54851bda52e5ccf741fd6 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:58:23.913604 23634 raft_consensus.cc:3054] T 5f0d500a91b54851bda52e5ccf741fd6 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:23.913899 23658 leader_election.cc:290] T 86b82e19759c4fd0b471a0dbff839aa5 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829), 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691)
I20250109 08:58:23.908425 23580 raft_consensus.cc:383] T 732e4aa69aae44e9b1153ac330a01a96 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:23.914403 23580 raft_consensus.cc:738] T 732e4aa69aae44e9b1153ac330a01a96 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed1efbce9ee3401e99a5260edeae4e7c, State: Initialized, Role: FOLLOWER
I20250109 08:58:23.915006 23580 consensus_queue.cc:260] T 732e4aa69aae44e9b1153ac330a01a96 P ed1efbce9ee3401e99a5260edeae4e7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:23.915920 23527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86b82e19759c4fd0b471a0dbff839aa5" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" is_pre_election: true
I20250109 08:58:23.916659 23580 ts_tablet_manager.cc:1428] T 732e4aa69aae44e9b1153ac330a01a96 P ed1efbce9ee3401e99a5260edeae4e7c: Time spent starting tablet: real 0.028s user 0.006s sys 0.000s
I20250109 08:58:23.917549 23580 tablet_bootstrap.cc:492] T de98c70e25494d2a8e581443be836bd3 P ed1efbce9ee3401e99a5260edeae4e7c: Bootstrap starting.
I20250109 08:58:23.916585 23572 tablet_bootstrap.cc:654] T de98c70e25494d2a8e581443be836bd3 P cf55383bef6649eb9e78cb2d2512dedc: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:23.924968 23580 tablet_bootstrap.cc:654] T de98c70e25494d2a8e581443be836bd3 P ed1efbce9ee3401e99a5260edeae4e7c: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:23.930156 23305 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86b82e19759c4fd0b471a0dbff839aa5" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada" is_pre_election: true
W20250109 08:58:23.934059 23346 leader_election.cc:343] T 86b82e19759c4fd0b471a0dbff839aa5 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:23.920781 23527 raft_consensus.cc:2463] T 86b82e19759c4fd0b471a0dbff839aa5 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed1efbce9ee3401e99a5260edeae4e7c in term 0.
I20250109 08:58:23.936771 23344 leader_election.cc:304] T 86b82e19759c4fd0b471a0dbff839aa5 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01, ed1efbce9ee3401e99a5260edeae4e7c; no voters: 235ad6af04c34c88a5261daa3604fada
I20250109 08:58:23.938133 23817 raft_consensus.cc:2798] T 86b82e19759c4fd0b471a0dbff839aa5 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:58:23.938558 23817 raft_consensus.cc:491] T 86b82e19759c4fd0b471a0dbff839aa5 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:58:23.938884 23817 raft_consensus.cc:3054] T 86b82e19759c4fd0b471a0dbff839aa5 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:23.940910 23583 raft_consensus.cc:513] T a7d2ab61d238471c9f287ac1d5db4ed0 P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:23.940951 23675 consensus_queue.cc:1035] T 8de6b2fb09bc46ce9e4033af06dffd04 P cf55383bef6649eb9e78cb2d2512dedc [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250109 08:58:23.950569 23675 raft_consensus.cc:491] T 0dab6c23ce444dda9427901a98f07b15 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:23.951089 23675 raft_consensus.cc:513] T 0dab6c23ce444dda9427901a98f07b15 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:23.952922 23527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a7d2ab61d238471c9f287ac1d5db4ed0" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01"
W20250109 08:58:23.954272 23418 leader_election.cc:343] T a7d2ab61d238471c9f287ac1d5db4ed0 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:23.943945 23634 raft_consensus.cc:513] T 5f0d500a91b54851bda52e5ccf741fd6 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:23.955794 23675 leader_election.cc:290] T 0dab6c23ce444dda9427901a98f07b15 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365), 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829)
I20250109 08:58:23.956652 23634 leader_election.cc:290] T 5f0d500a91b54851bda52e5ccf741fd6 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 election: Requested vote from peers cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551), 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691)
I20250109 08:58:23.960125 23451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f0d500a91b54851bda52e5ccf741fd6" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc"
I20250109 08:58:23.960868 23451 raft_consensus.cc:3054] T 5f0d500a91b54851bda52e5ccf741fd6 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:23.961759 23527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0dab6c23ce444dda9427901a98f07b15" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" is_pre_election: true
W20250109 08:58:23.963188 23418 leader_election.cc:343] T 0dab6c23ce444dda9427901a98f07b15 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:23.964143 23666 raft_consensus.cc:491] T 94f2be307f704c75bceb213e52307cfb P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:23.964649 23666 raft_consensus.cc:513] T 94f2be307f704c75bceb213e52307cfb P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:23.966638 23666 leader_election.cc:290] T 94f2be307f704c75bceb213e52307cfb P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551), 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829)
I20250109 08:58:23.967813 23451 raft_consensus.cc:2463] T 5f0d500a91b54851bda52e5ccf741fd6 P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05a74e1eb2474b3d8b3a2da1ef127e01 in term 1.
I20250109 08:58:23.968040 23377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0dab6c23ce444dda9427901a98f07b15" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" is_pre_election: true
I20250109 08:58:23.960584 23305 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f0d500a91b54851bda52e5ccf741fd6" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada"
I20250109 08:58:23.968905 23494 leader_election.cc:304] T 5f0d500a91b54851bda52e5ccf741fd6 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01, cf55383bef6649eb9e78cb2d2512dedc; no voters:
I20250109 08:58:23.969259 23450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94f2be307f704c75bceb213e52307cfb" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc" is_pre_election: true
W20250109 08:58:23.969643 23420 leader_election.cc:343] T 0dab6c23ce444dda9427901a98f07b15 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365): Illegal state: must be running to vote when last-logged opid is not known
W20250109 08:58:23.969861 23494 leader_election.cc:343] T 5f0d500a91b54851bda52e5ccf741fd6 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:23.970031 23634 consensus_queue.cc:1035] T d87d3b4755344b82b0aec7ffd1ce785c P 05a74e1eb2474b3d8b3a2da1ef127e01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:58:23.970314 23675 consensus_queue.cc:1035] T a09d8dffc2e446f1acd3c7f6cc1705bb P cf55383bef6649eb9e78cb2d2512dedc [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20250109 08:58:23.970595 23271 leader_election.cc:343] T 94f2be307f704c75bceb213e52307cfb P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:23.970790 23766 raft_consensus.cc:2798] T 5f0d500a91b54851bda52e5ccf741fd6 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:58:23.971268 23766 raft_consensus.cc:695] T 5f0d500a91b54851bda52e5ccf741fd6 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 LEADER]: Becoming Leader. State: Replica: 05a74e1eb2474b3d8b3a2da1ef127e01, State: Running, Role: LEADER
I20250109 08:58:23.943553 23583 leader_election.cc:290] T a7d2ab61d238471c9f287ac1d5db4ed0 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 election: Requested vote from peers 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829), 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691)
I20250109 08:58:23.969866 23527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94f2be307f704c75bceb213e52307cfb" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" is_pre_election: true
I20250109 08:58:23.980630 23527 raft_consensus.cc:2463] T 94f2be307f704c75bceb213e52307cfb P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 235ad6af04c34c88a5261daa3604fada in term 0.
I20250109 08:58:23.979918 23305 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a7d2ab61d238471c9f287ac1d5db4ed0" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada"
I20250109 08:58:23.981528 23305 raft_consensus.cc:3054] T a7d2ab61d238471c9f287ac1d5db4ed0 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:23.984887 23269 leader_election.cc:304] T 94f2be307f704c75bceb213e52307cfb P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01, 235ad6af04c34c88a5261daa3604fada; no voters: cf55383bef6649eb9e78cb2d2512dedc
I20250109 08:58:23.985895 23666 raft_consensus.cc:2798] T 94f2be307f704c75bceb213e52307cfb P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:58:23.986299 23666 raft_consensus.cc:491] T 94f2be307f704c75bceb213e52307cfb P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:58:23.986675 23666 raft_consensus.cc:3054] T 94f2be307f704c75bceb213e52307cfb P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:23.990697 23817 raft_consensus.cc:513] T 86b82e19759c4fd0b471a0dbff839aa5 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:23.993389 23572 tablet_bootstrap.cc:492] T de98c70e25494d2a8e581443be836bd3 P cf55383bef6649eb9e78cb2d2512dedc: No bootstrap required, opened a new log
I20250109 08:58:23.975070 23420 leader_election.cc:304] T 0dab6c23ce444dda9427901a98f07b15 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf55383bef6649eb9e78cb2d2512dedc; no voters: 05a74e1eb2474b3d8b3a2da1ef127e01, ed1efbce9ee3401e99a5260edeae4e7c
I20250109 08:58:23.993927 23572 ts_tablet_manager.cc:1397] T de98c70e25494d2a8e581443be836bd3 P cf55383bef6649eb9e78cb2d2512dedc: Time spent bootstrapping tablet: real 0.084s user 0.014s sys 0.008s
I20250109 08:58:23.995429 23817 leader_election.cc:290] T 86b82e19759c4fd0b471a0dbff839aa5 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 election: Requested vote from peers 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829), 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691)
I20250109 08:58:23.996683 23666 raft_consensus.cc:513] T 94f2be307f704c75bceb213e52307cfb P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:23.998721 23666 leader_election.cc:290] T 94f2be307f704c75bceb213e52307cfb P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 election: Requested vote from peers cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551), 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829)
I20250109 08:58:23.991053 23305 raft_consensus.cc:2463] T a7d2ab61d238471c9f287ac1d5db4ed0 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf55383bef6649eb9e78cb2d2512dedc in term 1.
W20250109 08:58:23.999735 23494 consensus_peers.cc:487] T d87d3b4755344b82b0aec7ffd1ce785c P 05a74e1eb2474b3d8b3a2da1ef127e01 -> Peer ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365): Couldn't send request to peer ed1efbce9ee3401e99a5260edeae4e7c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:24.000195 23304 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86b82e19759c4fd0b471a0dbff839aa5" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada"
I20250109 08:58:23.996819 23572 raft_consensus.cc:357] T de98c70e25494d2a8e581443be836bd3 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:24.001183 23572 raft_consensus.cc:383] T de98c70e25494d2a8e581443be836bd3 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:23.972066 23766 consensus_queue.cc:237] T 5f0d500a91b54851bda52e5ccf741fd6 P 05a74e1eb2474b3d8b3a2da1ef127e01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:24.001338 23527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86b82e19759c4fd0b471a0dbff839aa5" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01"
I20250109 08:58:24.001955 23574 tablet_bootstrap.cc:492] T db111e41c1be497e8e673a2c42b9ba66 P 235ad6af04c34c88a5261daa3604fada: No bootstrap required, opened a new log
I20250109 08:58:24.002118 23527 raft_consensus.cc:3054] T 86b82e19759c4fd0b471a0dbff839aa5 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:24.002430 23574 ts_tablet_manager.cc:1397] T db111e41c1be497e8e673a2c42b9ba66 P 235ad6af04c34c88a5261daa3604fada: Time spent bootstrapping tablet: real 0.135s user 0.027s sys 0.016s
I20250109 08:58:24.001470 23572 raft_consensus.cc:738] T de98c70e25494d2a8e581443be836bd3 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf55383bef6649eb9e78cb2d2512dedc, State: Initialized, Role: FOLLOWER
I20250109 08:58:24.004639 23572 consensus_queue.cc:260] T de98c70e25494d2a8e581443be836bd3 P cf55383bef6649eb9e78cb2d2512dedc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
W20250109 08:58:24.007542 23346 leader_election.cc:343] T 86b82e19759c4fd0b471a0dbff839aa5 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:24.011945 23572 ts_tablet_manager.cc:1428] T de98c70e25494d2a8e581443be836bd3 P cf55383bef6649eb9e78cb2d2512dedc: Time spent starting tablet: real 0.018s user 0.004s sys 0.002s
I20250109 08:58:24.013141 23572 tablet_bootstrap.cc:492] T 11aa76a3a30643708fececed660a5250 P cf55383bef6649eb9e78cb2d2512dedc: Bootstrap starting.
I20250109 08:58:24.018219 23574 raft_consensus.cc:357] T db111e41c1be497e8e673a2c42b9ba66 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:24.018996 23574 raft_consensus.cc:383] T db111e41c1be497e8e673a2c42b9ba66 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:24.019799 23451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94f2be307f704c75bceb213e52307cfb" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc"
W20250109 08:58:24.021620 23271 leader_election.cc:343] T 94f2be307f704c75bceb213e52307cfb P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:24.005033 23675 raft_consensus.cc:2743] T 0dab6c23ce444dda9427901a98f07b15 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:58:24.015141 23524 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94f2be307f704c75bceb213e52307cfb" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01"
I20250109 08:58:24.025835 23524 raft_consensus.cc:3054] T 94f2be307f704c75bceb213e52307cfb P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:24.027942 23527 raft_consensus.cc:2463] T 86b82e19759c4fd0b471a0dbff839aa5 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed1efbce9ee3401e99a5260edeae4e7c in term 1.
I20250109 08:58:24.029438 23420 leader_election.cc:304] T a7d2ab61d238471c9f287ac1d5db4ed0 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 235ad6af04c34c88a5261daa3604fada, cf55383bef6649eb9e78cb2d2512dedc; no voters: 05a74e1eb2474b3d8b3a2da1ef127e01
I20250109 08:58:24.030025 23569 tablet_bootstrap.cc:492] T 8de6b2fb09bc46ce9e4033af06dffd04 P 05a74e1eb2474b3d8b3a2da1ef127e01: No bootstrap required, opened a new log
I20250109 08:58:24.030230 23677 raft_consensus.cc:2798] T a7d2ab61d238471c9f287ac1d5db4ed0 P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:58:24.030753 23569 ts_tablet_manager.cc:1397] T 8de6b2fb09bc46ce9e4033af06dffd04 P 05a74e1eb2474b3d8b3a2da1ef127e01: Time spent bootstrapping tablet: real 0.188s user 0.019s sys 0.031s
I20250109 08:58:24.030728 23677 raft_consensus.cc:695] T a7d2ab61d238471c9f287ac1d5db4ed0 P cf55383bef6649eb9e78cb2d2512dedc [term 1 LEADER]: Becoming Leader. State: Replica: cf55383bef6649eb9e78cb2d2512dedc, State: Running, Role: LEADER
I20250109 08:58:24.031701 23677 consensus_queue.cc:237] T a7d2ab61d238471c9f287ac1d5db4ed0 P cf55383bef6649eb9e78cb2d2512dedc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:24.033635 23569 raft_consensus.cc:357] T 8de6b2fb09bc46ce9e4033af06dffd04 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:24.034372 23569 raft_consensus.cc:383] T 8de6b2fb09bc46ce9e4033af06dffd04 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:24.034777 23569 raft_consensus.cc:738] T 8de6b2fb09bc46ce9e4033af06dffd04 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05a74e1eb2474b3d8b3a2da1ef127e01, State: Initialized, Role: FOLLOWER
I20250109 08:58:24.035485 23569 consensus_queue.cc:260] T 8de6b2fb09bc46ce9e4033af06dffd04 P 05a74e1eb2474b3d8b3a2da1ef127e01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:24.019261 23574 raft_consensus.cc:738] T db111e41c1be497e8e673a2c42b9ba66 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Initialized, Role: FOLLOWER
I20250109 08:58:24.037477 23574 consensus_queue.cc:260] T db111e41c1be497e8e673a2c42b9ba66 P 235ad6af04c34c88a5261daa3604fada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:24.039271 23569 ts_tablet_manager.cc:1428] T 8de6b2fb09bc46ce9e4033af06dffd04 P 05a74e1eb2474b3d8b3a2da1ef127e01: Time spent starting tablet: real 0.008s user 0.005s sys 0.001s
I20250109 08:58:24.044996 23344 leader_election.cc:304] T 86b82e19759c4fd0b471a0dbff839aa5 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01, ed1efbce9ee3401e99a5260edeae4e7c; no voters: 235ad6af04c34c88a5261daa3604fada
I20250109 08:58:24.045801 23658 raft_consensus.cc:2798] T 86b82e19759c4fd0b471a0dbff839aa5 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:58:24.046252 23658 raft_consensus.cc:695] T 86b82e19759c4fd0b471a0dbff839aa5 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 LEADER]: Becoming Leader. State: Replica: ed1efbce9ee3401e99a5260edeae4e7c, State: Running, Role: LEADER
I20250109 08:58:24.047030 23658 consensus_queue.cc:237] T 86b82e19759c4fd0b471a0dbff839aa5 P ed1efbce9ee3401e99a5260edeae4e7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:24.039296 23817 raft_consensus.cc:491] T 732e4aa69aae44e9b1153ac330a01a96 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:24.048786 23817 raft_consensus.cc:513] T 732e4aa69aae44e9b1153ac330a01a96 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:24.050595 23817 leader_election.cc:290] T 732e4aa69aae44e9b1153ac330a01a96 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691), cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551)
I20250109 08:58:24.057174 23580 tablet_bootstrap.cc:492] T de98c70e25494d2a8e581443be836bd3 P ed1efbce9ee3401e99a5260edeae4e7c: No bootstrap required, opened a new log
I20250109 08:58:24.056885 23201 catalog_manager.cc:5526] T 5f0d500a91b54851bda52e5ccf741fd6 P 05a74e1eb2474b3d8b3a2da1ef127e01 reported cstate change: term changed from 0 to 1, leader changed from <none> to 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196). New cstate: current_term: 1 leader_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:24.057709 23580 ts_tablet_manager.cc:1397] T de98c70e25494d2a8e581443be836bd3 P ed1efbce9ee3401e99a5260edeae4e7c: Time spent bootstrapping tablet: real 0.140s user 0.023s sys 0.014s
I20250109 08:58:24.060914 23627 consensus_queue.cc:1035] T a09d8dffc2e446f1acd3c7f6cc1705bb P cf55383bef6649eb9e78cb2d2512dedc [LEADER]: Connected to new peer: Peer: permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:58:24.056229 23450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "732e4aa69aae44e9b1153ac330a01a96" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc" is_pre_election: true
I20250109 08:58:24.062367 23450 raft_consensus.cc:2463] T 732e4aa69aae44e9b1153ac330a01a96 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed1efbce9ee3401e99a5260edeae4e7c in term 0.
I20250109 08:58:24.063488 23569 tablet_bootstrap.cc:492] T 0dab6c23ce444dda9427901a98f07b15 P 05a74e1eb2474b3d8b3a2da1ef127e01: Bootstrap starting.
I20250109 08:58:24.065451 23574 ts_tablet_manager.cc:1428] T db111e41c1be497e8e673a2c42b9ba66 P 235ad6af04c34c88a5261daa3604fada: Time spent starting tablet: real 0.063s user 0.005s sys 0.003s
I20250109 08:58:24.067333 23574 tablet_bootstrap.cc:492] T 86b82e19759c4fd0b471a0dbff839aa5 P 235ad6af04c34c88a5261daa3604fada: Bootstrap starting.
I20250109 08:58:24.073060 23572 tablet_bootstrap.cc:654] T 11aa76a3a30643708fececed660a5250 P cf55383bef6649eb9e78cb2d2512dedc: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:24.074916 23346 leader_election.cc:304] T 732e4aa69aae44e9b1153ac330a01a96 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf55383bef6649eb9e78cb2d2512dedc, ed1efbce9ee3401e99a5260edeae4e7c; no voters:
I20250109 08:58:24.077216 23527 raft_consensus.cc:3054] T 8de6b2fb09bc46ce9e4033af06dffd04 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:24.081573 23574 tablet_bootstrap.cc:654] T 86b82e19759c4fd0b471a0dbff839aa5 P 235ad6af04c34c88a5261daa3604fada: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:24.082968 23816 raft_consensus.cc:2798] T 732e4aa69aae44e9b1153ac330a01a96 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:58:24.083367 23816 raft_consensus.cc:491] T 732e4aa69aae44e9b1153ac330a01a96 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:58:24.083704 23816 raft_consensus.cc:3054] T 732e4aa69aae44e9b1153ac330a01a96 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:24.089085 23303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "732e4aa69aae44e9b1153ac330a01a96" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada" is_pre_election: true
I20250109 08:58:24.089864 23303 raft_consensus.cc:2463] T 732e4aa69aae44e9b1153ac330a01a96 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed1efbce9ee3401e99a5260edeae4e7c in term 0.
I20250109 08:58:24.094909 23816 raft_consensus.cc:513] T 732e4aa69aae44e9b1153ac330a01a96 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:24.096642 23816 leader_election.cc:290] T 732e4aa69aae44e9b1153ac330a01a96 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 election: Requested vote from peers 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691), cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551)
W20250109 08:58:24.101508 23860 log.cc:926] Time spent T 5f0d500a91b54851bda52e5ccf741fd6 P 05a74e1eb2474b3d8b3a2da1ef127e01: Append to log took a long time: real 0.064s user 0.001s sys 0.000s
I20250109 08:58:24.111320 23569 tablet_bootstrap.cc:654] T 0dab6c23ce444dda9427901a98f07b15 P 05a74e1eb2474b3d8b3a2da1ef127e01: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:24.112187 23201 catalog_manager.cc:5526] T a7d2ab61d238471c9f287ac1d5db4ed0 P cf55383bef6649eb9e78cb2d2512dedc reported cstate change: term changed from 0 to 1, leader changed from <none> to cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195). New cstate: current_term: 1 leader_uuid: "cf55383bef6649eb9e78cb2d2512dedc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:24.122920 23580 raft_consensus.cc:357] T de98c70e25494d2a8e581443be836bd3 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:24.124254 23580 raft_consensus.cc:383] T de98c70e25494d2a8e581443be836bd3 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:24.124650 23580 raft_consensus.cc:738] T de98c70e25494d2a8e581443be836bd3 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed1efbce9ee3401e99a5260edeae4e7c, State: Initialized, Role: FOLLOWER
I20250109 08:58:24.128557 23580 consensus_queue.cc:260] T de98c70e25494d2a8e581443be836bd3 P ed1efbce9ee3401e99a5260edeae4e7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:24.134668 23450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "732e4aa69aae44e9b1153ac330a01a96" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc"
I20250109 08:58:24.135361 23450 raft_consensus.cc:3054] T 732e4aa69aae44e9b1153ac330a01a96 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:24.136179 23304 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "732e4aa69aae44e9b1153ac330a01a96" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada"
I20250109 08:58:24.136852 23304 raft_consensus.cc:3054] T 732e4aa69aae44e9b1153ac330a01a96 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:24.144994 23524 raft_consensus.cc:2463] T 94f2be307f704c75bceb213e52307cfb P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 235ad6af04c34c88a5261daa3604fada in term 1.
I20250109 08:58:24.146103 23572 tablet_bootstrap.cc:492] T 11aa76a3a30643708fececed660a5250 P cf55383bef6649eb9e78cb2d2512dedc: No bootstrap required, opened a new log
I20250109 08:58:24.146798 23572 ts_tablet_manager.cc:1397] T 11aa76a3a30643708fececed660a5250 P cf55383bef6649eb9e78cb2d2512dedc: Time spent bootstrapping tablet: real 0.134s user 0.010s sys 0.002s
I20250109 08:58:24.147137 23580 ts_tablet_manager.cc:1428] T de98c70e25494d2a8e581443be836bd3 P ed1efbce9ee3401e99a5260edeae4e7c: Time spent starting tablet: real 0.089s user 0.007s sys 0.000s
I20250109 08:58:24.148119 23580 tablet_bootstrap.cc:492] T db111e41c1be497e8e673a2c42b9ba66 P ed1efbce9ee3401e99a5260edeae4e7c: Bootstrap starting.
I20250109 08:58:24.149305 23572 raft_consensus.cc:357] T 11aa76a3a30643708fececed660a5250 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:24.150058 23572 raft_consensus.cc:383] T 11aa76a3a30643708fececed660a5250 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:24.150321 23572 raft_consensus.cc:738] T 11aa76a3a30643708fececed660a5250 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf55383bef6649eb9e78cb2d2512dedc, State: Initialized, Role: FOLLOWER
I20250109 08:58:24.151262 23572 consensus_queue.cc:260] T 11aa76a3a30643708fececed660a5250 P cf55383bef6649eb9e78cb2d2512dedc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:24.153962 23572 ts_tablet_manager.cc:1428] T 11aa76a3a30643708fececed660a5250 P cf55383bef6649eb9e78cb2d2512dedc: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20250109 08:58:24.153873 23304 raft_consensus.cc:2463] T 732e4aa69aae44e9b1153ac330a01a96 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed1efbce9ee3401e99a5260edeae4e7c in term 1.
I20250109 08:58:24.156899 23572 tablet_bootstrap.cc:492] T 1365130ef97c4581b623c22c39753831 P cf55383bef6649eb9e78cb2d2512dedc: Bootstrap starting.
I20250109 08:58:24.150373 23203 catalog_manager.cc:5526] T 86b82e19759c4fd0b471a0dbff839aa5 P ed1efbce9ee3401e99a5260edeae4e7c reported cstate change: term changed from 0 to 1, leader changed from <none> to ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194). New cstate: current_term: 1 leader_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } health_report { overall_health: HEALTHY } } }
I20250109 08:58:24.163569 23580 tablet_bootstrap.cc:654] T db111e41c1be497e8e673a2c42b9ba66 P ed1efbce9ee3401e99a5260edeae4e7c: Neither blocks nor log segments found. Creating new log.
W20250109 08:58:24.167981 23858 log.cc:926] Time spent T 8de6b2fb09bc46ce9e4033af06dffd04 P ed1efbce9ee3401e99a5260edeae4e7c: Append to log took a long time: real 0.063s user 0.000s sys 0.000s
I20250109 08:58:24.169304 23572 tablet_bootstrap.cc:654] T 1365130ef97c4581b623c22c39753831 P cf55383bef6649eb9e78cb2d2512dedc: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:24.189090 23569 tablet_bootstrap.cc:492] T 0dab6c23ce444dda9427901a98f07b15 P 05a74e1eb2474b3d8b3a2da1ef127e01: No bootstrap required, opened a new log
I20250109 08:58:24.189508 23569 ts_tablet_manager.cc:1397] T 0dab6c23ce444dda9427901a98f07b15 P 05a74e1eb2474b3d8b3a2da1ef127e01: Time spent bootstrapping tablet: real 0.132s user 0.009s sys 0.007s
I20250109 08:58:24.189942 23666 raft_consensus.cc:491] T db111e41c1be497e8e673a2c42b9ba66 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:24.190521 23666 raft_consensus.cc:513] T db111e41c1be497e8e673a2c42b9ba66 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:24.191793 23569 raft_consensus.cc:357] T 0dab6c23ce444dda9427901a98f07b15 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:24.192348 23666 leader_election.cc:290] T db111e41c1be497e8e673a2c42b9ba66 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365), cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551)
I20250109 08:58:24.192544 23569 raft_consensus.cc:383] T 0dab6c23ce444dda9427901a98f07b15 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:24.192914 23569 raft_consensus.cc:738] T 0dab6c23ce444dda9427901a98f07b15 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05a74e1eb2474b3d8b3a2da1ef127e01, State: Initialized, Role: FOLLOWER
I20250109 08:58:24.193584 23569 consensus_queue.cc:260] T 0dab6c23ce444dda9427901a98f07b15 P 05a74e1eb2474b3d8b3a2da1ef127e01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:24.194151 23378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db111e41c1be497e8e673a2c42b9ba66" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" is_pre_election: true
I20250109 08:58:24.195696 23451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db111e41c1be497e8e673a2c42b9ba66" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc" is_pre_election: true
I20250109 08:58:24.188522 23346 leader_election.cc:304] T 732e4aa69aae44e9b1153ac330a01a96 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 235ad6af04c34c88a5261daa3604fada, ed1efbce9ee3401e99a5260edeae4e7c; no voters:
I20250109 08:58:24.197202 23569 ts_tablet_manager.cc:1428] T 0dab6c23ce444dda9427901a98f07b15 P 05a74e1eb2474b3d8b3a2da1ef127e01: Time spent starting tablet: real 0.007s user 0.005s sys 0.001s
W20250109 08:58:24.197930 23271 leader_election.cc:343] T db111e41c1be497e8e673a2c42b9ba66 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:24.198119 23569 tablet_bootstrap.cc:492] T 5c37c58abaa54656b31c72160b4dec9e P 05a74e1eb2474b3d8b3a2da1ef127e01: Bootstrap starting.
I20250109 08:58:24.200134 23450 raft_consensus.cc:2463] T 732e4aa69aae44e9b1153ac330a01a96 P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed1efbce9ee3401e99a5260edeae4e7c in term 1.
I20250109 08:58:24.197285 23658 raft_consensus.cc:2798] T 732e4aa69aae44e9b1153ac330a01a96 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:58:24.200935 23658 raft_consensus.cc:695] T 732e4aa69aae44e9b1153ac330a01a96 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 LEADER]: Becoming Leader. State: Replica: ed1efbce9ee3401e99a5260edeae4e7c, State: Running, Role: LEADER
I20250109 08:58:24.196458 23269 leader_election.cc:304] T 94f2be307f704c75bceb213e52307cfb P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01, 235ad6af04c34c88a5261daa3604fada; no voters: cf55383bef6649eb9e78cb2d2512dedc
I20250109 08:58:24.196336 23451 raft_consensus.cc:2463] T db111e41c1be497e8e673a2c42b9ba66 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 235ad6af04c34c88a5261daa3604fada in term 0.
I20250109 08:58:24.201718 23658 consensus_queue.cc:237] T 732e4aa69aae44e9b1153ac330a01a96 P ed1efbce9ee3401e99a5260edeae4e7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:24.203326 23767 raft_consensus.cc:2798] T 94f2be307f704c75bceb213e52307cfb P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:58:24.203792 23767 raft_consensus.cc:695] T 94f2be307f704c75bceb213e52307cfb P 235ad6af04c34c88a5261daa3604fada [term 1 LEADER]: Becoming Leader. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Running, Role: LEADER
I20250109 08:58:24.203933 23569 tablet_bootstrap.cc:654] T 5c37c58abaa54656b31c72160b4dec9e P 05a74e1eb2474b3d8b3a2da1ef127e01: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:24.205128 23271 leader_election.cc:304] T db111e41c1be497e8e673a2c42b9ba66 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 235ad6af04c34c88a5261daa3604fada, cf55383bef6649eb9e78cb2d2512dedc; no voters: ed1efbce9ee3401e99a5260edeae4e7c
I20250109 08:58:24.207218 23677 consensus_queue.cc:1035] T 8de6b2fb09bc46ce9e4033af06dffd04 P cf55383bef6649eb9e78cb2d2512dedc [LEADER]: Connected to new peer: Peer: permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:58:24.211354 23666 raft_consensus.cc:2798] T db111e41c1be497e8e673a2c42b9ba66 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:58:24.222147 23574 tablet_bootstrap.cc:492] T 86b82e19759c4fd0b471a0dbff839aa5 P 235ad6af04c34c88a5261daa3604fada: No bootstrap required, opened a new log
I20250109 08:58:24.222543 23574 ts_tablet_manager.cc:1397] T 86b82e19759c4fd0b471a0dbff839aa5 P 235ad6af04c34c88a5261daa3604fada: Time spent bootstrapping tablet: real 0.156s user 0.021s sys 0.026s
W20250109 08:58:24.217914 23862 log.cc:926] Time spent T a7d2ab61d238471c9f287ac1d5db4ed0 P cf55383bef6649eb9e78cb2d2512dedc: Append to log took a long time: real 0.075s user 0.000s sys 0.001s
I20250109 08:58:24.224231 23574 raft_consensus.cc:357] T 86b82e19759c4fd0b471a0dbff839aa5 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:24.224861 23574 raft_consensus.cc:383] T 86b82e19759c4fd0b471a0dbff839aa5 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:24.224545 23767 consensus_queue.cc:237] T 94f2be307f704c75bceb213e52307cfb P 235ad6af04c34c88a5261daa3604fada [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:24.225145 23574 raft_consensus.cc:738] T 86b82e19759c4fd0b471a0dbff839aa5 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Initialized, Role: FOLLOWER
I20250109 08:58:24.219718 23666 raft_consensus.cc:491] T db111e41c1be497e8e673a2c42b9ba66 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:58:24.225885 23574 consensus_queue.cc:260] T 86b82e19759c4fd0b471a0dbff839aa5 P 235ad6af04c34c88a5261daa3604fada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:24.228704 23666 raft_consensus.cc:3054] T db111e41c1be497e8e673a2c42b9ba66 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:24.232638 23677 raft_consensus.cc:491] T de98c70e25494d2a8e581443be836bd3 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:24.233180 23677 raft_consensus.cc:513] T de98c70e25494d2a8e581443be836bd3 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:24.235216 23677 leader_election.cc:290] T de98c70e25494d2a8e581443be836bd3 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691), ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365)
I20250109 08:58:24.236400 23580 tablet_bootstrap.cc:492] T db111e41c1be497e8e673a2c42b9ba66 P ed1efbce9ee3401e99a5260edeae4e7c: No bootstrap required, opened a new log
I20250109 08:58:24.236869 23580 ts_tablet_manager.cc:1397] T db111e41c1be497e8e673a2c42b9ba66 P ed1efbce9ee3401e99a5260edeae4e7c: Time spent bootstrapping tablet: real 0.089s user 0.018s sys 0.006s
I20250109 08:58:24.240357 23305 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de98c70e25494d2a8e581443be836bd3" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada" is_pre_election: true
I20250109 08:58:24.241801 23574 ts_tablet_manager.cc:1428] T 86b82e19759c4fd0b471a0dbff839aa5 P 235ad6af04c34c88a5261daa3604fada: Time spent starting tablet: real 0.019s user 0.005s sys 0.001s
I20250109 08:58:24.239326 23580 raft_consensus.cc:357] T db111e41c1be497e8e673a2c42b9ba66 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:24.242314 23580 raft_consensus.cc:383] T db111e41c1be497e8e673a2c42b9ba66 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:24.242817 23580 raft_consensus.cc:738] T db111e41c1be497e8e673a2c42b9ba66 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed1efbce9ee3401e99a5260edeae4e7c, State: Initialized, Role: FOLLOWER
I20250109 08:58:24.243110 23574 tablet_bootstrap.cc:492] T f1f9274aeda748f493f5752814fb262d P 235ad6af04c34c88a5261daa3604fada: Bootstrap starting.
I20250109 08:58:24.245294 23378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de98c70e25494d2a8e581443be836bd3" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" is_pre_election: true
I20250109 08:58:24.245939 23378 raft_consensus.cc:2463] T de98c70e25494d2a8e581443be836bd3 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf55383bef6649eb9e78cb2d2512dedc in term 0.
I20250109 08:58:24.247185 23420 leader_election.cc:304] T de98c70e25494d2a8e581443be836bd3 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf55383bef6649eb9e78cb2d2512dedc, ed1efbce9ee3401e99a5260edeae4e7c; no voters:
I20250109 08:58:24.248013 23583 raft_consensus.cc:2798] T de98c70e25494d2a8e581443be836bd3 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:58:24.248386 23583 raft_consensus.cc:491] T de98c70e25494d2a8e581443be836bd3 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:58:24.248705 23583 raft_consensus.cc:3054] T de98c70e25494d2a8e581443be836bd3 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:24.249711 23574 tablet_bootstrap.cc:654] T f1f9274aeda748f493f5752814fb262d P 235ad6af04c34c88a5261daa3604fada: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:24.243806 23580 consensus_queue.cc:260] T db111e41c1be497e8e673a2c42b9ba66 P ed1efbce9ee3401e99a5260edeae4e7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:24.252976 23580 ts_tablet_manager.cc:1428] T db111e41c1be497e8e673a2c42b9ba66 P ed1efbce9ee3401e99a5260edeae4e7c: Time spent starting tablet: real 0.016s user 0.006s sys 0.000s
I20250109 08:58:24.253875 23580 tablet_bootstrap.cc:492] T ecc833ad1afa4079a75e41b1a807c923 P ed1efbce9ee3401e99a5260edeae4e7c: Bootstrap starting.
I20250109 08:58:24.259431 23580 tablet_bootstrap.cc:654] T ecc833ad1afa4079a75e41b1a807c923 P ed1efbce9ee3401e99a5260edeae4e7c: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:24.255944 23583 raft_consensus.cc:513] T de98c70e25494d2a8e581443be836bd3 P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:24.281494 23201 catalog_manager.cc:5526] T 732e4aa69aae44e9b1153ac330a01a96 P ed1efbce9ee3401e99a5260edeae4e7c reported cstate change: term changed from 0 to 1, leader changed from <none> to ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194). New cstate: current_term: 1 leader_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } health_report { overall_health: UNKNOWN } } }
W20250109 08:58:24.304682 23269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.226.196:41829, user_credentials={real_user=slave}} blocked reactor thread for 64294.3us
W20250109 08:58:24.317642 23420 leader_election.cc:343] T de98c70e25494d2a8e581443be836bd3 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:24.319106 23675 consensus_queue.cc:1035] T e5251927614642639bd46ecd5373d2a4 P cf55383bef6649eb9e78cb2d2512dedc [LEADER]: Connected to new peer: Peer: permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:58:24.335235 23569 tablet_bootstrap.cc:492] T 5c37c58abaa54656b31c72160b4dec9e P 05a74e1eb2474b3d8b3a2da1ef127e01: No bootstrap required, opened a new log
I20250109 08:58:24.335611 23580 tablet_bootstrap.cc:492] T ecc833ad1afa4079a75e41b1a807c923 P ed1efbce9ee3401e99a5260edeae4e7c: No bootstrap required, opened a new log
I20250109 08:58:24.335775 23569 ts_tablet_manager.cc:1397] T 5c37c58abaa54656b31c72160b4dec9e P 05a74e1eb2474b3d8b3a2da1ef127e01: Time spent bootstrapping tablet: real 0.138s user 0.012s sys 0.023s
I20250109 08:58:24.336311 23580 ts_tablet_manager.cc:1397] T ecc833ad1afa4079a75e41b1a807c923 P ed1efbce9ee3401e99a5260edeae4e7c: Time spent bootstrapping tablet: real 0.083s user 0.017s sys 0.030s
I20250109 08:58:24.344388 23574 tablet_bootstrap.cc:492] T f1f9274aeda748f493f5752814fb262d P 235ad6af04c34c88a5261daa3604fada: No bootstrap required, opened a new log
I20250109 08:58:24.345113 23574 ts_tablet_manager.cc:1397] T f1f9274aeda748f493f5752814fb262d P 235ad6af04c34c88a5261daa3604fada: Time spent bootstrapping tablet: real 0.102s user 0.029s sys 0.031s
I20250109 08:58:24.349612 23569 raft_consensus.cc:357] T 5c37c58abaa54656b31c72160b4dec9e P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:24.352991 23569 raft_consensus.cc:383] T 5c37c58abaa54656b31c72160b4dec9e P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:24.353315 23569 raft_consensus.cc:738] T 5c37c58abaa54656b31c72160b4dec9e P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05a74e1eb2474b3d8b3a2da1ef127e01, State: Initialized, Role: FOLLOWER
I20250109 08:58:24.357762 23569 consensus_queue.cc:260] T 5c37c58abaa54656b31c72160b4dec9e P 05a74e1eb2474b3d8b3a2da1ef127e01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:24.366990 23580 raft_consensus.cc:357] T ecc833ad1afa4079a75e41b1a807c923 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:24.367942 23569 ts_tablet_manager.cc:1428] T 5c37c58abaa54656b31c72160b4dec9e P 05a74e1eb2474b3d8b3a2da1ef127e01: Time spent starting tablet: real 0.032s user 0.004s sys 0.004s
I20250109 08:58:24.367964 23580 raft_consensus.cc:383] T ecc833ad1afa4079a75e41b1a807c923 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:24.368506 23580 raft_consensus.cc:738] T ecc833ad1afa4079a75e41b1a807c923 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed1efbce9ee3401e99a5260edeae4e7c, State: Initialized, Role: FOLLOWER
I20250109 08:58:24.369170 23580 consensus_queue.cc:260] T ecc833ad1afa4079a75e41b1a807c923 P ed1efbce9ee3401e99a5260edeae4e7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:24.374614 23569 tablet_bootstrap.cc:492] T 444507f7812441ceaeebc44bc8f06293 P 05a74e1eb2474b3d8b3a2da1ef127e01: Bootstrap starting.
W20250109 08:58:24.359277 23344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.226.196:41829, user_credentials={real_user=slave}} blocked reactor thread for 49899.1us
I20250109 08:58:24.372015 23201 catalog_manager.cc:5526] T 94f2be307f704c75bceb213e52307cfb P 235ad6af04c34c88a5261daa3604fada reported cstate change: term changed from 0 to 1, leader changed from <none> to 235ad6af04c34c88a5261daa3604fada (127.19.226.193). New cstate: current_term: 1 leader_uuid: "235ad6af04c34c88a5261daa3604fada" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } health_report { overall_health: HEALTHY } } }
I20250109 08:58:24.373649 23583 leader_election.cc:290] T de98c70e25494d2a8e581443be836bd3 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 election: Requested vote from peers 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691), ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365)
I20250109 08:58:24.373029 23666 raft_consensus.cc:513] T db111e41c1be497e8e673a2c42b9ba66 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:24.433523 23378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de98c70e25494d2a8e581443be836bd3" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c"
I20250109 08:58:24.373775 23675 raft_consensus.cc:491] T 11aa76a3a30643708fececed660a5250 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:24.436719 23766 raft_consensus.cc:491] T 5c37c58abaa54656b31c72160b4dec9e P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:24.441032 23766 raft_consensus.cc:513] T 5c37c58abaa54656b31c72160b4dec9e P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:24.442922 23766 leader_election.cc:290] T 5c37c58abaa54656b31c72160b4dec9e P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551), 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691)
I20250109 08:58:24.443481 23378 raft_consensus.cc:3054] T de98c70e25494d2a8e581443be836bd3 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:24.447746 23574 raft_consensus.cc:357] T f1f9274aeda748f493f5752814fb262d P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:24.448709 23574 raft_consensus.cc:383] T f1f9274aeda748f493f5752814fb262d P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:24.449038 23574 raft_consensus.cc:738] T f1f9274aeda748f493f5752814fb262d P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Initialized, Role: FOLLOWER
I20250109 08:58:24.449859 23574 consensus_queue.cc:260] T f1f9274aeda748f493f5752814fb262d P 235ad6af04c34c88a5261daa3604fada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:24.463445 23378 raft_consensus.cc:2463] T de98c70e25494d2a8e581443be836bd3 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf55383bef6649eb9e78cb2d2512dedc in term 1.
I20250109 08:58:24.465495 23666 leader_election.cc:290] T db111e41c1be497e8e673a2c42b9ba66 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 election: Requested vote from peers ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365), cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551)
I20250109 08:58:24.467830 23572 tablet_bootstrap.cc:492] T 1365130ef97c4581b623c22c39753831 P cf55383bef6649eb9e78cb2d2512dedc: No bootstrap required, opened a new log
I20250109 08:58:24.478684 23572 ts_tablet_manager.cc:1397] T 1365130ef97c4581b623c22c39753831 P cf55383bef6649eb9e78cb2d2512dedc: Time spent bootstrapping tablet: real 0.322s user 0.025s sys 0.013s
I20250109 08:58:24.474730 23451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c37c58abaa54656b31c72160b4dec9e" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc" is_pre_election: true
I20250109 08:58:24.476109 23305 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c37c58abaa54656b31c72160b4dec9e" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada" is_pre_election: true
I20250109 08:58:24.476768 23816 consensus_queue.cc:1035] T 86b82e19759c4fd0b471a0dbff839aa5 P ed1efbce9ee3401e99a5260edeae4e7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:58:24.444135 23675 raft_consensus.cc:513] T 11aa76a3a30643708fececed660a5250 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:24.485409 23572 raft_consensus.cc:357] T 1365130ef97c4581b623c22c39753831 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:24.486235 23572 raft_consensus.cc:383] T 1365130ef97c4581b623c22c39753831 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:24.486536 23572 raft_consensus.cc:738] T 1365130ef97c4581b623c22c39753831 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf55383bef6649eb9e78cb2d2512dedc, State: Initialized, Role: FOLLOWER
I20250109 08:58:24.487459 23675 leader_election.cc:290] T 11aa76a3a30643708fececed660a5250 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365), 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829)
I20250109 08:58:24.488591 23377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11aa76a3a30643708fececed660a5250" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" is_pre_election: true
I20250109 08:58:24.488830 23572 consensus_queue.cc:260] T 1365130ef97c4581b623c22c39753831 P cf55383bef6649eb9e78cb2d2512dedc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
W20250109 08:58:24.490033 23420 leader_election.cc:343] T 11aa76a3a30643708fececed660a5250 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:24.491155 23572 ts_tablet_manager.cc:1428] T 1365130ef97c4581b623c22c39753831 P cf55383bef6649eb9e78cb2d2512dedc: Time spent starting tablet: real 0.012s user 0.003s sys 0.002s
I20250109 08:58:24.492070 23305 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de98c70e25494d2a8e581443be836bd3" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada"
I20250109 08:58:24.496544 23627 consensus_queue.cc:1035] T e5251927614642639bd46ecd5373d2a4 P cf55383bef6649eb9e78cb2d2512dedc [LEADER]: Connected to new peer: Peer: permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:58:24.499468 23524 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11aa76a3a30643708fececed660a5250" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" is_pre_election: true
W20250109 08:58:24.502367 23418 consensus_peers.cc:487] T a7d2ab61d238471c9f287ac1d5db4ed0 P cf55383bef6649eb9e78cb2d2512dedc -> Peer 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829): Couldn't send request to peer 05a74e1eb2474b3d8b3a2da1ef127e01. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20250109 08:58:24.503000 23418 leader_election.cc:343] T 11aa76a3a30643708fececed660a5250 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:24.503398 23418 leader_election.cc:304] T 11aa76a3a30643708fececed660a5250 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf55383bef6649eb9e78cb2d2512dedc; no voters: 05a74e1eb2474b3d8b3a2da1ef127e01, ed1efbce9ee3401e99a5260edeae4e7c
I20250109 08:58:24.504709 23494 leader_election.cc:304] T 5c37c58abaa54656b31c72160b4dec9e P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01; no voters: 235ad6af04c34c88a5261daa3604fada, cf55383bef6649eb9e78cb2d2512dedc
I20250109 08:58:24.504865 23677 raft_consensus.cc:2743] T 11aa76a3a30643708fececed660a5250 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:58:24.505478 23766 raft_consensus.cc:2743] T 5c37c58abaa54656b31c72160b4dec9e P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:58:24.507455 23572 tablet_bootstrap.cc:492] T f1f9274aeda748f493f5752814fb262d P cf55383bef6649eb9e78cb2d2512dedc: Bootstrap starting.
W20250109 08:58:24.512089 23420 leader_election.cc:343] T de98c70e25494d2a8e581443be836bd3 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:24.512151 23380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db111e41c1be497e8e673a2c42b9ba66" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c"
I20250109 08:58:24.513413 23380 raft_consensus.cc:3054] T db111e41c1be497e8e673a2c42b9ba66 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:24.479334 23580 ts_tablet_manager.cc:1428] T ecc833ad1afa4079a75e41b1a807c923 P ed1efbce9ee3401e99a5260edeae4e7c: Time spent starting tablet: real 0.143s user 0.006s sys 0.000s
I20250109 08:58:24.520012 23574 ts_tablet_manager.cc:1428] T f1f9274aeda748f493f5752814fb262d P 235ad6af04c34c88a5261daa3604fada: Time spent starting tablet: real 0.174s user 0.006s sys 0.000s
I20250109 08:58:24.521063 23574 tablet_bootstrap.cc:492] T fc1038255dd64cb5ba2425037d22f1f7 P 235ad6af04c34c88a5261daa3604fada: Bootstrap starting.
I20250109 08:58:24.521551 23420 leader_election.cc:304] T de98c70e25494d2a8e581443be836bd3 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf55383bef6649eb9e78cb2d2512dedc, ed1efbce9ee3401e99a5260edeae4e7c; no voters: 235ad6af04c34c88a5261daa3604fada
I20250109 08:58:24.522814 23380 raft_consensus.cc:2463] T db111e41c1be497e8e673a2c42b9ba66 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 235ad6af04c34c88a5261daa3604fada in term 1.
I20250109 08:58:24.526893 23451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db111e41c1be497e8e673a2c42b9ba66" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc"
I20250109 08:58:24.527593 23451 raft_consensus.cc:3054] T db111e41c1be497e8e673a2c42b9ba66 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Advancing to term 1
W20250109 08:58:24.536357 23251 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20250109 08:58:24.536670 23675 raft_consensus.cc:2798] T de98c70e25494d2a8e581443be836bd3 P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:58:24.539537 23675 raft_consensus.cc:695] T de98c70e25494d2a8e581443be836bd3 P cf55383bef6649eb9e78cb2d2512dedc [term 1 LEADER]: Becoming Leader. State: Replica: cf55383bef6649eb9e78cb2d2512dedc, State: Running, Role: LEADER
I20250109 08:58:24.540338 23675 consensus_queue.cc:237] T de98c70e25494d2a8e581443be836bd3 P cf55383bef6649eb9e78cb2d2512dedc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:24.544731 23271 leader_election.cc:304] T db111e41c1be497e8e673a2c42b9ba66 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 235ad6af04c34c88a5261daa3604fada, ed1efbce9ee3401e99a5260edeae4e7c; no voters:
I20250109 08:58:24.537349 23580 tablet_bootstrap.cc:492] T 1f9c6525267145d6bb73dde32ed9b7d4 P ed1efbce9ee3401e99a5260edeae4e7c: Bootstrap starting.
I20250109 08:58:24.545682 23873 raft_consensus.cc:2798] T db111e41c1be497e8e673a2c42b9ba66 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:58:24.546375 23873 raft_consensus.cc:695] T db111e41c1be497e8e673a2c42b9ba66 P 235ad6af04c34c88a5261daa3604fada [term 1 LEADER]: Becoming Leader. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Running, Role: LEADER
I20250109 08:58:24.547607 23569 tablet_bootstrap.cc:654] T 444507f7812441ceaeebc44bc8f06293 P 05a74e1eb2474b3d8b3a2da1ef127e01: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:24.547232 23873 consensus_queue.cc:237] T db111e41c1be497e8e673a2c42b9ba66 P 235ad6af04c34c88a5261daa3604fada [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:24.549588 23451 raft_consensus.cc:2463] T db111e41c1be497e8e673a2c42b9ba66 P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 235ad6af04c34c88a5261daa3604fada in term 1.
I20250109 08:58:24.564169 23574 tablet_bootstrap.cc:654] T fc1038255dd64cb5ba2425037d22f1f7 P 235ad6af04c34c88a5261daa3604fada: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:24.567513 23580 tablet_bootstrap.cc:654] T 1f9c6525267145d6bb73dde32ed9b7d4 P ed1efbce9ee3401e99a5260edeae4e7c: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:24.569413 23528 raft_consensus.cc:3054] T 5c37c58abaa54656b31c72160b4dec9e P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:24.572057 23201 catalog_manager.cc:5526] T de98c70e25494d2a8e581443be836bd3 P cf55383bef6649eb9e78cb2d2512dedc reported cstate change: term changed from 0 to 1, leader changed from <none> to cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195). New cstate: current_term: 1 leader_uuid: "cf55383bef6649eb9e78cb2d2512dedc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } health_report { overall_health: HEALTHY } } }
I20250109 08:58:24.581822 23572 tablet_bootstrap.cc:654] T f1f9274aeda748f493f5752814fb262d P cf55383bef6649eb9e78cb2d2512dedc: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:24.601148 23569 tablet_bootstrap.cc:492] T 444507f7812441ceaeebc44bc8f06293 P 05a74e1eb2474b3d8b3a2da1ef127e01: No bootstrap required, opened a new log
I20250109 08:58:24.601591 23569 ts_tablet_manager.cc:1397] T 444507f7812441ceaeebc44bc8f06293 P 05a74e1eb2474b3d8b3a2da1ef127e01: Time spent bootstrapping tablet: real 0.227s user 0.014s sys 0.002s
I20250109 08:58:24.604507 23305 raft_consensus.cc:3054] T 86b82e19759c4fd0b471a0dbff839aa5 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:24.612200 23569 raft_consensus.cc:357] T 444507f7812441ceaeebc44bc8f06293 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:24.615439 23569 raft_consensus.cc:383] T 444507f7812441ceaeebc44bc8f06293 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:24.617674 23873 raft_consensus.cc:491] T 86b82e19759c4fd0b471a0dbff839aa5 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:24.618166 23873 raft_consensus.cc:513] T 86b82e19759c4fd0b471a0dbff839aa5 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:24.615716 23569 raft_consensus.cc:738] T 444507f7812441ceaeebc44bc8f06293 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05a74e1eb2474b3d8b3a2da1ef127e01, State: Initialized, Role: FOLLOWER
I20250109 08:58:24.620967 23569 consensus_queue.cc:260] T 444507f7812441ceaeebc44bc8f06293 P 05a74e1eb2474b3d8b3a2da1ef127e01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:24.624238 23766 consensus_queue.cc:1035] T 5f0d500a91b54851bda52e5ccf741fd6 P 05a74e1eb2474b3d8b3a2da1ef127e01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:58:24.627312 23569 ts_tablet_manager.cc:1428] T 444507f7812441ceaeebc44bc8f06293 P 05a74e1eb2474b3d8b3a2da1ef127e01: Time spent starting tablet: real 0.026s user 0.006s sys 0.000s
I20250109 08:58:24.628286 23569 tablet_bootstrap.cc:492] T ecc833ad1afa4079a75e41b1a807c923 P 05a74e1eb2474b3d8b3a2da1ef127e01: Bootstrap starting.
I20250109 08:58:24.631708 23878 consensus_queue.cc:1035] T 86b82e19759c4fd0b471a0dbff839aa5 P ed1efbce9ee3401e99a5260edeae4e7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:58:24.637970 23574 tablet_bootstrap.cc:492] T fc1038255dd64cb5ba2425037d22f1f7 P 235ad6af04c34c88a5261daa3604fada: No bootstrap required, opened a new log
I20250109 08:58:24.638468 23574 ts_tablet_manager.cc:1397] T fc1038255dd64cb5ba2425037d22f1f7 P 235ad6af04c34c88a5261daa3604fada: Time spent bootstrapping tablet: real 0.118s user 0.016s sys 0.004s
W20250109 08:58:24.642608 23494 consensus_peers.cc:487] T 5f0d500a91b54851bda52e5ccf741fd6 P 05a74e1eb2474b3d8b3a2da1ef127e01 -> Peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Couldn't send request to peer 235ad6af04c34c88a5261daa3604fada. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:24.645303 23677 consensus_queue.cc:1035] T a7d2ab61d238471c9f287ac1d5db4ed0 P cf55383bef6649eb9e78cb2d2512dedc [LEADER]: Connected to new peer: Peer: permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:58:24.645954 23873 leader_election.cc:290] T 86b82e19759c4fd0b471a0dbff839aa5 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829), ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365)
I20250109 08:58:24.647614 23380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86b82e19759c4fd0b471a0dbff839aa5" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" is_pre_election: true
I20250109 08:58:24.651437 23203 catalog_manager.cc:5526] T db111e41c1be497e8e673a2c42b9ba66 P 235ad6af04c34c88a5261daa3604fada reported cstate change: term changed from 0 to 1, leader changed from <none> to 235ad6af04c34c88a5261daa3604fada (127.19.226.193). New cstate: current_term: 1 leader_uuid: "235ad6af04c34c88a5261daa3604fada" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:24.670126 23580 tablet_bootstrap.cc:492] T 1f9c6525267145d6bb73dde32ed9b7d4 P ed1efbce9ee3401e99a5260edeae4e7c: No bootstrap required, opened a new log
I20250109 08:58:24.670686 23580 ts_tablet_manager.cc:1397] T 1f9c6525267145d6bb73dde32ed9b7d4 P ed1efbce9ee3401e99a5260edeae4e7c: Time spent bootstrapping tablet: real 0.134s user 0.018s sys 0.014s
I20250109 08:58:24.661367 23766 raft_consensus.cc:491] T 0dab6c23ce444dda9427901a98f07b15 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:24.673038 23527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86b82e19759c4fd0b471a0dbff839aa5" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" is_pre_election: true
I20250109 08:58:24.686574 23580 raft_consensus.cc:357] T 1f9c6525267145d6bb73dde32ed9b7d4 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:24.689642 23580 raft_consensus.cc:383] T 1f9c6525267145d6bb73dde32ed9b7d4 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:24.689886 23580 raft_consensus.cc:738] T 1f9c6525267145d6bb73dde32ed9b7d4 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed1efbce9ee3401e99a5260edeae4e7c, State: Initialized, Role: FOLLOWER
I20250109 08:58:24.690438 23580 consensus_queue.cc:260] T 1f9c6525267145d6bb73dde32ed9b7d4 P ed1efbce9ee3401e99a5260edeae4e7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:24.676461 23766 raft_consensus.cc:513] T 0dab6c23ce444dda9427901a98f07b15 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:24.694476 23580 ts_tablet_manager.cc:1428] T 1f9c6525267145d6bb73dde32ed9b7d4 P ed1efbce9ee3401e99a5260edeae4e7c: Time spent starting tablet: real 0.024s user 0.001s sys 0.005s
I20250109 08:58:24.687649 23878 raft_consensus.cc:491] T ecc833ad1afa4079a75e41b1a807c923 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:24.697873 23580 tablet_bootstrap.cc:492] T d714271c7ea64c0b8b13fcf63591ec07 P ed1efbce9ee3401e99a5260edeae4e7c: Bootstrap starting.
I20250109 08:58:24.698300 23574 raft_consensus.cc:357] T fc1038255dd64cb5ba2425037d22f1f7 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:24.699107 23574 raft_consensus.cc:383] T fc1038255dd64cb5ba2425037d22f1f7 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:24.699564 23574 raft_consensus.cc:738] T fc1038255dd64cb5ba2425037d22f1f7 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Initialized, Role: FOLLOWER
I20250109 08:58:24.700354 23574 consensus_queue.cc:260] T fc1038255dd64cb5ba2425037d22f1f7 P 235ad6af04c34c88a5261daa3604fada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:24.701418 23766 leader_election.cc:290] T 0dab6c23ce444dda9427901a98f07b15 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365), cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551)
I20250109 08:58:24.706192 23269 leader_election.cc:304] T 86b82e19759c4fd0b471a0dbff839aa5 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 235ad6af04c34c88a5261daa3604fada; no voters: 05a74e1eb2474b3d8b3a2da1ef127e01, ed1efbce9ee3401e99a5260edeae4e7c
I20250109 08:58:24.707073 23873 raft_consensus.cc:2743] T 86b82e19759c4fd0b471a0dbff839aa5 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250109 08:58:24.708103 23380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0dab6c23ce444dda9427901a98f07b15" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" is_pre_election: true
W20250109 08:58:24.711503 23494 leader_election.cc:343] T 0dab6c23ce444dda9427901a98f07b15 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:24.712244 23580 tablet_bootstrap.cc:654] T d714271c7ea64c0b8b13fcf63591ec07 P ed1efbce9ee3401e99a5260edeae4e7c: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:24.719748 23450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0dab6c23ce444dda9427901a98f07b15" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc" is_pre_election: true
I20250109 08:58:24.720544 23450 raft_consensus.cc:2463] T 0dab6c23ce444dda9427901a98f07b15 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05a74e1eb2474b3d8b3a2da1ef127e01 in term 0.
I20250109 08:58:24.722003 23494 leader_election.cc:304] T 0dab6c23ce444dda9427901a98f07b15 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01, cf55383bef6649eb9e78cb2d2512dedc; no voters: ed1efbce9ee3401e99a5260edeae4e7c
I20250109 08:58:24.723866 23574 ts_tablet_manager.cc:1428] T fc1038255dd64cb5ba2425037d22f1f7 P 235ad6af04c34c88a5261daa3604fada: Time spent starting tablet: real 0.085s user 0.005s sys 0.001s
I20250109 08:58:24.724516 23634 raft_consensus.cc:2798] T 0dab6c23ce444dda9427901a98f07b15 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:58:24.724923 23634 raft_consensus.cc:491] T 0dab6c23ce444dda9427901a98f07b15 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:58:24.727319 23574 tablet_bootstrap.cc:492] T de98c70e25494d2a8e581443be836bd3 P 235ad6af04c34c88a5261daa3604fada: Bootstrap starting.
I20250109 08:58:24.726106 23569 tablet_bootstrap.cc:654] T ecc833ad1afa4079a75e41b1a807c923 P 05a74e1eb2474b3d8b3a2da1ef127e01: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:24.698266 23878 raft_consensus.cc:513] T ecc833ad1afa4079a75e41b1a807c923 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:24.726881 23634 raft_consensus.cc:3054] T 0dab6c23ce444dda9427901a98f07b15 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:24.741781 23572 tablet_bootstrap.cc:492] T f1f9274aeda748f493f5752814fb262d P cf55383bef6649eb9e78cb2d2512dedc: No bootstrap required, opened a new log
I20250109 08:58:24.742290 23572 ts_tablet_manager.cc:1397] T f1f9274aeda748f493f5752814fb262d P cf55383bef6649eb9e78cb2d2512dedc: Time spent bootstrapping tablet: real 0.235s user 0.012s sys 0.017s
W20250109 08:58:24.742832 23887 log.cc:926] Time spent T 5c37c58abaa54656b31c72160b4dec9e P 05a74e1eb2474b3d8b3a2da1ef127e01: Append to log took a long time: real 0.052s user 0.001s sys 0.000s
I20250109 08:58:24.744669 23766 raft_consensus.cc:491] T cc9886333c0343798226d59758a1ea5e P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:24.745162 23766 raft_consensus.cc:513] T cc9886333c0343798226d59758a1ea5e P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:24.746871 23878 leader_election.cc:290] T ecc833ad1afa4079a75e41b1a807c923 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551), 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829)
I20250109 08:58:24.747033 23766 leader_election.cc:290] T cc9886333c0343798226d59758a1ea5e P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551), 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691)
I20250109 08:58:24.749017 23572 raft_consensus.cc:357] T f1f9274aeda748f493f5752814fb262d P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:24.749790 23572 raft_consensus.cc:383] T f1f9274aeda748f493f5752814fb262d P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:24.750046 23572 raft_consensus.cc:738] T f1f9274aeda748f493f5752814fb262d P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf55383bef6649eb9e78cb2d2512dedc, State: Initialized, Role: FOLLOWER
I20250109 08:58:24.750656 23572 consensus_queue.cc:260] T f1f9274aeda748f493f5752814fb262d P cf55383bef6649eb9e78cb2d2512dedc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:24.756528 23574 tablet_bootstrap.cc:654] T de98c70e25494d2a8e581443be836bd3 P 235ad6af04c34c88a5261daa3604fada: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:24.756780 23528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ecc833ad1afa4079a75e41b1a807c923" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" is_pre_election: true
I20250109 08:58:24.757601 23452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc9886333c0343798226d59758a1ea5e" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc" is_pre_election: true
I20250109 08:58:24.758813 23303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc9886333c0343798226d59758a1ea5e" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada" is_pre_election: true
I20250109 08:58:24.759466 23303 raft_consensus.cc:2463] T cc9886333c0343798226d59758a1ea5e P 235ad6af04c34c88a5261daa3604fada [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05a74e1eb2474b3d8b3a2da1ef127e01 in term 2.
W20250109 08:58:24.760736 23344 leader_election.cc:343] T ecc833ad1afa4079a75e41b1a807c923 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:24.761983 23494 leader_election.cc:304] T cc9886333c0343798226d59758a1ea5e P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01, 235ad6af04c34c88a5261daa3604fada; no voters: cf55383bef6649eb9e78cb2d2512dedc
I20250109 08:58:24.762986 23766 raft_consensus.cc:2798] T cc9886333c0343798226d59758a1ea5e P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20250109 08:58:24.763324 23766 raft_consensus.cc:491] T cc9886333c0343798226d59758a1ea5e P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:58:24.763625 23766 raft_consensus.cc:3054] T cc9886333c0343798226d59758a1ea5e P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 FOLLOWER]: Advancing to term 3
I20250109 08:58:24.764317 23452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ecc833ad1afa4079a75e41b1a807c923" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc" is_pre_election: true
I20250109 08:58:24.767163 23873 consensus_queue.cc:1035] T 94f2be307f704c75bceb213e52307cfb P 235ad6af04c34c88a5261daa3604fada [LEADER]: Connected to new peer: Peer: permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20250109 08:58:24.769291 23346 leader_election.cc:343] T ecc833ad1afa4079a75e41b1a807c923 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:24.769770 23346 leader_election.cc:304] T ecc833ad1afa4079a75e41b1a807c923 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ed1efbce9ee3401e99a5260edeae4e7c; no voters: 05a74e1eb2474b3d8b3a2da1ef127e01, cf55383bef6649eb9e78cb2d2512dedc
I20250109 08:58:24.770511 23658 raft_consensus.cc:2743] T ecc833ad1afa4079a75e41b1a807c923 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:58:24.779923 23878 consensus_queue.cc:1035] T 732e4aa69aae44e9b1153ac330a01a96 P ed1efbce9ee3401e99a5260edeae4e7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250109 08:58:24.781633 23877 consensus_queue.cc:1035] T 732e4aa69aae44e9b1153ac330a01a96 P ed1efbce9ee3401e99a5260edeae4e7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:58:24.782631 23580 tablet_bootstrap.cc:492] T d714271c7ea64c0b8b13fcf63591ec07 P ed1efbce9ee3401e99a5260edeae4e7c: No bootstrap required, opened a new log
I20250109 08:58:24.783164 23580 ts_tablet_manager.cc:1397] T d714271c7ea64c0b8b13fcf63591ec07 P ed1efbce9ee3401e99a5260edeae4e7c: Time spent bootstrapping tablet: real 0.086s user 0.026s sys 0.001s
I20250109 08:58:24.775231 23634 raft_consensus.cc:513] T 0dab6c23ce444dda9427901a98f07b15 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:24.785437 23572 ts_tablet_manager.cc:1428] T f1f9274aeda748f493f5752814fb262d P cf55383bef6649eb9e78cb2d2512dedc: Time spent starting tablet: real 0.043s user 0.006s sys 0.002s
I20250109 08:58:24.788880 23572 tablet_bootstrap.cc:492] T 750dd1e1055f4570b0aecdda3ebe8772 P cf55383bef6649eb9e78cb2d2512dedc: Bootstrap starting.
I20250109 08:58:24.796945 23766 raft_consensus.cc:513] T cc9886333c0343798226d59758a1ea5e P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:24.798950 23766 leader_election.cc:290] T cc9886333c0343798226d59758a1ea5e P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 3 election: Requested vote from peers cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551), 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691)
I20250109 08:58:24.800346 23303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc9886333c0343798226d59758a1ea5e" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada"
I20250109 08:58:24.801123 23303 raft_consensus.cc:3054] T cc9886333c0343798226d59758a1ea5e P 235ad6af04c34c88a5261daa3604fada [term 2 FOLLOWER]: Advancing to term 3
I20250109 08:58:24.809173 23451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc9886333c0343798226d59758a1ea5e" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc"
I20250109 08:58:24.813977 23580 raft_consensus.cc:357] T d714271c7ea64c0b8b13fcf63591ec07 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:24.818079 23580 raft_consensus.cc:383] T d714271c7ea64c0b8b13fcf63591ec07 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:24.818372 23580 raft_consensus.cc:738] T d714271c7ea64c0b8b13fcf63591ec07 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed1efbce9ee3401e99a5260edeae4e7c, State: Initialized, Role: FOLLOWER
I20250109 08:58:24.819799 23580 consensus_queue.cc:260] T d714271c7ea64c0b8b13fcf63591ec07 P ed1efbce9ee3401e99a5260edeae4e7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
W20250109 08:58:24.818919 23420 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.226.193:42691, user_credentials={real_user=slave}} blocked reactor thread for 65592.6us
I20250109 08:58:24.833029 23675 raft_consensus.cc:491] T cc9886333c0343798226d59758a1ea5e P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 05a74e1eb2474b3d8b3a2da1ef127e01)
I20250109 08:58:24.833560 23675 raft_consensus.cc:513] T cc9886333c0343798226d59758a1ea5e P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:24.835173 23303 raft_consensus.cc:2463] T cc9886333c0343798226d59758a1ea5e P 235ad6af04c34c88a5261daa3604fada [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05a74e1eb2474b3d8b3a2da1ef127e01 in term 3.
I20250109 08:58:24.836405 23494 leader_election.cc:304] T cc9886333c0343798226d59758a1ea5e P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01, 235ad6af04c34c88a5261daa3604fada; no voters: cf55383bef6649eb9e78cb2d2512dedc
I20250109 08:58:24.836979 23580 ts_tablet_manager.cc:1428] T d714271c7ea64c0b8b13fcf63591ec07 P ed1efbce9ee3401e99a5260edeae4e7c: Time spent starting tablet: real 0.053s user 0.006s sys 0.000s
I20250109 08:58:24.837108 23675 leader_election.cc:290] T cc9886333c0343798226d59758a1ea5e P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829), 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691)
I20250109 08:58:24.833300 23767 raft_consensus.cc:491] T f1f9274aeda748f493f5752814fb262d P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:24.838086 23767 raft_consensus.cc:513] T f1f9274aeda748f493f5752814fb262d P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:24.838689 23572 tablet_bootstrap.cc:654] T 750dd1e1055f4570b0aecdda3ebe8772 P cf55383bef6649eb9e78cb2d2512dedc: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:24.839097 23766 raft_consensus.cc:2798] T cc9886333c0343798226d59758a1ea5e P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 3 FOLLOWER]: Leader election won for term 3
I20250109 08:58:24.839601 23766 raft_consensus.cc:695] T cc9886333c0343798226d59758a1ea5e P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 3 LEADER]: Becoming Leader. State: Replica: 05a74e1eb2474b3d8b3a2da1ef127e01, State: Running, Role: LEADER
I20250109 08:58:24.839973 23767 leader_election.cc:290] T f1f9274aeda748f493f5752814fb262d P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365), cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551)
I20250109 08:58:24.841867 23580 tablet_bootstrap.cc:492] T 1365130ef97c4581b623c22c39753831 P ed1efbce9ee3401e99a5260edeae4e7c: Bootstrap starting.
I20250109 08:58:24.840740 23528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc9886333c0343798226d59758a1ea5e" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" is_pre_election: true
I20250109 08:58:24.840409 23766 consensus_queue.cc:237] T cc9886333c0343798226d59758a1ea5e P 05a74e1eb2474b3d8b3a2da1ef127e01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:24.858000 23634 leader_election.cc:290] T 0dab6c23ce444dda9427901a98f07b15 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 election: Requested vote from peers ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365), cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551)
I20250109 08:58:24.864100 23380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0dab6c23ce444dda9427901a98f07b15" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c"
I20250109 08:58:24.866160 23452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0dab6c23ce444dda9427901a98f07b15" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc"
W20250109 08:58:24.870476 23494 leader_election.cc:343] T 0dab6c23ce444dda9427901a98f07b15 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:24.872866 23380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1f9274aeda748f493f5752814fb262d" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" is_pre_election: true
I20250109 08:58:24.873651 23380 raft_consensus.cc:2463] T f1f9274aeda748f493f5752814fb262d P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 235ad6af04c34c88a5261daa3604fada in term 0.
I20250109 08:58:24.876057 23271 leader_election.cc:304] T f1f9274aeda748f493f5752814fb262d P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 235ad6af04c34c88a5261daa3604fada, ed1efbce9ee3401e99a5260edeae4e7c; no voters:
I20250109 08:58:24.876951 23586 raft_consensus.cc:2798] T f1f9274aeda748f493f5752814fb262d P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:58:24.877312 23586 raft_consensus.cc:491] T f1f9274aeda748f493f5752814fb262d P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:58:24.877642 23586 raft_consensus.cc:3054] T f1f9274aeda748f493f5752814fb262d P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:24.877894 23451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1f9274aeda748f493f5752814fb262d" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc" is_pre_election: true
I20250109 08:58:24.878618 23451 raft_consensus.cc:2463] T f1f9274aeda748f493f5752814fb262d P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 235ad6af04c34c88a5261daa3604fada in term 0.
I20250109 08:58:24.880437 23305 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc9886333c0343798226d59758a1ea5e" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada" is_pre_election: true
I20250109 08:58:24.881153 23305 raft_consensus.cc:2363] T cc9886333c0343798226d59758a1ea5e P 235ad6af04c34c88a5261daa3604fada [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf55383bef6649eb9e78cb2d2512dedc for earlier term 2. Current term is 3.
I20250109 08:58:24.888618 23420 leader_election.cc:400] T cc9886333c0343798226d59758a1ea5e P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 2 pre-election: Vote denied by peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691) with higher term. Message: Invalid argument: T cc9886333c0343798226d59758a1ea5e P 235ad6af04c34c88a5261daa3604fada [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf55383bef6649eb9e78cb2d2512dedc for earlier term 2. Current term is 3.
I20250109 08:58:24.889376 23894 raft_consensus.cc:491] T 1365130ef97c4581b623c22c39753831 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:24.889881 23894 raft_consensus.cc:513] T 1365130ef97c4581b623c22c39753831 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:24.888394 23452 raft_consensus.cc:3054] T 0dab6c23ce444dda9427901a98f07b15 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:24.891938 23894 leader_election.cc:290] T 1365130ef97c4581b623c22c39753831 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691), ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365)
I20250109 08:58:24.889024 23420 leader_election.cc:403] T cc9886333c0343798226d59758a1ea5e P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20250109 08:58:24.897713 23894 raft_consensus.cc:3054] T cc9886333c0343798226d59758a1ea5e P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Advancing to term 3
I20250109 08:58:24.914140 23380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1365130ef97c4581b623c22c39753831" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" is_pre_election: true
W20250109 08:58:24.915827 23420 leader_election.cc:343] T 1365130ef97c4581b623c22c39753831 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:24.919003 23305 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1365130ef97c4581b623c22c39753831" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada" is_pre_election: true
W20250109 08:58:24.920958 23420 leader_election.cc:343] T 1365130ef97c4581b623c22c39753831 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:24.921416 23420 leader_election.cc:304] T 1365130ef97c4581b623c22c39753831 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf55383bef6649eb9e78cb2d2512dedc; no voters: 235ad6af04c34c88a5261daa3604fada, ed1efbce9ee3401e99a5260edeae4e7c
I20250109 08:58:24.922176 23583 raft_consensus.cc:2743] T 1365130ef97c4581b623c22c39753831 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:58:24.924583 23580 tablet_bootstrap.cc:654] T 1365130ef97c4581b623c22c39753831 P ed1efbce9ee3401e99a5260edeae4e7c: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:24.931077 23569 tablet_bootstrap.cc:492] T ecc833ad1afa4079a75e41b1a807c923 P 05a74e1eb2474b3d8b3a2da1ef127e01: No bootstrap required, opened a new log
I20250109 08:58:24.925755 23586 raft_consensus.cc:513] T f1f9274aeda748f493f5752814fb262d P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
W20250109 08:58:24.931283 23271 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.226.194:41365, user_credentials={real_user=slave}} blocked reactor thread for 55308.8us
I20250109 08:58:24.925513 23894 raft_consensus.cc:2743] T cc9886333c0343798226d59758a1ea5e P cf55383bef6649eb9e78cb2d2512dedc [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691) with higher term. Message: Invalid argument: T cc9886333c0343798226d59758a1ea5e P 235ad6af04c34c88a5261daa3604fada [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf55383bef6649eb9e78cb2d2512dedc for earlier term 2. Current term is 3.
I20250109 08:58:24.931561 23569 ts_tablet_manager.cc:1397] T ecc833ad1afa4079a75e41b1a807c923 P 05a74e1eb2474b3d8b3a2da1ef127e01: Time spent bootstrapping tablet: real 0.303s user 0.020s sys 0.004s
I20250109 08:58:24.935762 23572 tablet_bootstrap.cc:492] T 750dd1e1055f4570b0aecdda3ebe8772 P cf55383bef6649eb9e78cb2d2512dedc: No bootstrap required, opened a new log
I20250109 08:58:24.936553 23572 ts_tablet_manager.cc:1397] T 750dd1e1055f4570b0aecdda3ebe8772 P cf55383bef6649eb9e78cb2d2512dedc: Time spent bootstrapping tablet: real 0.148s user 0.008s sys 0.004s
I20250109 08:58:24.939390 23574 tablet_bootstrap.cc:492] T de98c70e25494d2a8e581443be836bd3 P 235ad6af04c34c88a5261daa3604fada: No bootstrap required, opened a new log
I20250109 08:58:24.939857 23574 ts_tablet_manager.cc:1397] T de98c70e25494d2a8e581443be836bd3 P 235ad6af04c34c88a5261daa3604fada: Time spent bootstrapping tablet: real 0.213s user 0.044s sys 0.030s
I20250109 08:58:24.935091 23767 raft_consensus.cc:491] T fc1038255dd64cb5ba2425037d22f1f7 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:24.940901 23767 raft_consensus.cc:513] T fc1038255dd64cb5ba2425037d22f1f7 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
W20250109 08:58:24.941670 23271 consensus_peers.cc:487] T 94f2be307f704c75bceb213e52307cfb P 235ad6af04c34c88a5261daa3604fada -> Peer cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551): Couldn't send request to peer cf55383bef6649eb9e78cb2d2512dedc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:24.942755 23767 leader_election.cc:290] T fc1038255dd64cb5ba2425037d22f1f7 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829), ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365)
I20250109 08:58:24.944761 23452 raft_consensus.cc:2463] T 0dab6c23ce444dda9427901a98f07b15 P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05a74e1eb2474b3d8b3a2da1ef127e01 in term 1.
I20250109 08:58:24.944793 23380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc1038255dd64cb5ba2425037d22f1f7" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" is_pre_election: true
I20250109 08:58:24.945652 23380 raft_consensus.cc:2463] T fc1038255dd64cb5ba2425037d22f1f7 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 235ad6af04c34c88a5261daa3604fada in term 0.
I20250109 08:58:24.946579 23271 leader_election.cc:304] T fc1038255dd64cb5ba2425037d22f1f7 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 235ad6af04c34c88a5261daa3604fada, ed1efbce9ee3401e99a5260edeae4e7c; no voters:
I20250109 08:58:24.946997 23494 leader_election.cc:304] T 0dab6c23ce444dda9427901a98f07b15 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01, cf55383bef6649eb9e78cb2d2512dedc; no voters: ed1efbce9ee3401e99a5260edeae4e7c
I20250109 08:58:24.947365 23574 raft_consensus.cc:357] T de98c70e25494d2a8e581443be836bd3 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:24.947846 23873 raft_consensus.cc:2798] T fc1038255dd64cb5ba2425037d22f1f7 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:58:24.948356 23873 raft_consensus.cc:491] T fc1038255dd64cb5ba2425037d22f1f7 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:58:24.948721 23873 raft_consensus.cc:3054] T fc1038255dd64cb5ba2425037d22f1f7 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:24.949225 23586 leader_election.cc:290] T f1f9274aeda748f493f5752814fb262d P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 election: Requested vote from peers ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365), cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551)
I20250109 08:58:24.950156 23380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1f9274aeda748f493f5752814fb262d" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c"
I20250109 08:58:24.950723 23380 raft_consensus.cc:3054] T f1f9274aeda748f493f5752814fb262d P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:24.955340 23569 raft_consensus.cc:357] T ecc833ad1afa4079a75e41b1a807c923 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:24.956106 23574 raft_consensus.cc:383] T de98c70e25494d2a8e581443be836bd3 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:24.956008 23451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1f9274aeda748f493f5752814fb262d" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc"
I20250109 08:58:24.956656 23574 raft_consensus.cc:738] T de98c70e25494d2a8e581443be836bd3 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Initialized, Role: FOLLOWER
I20250109 08:58:24.956826 23451 raft_consensus.cc:3054] T f1f9274aeda748f493f5752814fb262d P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:24.956051 23569 raft_consensus.cc:383] T ecc833ad1afa4079a75e41b1a807c923 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:24.957859 23569 raft_consensus.cc:738] T ecc833ad1afa4079a75e41b1a807c923 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05a74e1eb2474b3d8b3a2da1ef127e01, State: Initialized, Role: FOLLOWER
I20250109 08:58:24.958528 23569 consensus_queue.cc:260] T ecc833ad1afa4079a75e41b1a807c923 P 05a74e1eb2474b3d8b3a2da1ef127e01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:24.958981 23528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc1038255dd64cb5ba2425037d22f1f7" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" is_pre_election: true
I20250109 08:58:24.960026 23873 raft_consensus.cc:513] T fc1038255dd64cb5ba2425037d22f1f7 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:24.960819 23569 ts_tablet_manager.cc:1428] T ecc833ad1afa4079a75e41b1a807c923 P 05a74e1eb2474b3d8b3a2da1ef127e01: Time spent starting tablet: real 0.025s user 0.004s sys 0.001s
I20250109 08:58:24.961766 23569 tablet_bootstrap.cc:492] T 750dd1e1055f4570b0aecdda3ebe8772 P 05a74e1eb2474b3d8b3a2da1ef127e01: Bootstrap starting.
I20250109 08:58:24.957432 23574 consensus_queue.cc:260] T de98c70e25494d2a8e581443be836bd3 P 235ad6af04c34c88a5261daa3604fada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
W20250109 08:58:24.962601 23269 leader_election.cc:343] T fc1038255dd64cb5ba2425037d22f1f7 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:24.964272 23574 ts_tablet_manager.cc:1428] T de98c70e25494d2a8e581443be836bd3 P 235ad6af04c34c88a5261daa3604fada: Time spent starting tablet: real 0.024s user 0.006s sys 0.000s
I20250109 08:58:24.965510 23574 tablet_bootstrap.cc:492] T d714271c7ea64c0b8b13fcf63591ec07 P 235ad6af04c34c88a5261daa3604fada: Bootstrap starting.
I20250109 08:58:24.966002 23380 raft_consensus.cc:2463] T f1f9274aeda748f493f5752814fb262d P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 235ad6af04c34c88a5261daa3604fada in term 1.
I20250109 08:58:24.967008 23569 tablet_bootstrap.cc:654] T 750dd1e1055f4570b0aecdda3ebe8772 P 05a74e1eb2474b3d8b3a2da1ef127e01: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:24.953857 23572 raft_consensus.cc:357] T 750dd1e1055f4570b0aecdda3ebe8772 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:24.967386 23572 raft_consensus.cc:383] T 750dd1e1055f4570b0aecdda3ebe8772 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:24.967763 23572 raft_consensus.cc:738] T 750dd1e1055f4570b0aecdda3ebe8772 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf55383bef6649eb9e78cb2d2512dedc, State: Initialized, Role: FOLLOWER
I20250109 08:58:24.968425 23572 consensus_queue.cc:260] T 750dd1e1055f4570b0aecdda3ebe8772 P cf55383bef6649eb9e78cb2d2512dedc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:24.966747 23634 raft_consensus.cc:2798] T 0dab6c23ce444dda9427901a98f07b15 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:58:24.966588 23666 consensus_queue.cc:1035] T db111e41c1be497e8e673a2c42b9ba66 P 235ad6af04c34c88a5261daa3604fada [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20250109 08:58:24.969682 23634 raft_consensus.cc:695] T 0dab6c23ce444dda9427901a98f07b15 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 LEADER]: Becoming Leader. State: Replica: 05a74e1eb2474b3d8b3a2da1ef127e01, State: Running, Role: LEADER
I20250109 08:58:24.970395 23572 ts_tablet_manager.cc:1428] T 750dd1e1055f4570b0aecdda3ebe8772 P cf55383bef6649eb9e78cb2d2512dedc: Time spent starting tablet: real 0.034s user 0.005s sys 0.000s
I20250109 08:58:24.970752 23766 raft_consensus.cc:491] T ecc833ad1afa4079a75e41b1a807c923 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:24.971369 23572 tablet_bootstrap.cc:492] T 77597d68423749e7b87f9f0db7a1ff3b P cf55383bef6649eb9e78cb2d2512dedc: Bootstrap starting.
I20250109 08:58:24.971190 23766 raft_consensus.cc:513] T ecc833ad1afa4079a75e41b1a807c923 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:24.971993 23271 leader_election.cc:304] T f1f9274aeda748f493f5752814fb262d P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 235ad6af04c34c88a5261daa3604fada, ed1efbce9ee3401e99a5260edeae4e7c; no voters:
I20250109 08:58:24.973079 23766 leader_election.cc:290] T ecc833ad1afa4079a75e41b1a807c923 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365), cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551)
I20250109 08:58:24.975373 23902 raft_consensus.cc:2798] T f1f9274aeda748f493f5752814fb262d P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:58:24.975808 23902 raft_consensus.cc:695] T f1f9274aeda748f493f5752814fb262d P 235ad6af04c34c88a5261daa3604fada [term 1 LEADER]: Becoming Leader. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Running, Role: LEADER
I20250109 08:58:24.976938 23379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ecc833ad1afa4079a75e41b1a807c923" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" is_pre_election: true
I20250109 08:58:24.977628 23379 raft_consensus.cc:2463] T ecc833ad1afa4079a75e41b1a807c923 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05a74e1eb2474b3d8b3a2da1ef127e01 in term 0.
I20250109 08:58:24.977070 23873 leader_election.cc:290] T fc1038255dd64cb5ba2425037d22f1f7 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 election: Requested vote from peers 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829), ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365)
I20250109 08:58:24.977946 23572 tablet_bootstrap.cc:654] T 77597d68423749e7b87f9f0db7a1ff3b P cf55383bef6649eb9e78cb2d2512dedc: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:24.978811 23494 leader_election.cc:304] T ecc833ad1afa4079a75e41b1a807c923 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01, ed1efbce9ee3401e99a5260edeae4e7c; no voters:
I20250109 08:58:24.979480 23528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc1038255dd64cb5ba2425037d22f1f7" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01"
I20250109 08:58:24.980149 23452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ecc833ad1afa4079a75e41b1a807c923" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc" is_pre_election: true
W20250109 08:58:24.980960 23269 leader_election.cc:343] T fc1038255dd64cb5ba2425037d22f1f7 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829): Illegal state: must be running to vote when last-logged opid is not known
W20250109 08:58:24.981578 23494 leader_election.cc:343] T ecc833ad1afa4079a75e41b1a807c923 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:24.982051 23574 tablet_bootstrap.cc:654] T d714271c7ea64c0b8b13fcf63591ec07 P 235ad6af04c34c88a5261daa3604fada: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:24.982256 23379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc1038255dd64cb5ba2425037d22f1f7" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c"
I20250109 08:58:24.982508 23766 raft_consensus.cc:2798] T ecc833ad1afa4079a75e41b1a807c923 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:58:24.970490 23634 consensus_queue.cc:237] T 0dab6c23ce444dda9427901a98f07b15 P 05a74e1eb2474b3d8b3a2da1ef127e01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:24.982834 23379 raft_consensus.cc:3054] T fc1038255dd64cb5ba2425037d22f1f7 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:24.977931 23902 consensus_queue.cc:237] T f1f9274aeda748f493f5752814fb262d P 235ad6af04c34c88a5261daa3604fada [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:24.995277 23580 tablet_bootstrap.cc:492] T 1365130ef97c4581b623c22c39753831 P ed1efbce9ee3401e99a5260edeae4e7c: No bootstrap required, opened a new log
I20250109 08:58:24.995829 23580 ts_tablet_manager.cc:1397] T 1365130ef97c4581b623c22c39753831 P ed1efbce9ee3401e99a5260edeae4e7c: Time spent bootstrapping tablet: real 0.154s user 0.012s sys 0.004s
I20250109 08:58:24.994740 23766 raft_consensus.cc:491] T ecc833ad1afa4079a75e41b1a807c923 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:58:24.998634 23766 raft_consensus.cc:3054] T ecc833ad1afa4079a75e41b1a807c923 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:25.000674 23666 consensus_queue.cc:1035] T db111e41c1be497e8e673a2c42b9ba66 P 235ad6af04c34c88a5261daa3604fada [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:58:25.003994 23451 raft_consensus.cc:2463] T f1f9274aeda748f493f5752814fb262d P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 235ad6af04c34c88a5261daa3604fada in term 1.
I20250109 08:58:25.005968 23893 raft_consensus.cc:491] T 444507f7812441ceaeebc44bc8f06293 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:25.006575 23893 raft_consensus.cc:513] T 444507f7812441ceaeebc44bc8f06293 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:25.009413 23305 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "444507f7812441ceaeebc44bc8f06293" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada" is_pre_election: true
W20250109 08:58:25.010725 23494 leader_election.cc:343] T 444507f7812441ceaeebc44bc8f06293 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:25.014813 23893 leader_election.cc:290] T 444507f7812441ceaeebc44bc8f06293 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691), ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365)
I20250109 08:58:25.015869 23378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "444507f7812441ceaeebc44bc8f06293" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" is_pre_election: true
W20250109 08:58:25.018126 23494 leader_election.cc:343] T 444507f7812441ceaeebc44bc8f06293 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:25.018528 23494 leader_election.cc:304] T 444507f7812441ceaeebc44bc8f06293 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01; no voters: 235ad6af04c34c88a5261daa3604fada, ed1efbce9ee3401e99a5260edeae4e7c
I20250109 08:58:25.019291 23878 raft_consensus.cc:491] T 750dd1e1055f4570b0aecdda3ebe8772 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:24.996451 23203 catalog_manager.cc:5526] T cc9886333c0343798226d59758a1ea5e P 05a74e1eb2474b3d8b3a2da1ef127e01 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:25.019519 23893 raft_consensus.cc:2743] T 444507f7812441ceaeebc44bc8f06293 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:58:25.019858 23878 raft_consensus.cc:513] T 750dd1e1055f4570b0aecdda3ebe8772 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:25.025535 23572 tablet_bootstrap.cc:492] T 77597d68423749e7b87f9f0db7a1ff3b P cf55383bef6649eb9e78cb2d2512dedc: No bootstrap required, opened a new log
I20250109 08:58:25.026003 23572 ts_tablet_manager.cc:1397] T 77597d68423749e7b87f9f0db7a1ff3b P cf55383bef6649eb9e78cb2d2512dedc: Time spent bootstrapping tablet: real 0.055s user 0.018s sys 0.026s
I20250109 08:58:25.026868 23878 leader_election.cc:290] T 750dd1e1055f4570b0aecdda3ebe8772 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551), 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829)
I20250109 08:58:25.028601 23572 raft_consensus.cc:357] T 77597d68423749e7b87f9f0db7a1ff3b P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:25.029067 23528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "750dd1e1055f4570b0aecdda3ebe8772" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" is_pre_election: true
I20250109 08:58:25.029397 23572 raft_consensus.cc:383] T 77597d68423749e7b87f9f0db7a1ff3b P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:25.029685 23572 raft_consensus.cc:738] T 77597d68423749e7b87f9f0db7a1ff3b P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf55383bef6649eb9e78cb2d2512dedc, State: Initialized, Role: FOLLOWER
W20250109 08:58:25.030441 23344 leader_election.cc:343] T 750dd1e1055f4570b0aecdda3ebe8772 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:25.030575 23572 consensus_queue.cc:260] T 77597d68423749e7b87f9f0db7a1ff3b P cf55383bef6649eb9e78cb2d2512dedc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:25.032790 23572 ts_tablet_manager.cc:1428] T 77597d68423749e7b87f9f0db7a1ff3b P cf55383bef6649eb9e78cb2d2512dedc: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20250109 08:58:25.033814 23572 tablet_bootstrap.cc:492] T d714271c7ea64c0b8b13fcf63591ec07 P cf55383bef6649eb9e78cb2d2512dedc: Bootstrap starting.
I20250109 08:58:25.043792 23572 tablet_bootstrap.cc:654] T d714271c7ea64c0b8b13fcf63591ec07 P cf55383bef6649eb9e78cb2d2512dedc: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:25.045709 23580 raft_consensus.cc:357] T 1365130ef97c4581b623c22c39753831 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:25.046499 23580 raft_consensus.cc:383] T 1365130ef97c4581b623c22c39753831 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:25.049546 23580 raft_consensus.cc:738] T 1365130ef97c4581b623c22c39753831 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed1efbce9ee3401e99a5260edeae4e7c, State: Initialized, Role: FOLLOWER
I20250109 08:58:25.048800 23379 raft_consensus.cc:2463] T fc1038255dd64cb5ba2425037d22f1f7 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 235ad6af04c34c88a5261daa3604fada in term 1.
I20250109 08:58:25.050237 23451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "750dd1e1055f4570b0aecdda3ebe8772" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc" is_pre_election: true
I20250109 08:58:25.051172 23574 tablet_bootstrap.cc:492] T d714271c7ea64c0b8b13fcf63591ec07 P 235ad6af04c34c88a5261daa3604fada: No bootstrap required, opened a new log
I20250109 08:58:25.051219 23451 raft_consensus.cc:2463] T 750dd1e1055f4570b0aecdda3ebe8772 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed1efbce9ee3401e99a5260edeae4e7c in term 0.
I20250109 08:58:25.051688 23574 ts_tablet_manager.cc:1397] T d714271c7ea64c0b8b13fcf63591ec07 P 235ad6af04c34c88a5261daa3604fada: Time spent bootstrapping tablet: real 0.086s user 0.006s sys 0.014s
I20250109 08:58:25.054847 23271 leader_election.cc:304] T fc1038255dd64cb5ba2425037d22f1f7 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 235ad6af04c34c88a5261daa3604fada, ed1efbce9ee3401e99a5260edeae4e7c; no voters: 05a74e1eb2474b3d8b3a2da1ef127e01
I20250109 08:58:25.060161 23580 consensus_queue.cc:260] T 1365130ef97c4581b623c22c39753831 P ed1efbce9ee3401e99a5260edeae4e7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:25.062973 23666 raft_consensus.cc:2798] T fc1038255dd64cb5ba2425037d22f1f7 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:58:25.063958 23346 leader_election.cc:304] T 750dd1e1055f4570b0aecdda3ebe8772 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf55383bef6649eb9e78cb2d2512dedc, ed1efbce9ee3401e99a5260edeae4e7c; no voters: 05a74e1eb2474b3d8b3a2da1ef127e01
I20250109 08:58:25.065619 23877 raft_consensus.cc:2798] T 750dd1e1055f4570b0aecdda3ebe8772 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:58:25.066049 23877 raft_consensus.cc:491] T 750dd1e1055f4570b0aecdda3ebe8772 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:58:25.066421 23877 raft_consensus.cc:3054] T 750dd1e1055f4570b0aecdda3ebe8772 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:25.065693 23666 raft_consensus.cc:695] T fc1038255dd64cb5ba2425037d22f1f7 P 235ad6af04c34c88a5261daa3604fada [term 1 LEADER]: Becoming Leader. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Running, Role: LEADER
I20250109 08:58:25.069298 23666 consensus_queue.cc:237] T fc1038255dd64cb5ba2425037d22f1f7 P 235ad6af04c34c88a5261daa3604fada [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:25.074456 23877 raft_consensus.cc:513] T 750dd1e1055f4570b0aecdda3ebe8772 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:25.077584 23574 raft_consensus.cc:357] T d714271c7ea64c0b8b13fcf63591ec07 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:25.079330 23574 raft_consensus.cc:383] T d714271c7ea64c0b8b13fcf63591ec07 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:25.079602 23574 raft_consensus.cc:738] T d714271c7ea64c0b8b13fcf63591ec07 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Initialized, Role: FOLLOWER
I20250109 08:58:25.078541 23766 raft_consensus.cc:513] T ecc833ad1afa4079a75e41b1a807c923 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:25.083788 23301 raft_consensus.cc:3054] T de98c70e25494d2a8e581443be836bd3 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:25.085352 23766 leader_election.cc:290] T ecc833ad1afa4079a75e41b1a807c923 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 election: Requested vote from peers ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365), cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551)
I20250109 08:58:25.085664 23580 ts_tablet_manager.cc:1428] T 1365130ef97c4581b623c22c39753831 P ed1efbce9ee3401e99a5260edeae4e7c: Time spent starting tablet: real 0.090s user 0.006s sys 0.001s
I20250109 08:58:25.086380 23452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ecc833ad1afa4079a75e41b1a807c923" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc"
W20250109 08:58:25.087575 23494 leader_election.cc:343] T ecc833ad1afa4079a75e41b1a807c923 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:25.083364 23574 consensus_queue.cc:260] T d714271c7ea64c0b8b13fcf63591ec07 P 235ad6af04c34c88a5261daa3604fada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:25.090641 23574 ts_tablet_manager.cc:1428] T d714271c7ea64c0b8b13fcf63591ec07 P 235ad6af04c34c88a5261daa3604fada: Time spent starting tablet: real 0.031s user 0.004s sys 0.000s
I20250109 08:58:25.091459 23574 tablet_bootstrap.cc:492] T 060daed1d3334c6188413dd381b82bf9 P 235ad6af04c34c88a5261daa3604fada: Bootstrap starting.
I20250109 08:58:25.103018 23583 consensus_queue.cc:1035] T de98c70e25494d2a8e581443be836bd3 P cf55383bef6649eb9e78cb2d2512dedc [LEADER]: Connected to new peer: Peer: permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250109 08:58:25.103097 23574 tablet_bootstrap.cc:654] T 060daed1d3334c6188413dd381b82bf9 P 235ad6af04c34c88a5261daa3604fada: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:25.106566 23569 tablet_bootstrap.cc:492] T 750dd1e1055f4570b0aecdda3ebe8772 P 05a74e1eb2474b3d8b3a2da1ef127e01: No bootstrap required, opened a new log
I20250109 08:58:25.107079 23569 ts_tablet_manager.cc:1397] T 750dd1e1055f4570b0aecdda3ebe8772 P 05a74e1eb2474b3d8b3a2da1ef127e01: Time spent bootstrapping tablet: real 0.145s user 0.011s sys 0.014s
I20250109 08:58:25.109695 23569 raft_consensus.cc:357] T 750dd1e1055f4570b0aecdda3ebe8772 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:25.110366 23569 raft_consensus.cc:383] T 750dd1e1055f4570b0aecdda3ebe8772 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:25.110625 23569 raft_consensus.cc:738] T 750dd1e1055f4570b0aecdda3ebe8772 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05a74e1eb2474b3d8b3a2da1ef127e01, State: Initialized, Role: FOLLOWER
I20250109 08:58:25.111243 23569 consensus_queue.cc:260] T 750dd1e1055f4570b0aecdda3ebe8772 P 05a74e1eb2474b3d8b3a2da1ef127e01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:25.133266 23569 ts_tablet_manager.cc:1428] T 750dd1e1055f4570b0aecdda3ebe8772 P 05a74e1eb2474b3d8b3a2da1ef127e01: Time spent starting tablet: real 0.026s user 0.005s sys 0.002s
I20250109 08:58:25.134024 23569 tablet_bootstrap.cc:492] T 11aa76a3a30643708fececed660a5250 P 05a74e1eb2474b3d8b3a2da1ef127e01: Bootstrap starting.
I20250109 08:58:25.138692 23572 tablet_bootstrap.cc:492] T d714271c7ea64c0b8b13fcf63591ec07 P cf55383bef6649eb9e78cb2d2512dedc: No bootstrap required, opened a new log
I20250109 08:58:25.139164 23572 ts_tablet_manager.cc:1397] T d714271c7ea64c0b8b13fcf63591ec07 P cf55383bef6649eb9e78cb2d2512dedc: Time spent bootstrapping tablet: real 0.106s user 0.016s sys 0.001s
I20250109 08:58:25.137759 23451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "750dd1e1055f4570b0aecdda3ebe8772" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc"
I20250109 08:58:25.140455 23451 raft_consensus.cc:3054] T 750dd1e1055f4570b0aecdda3ebe8772 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:25.148919 23580 tablet_bootstrap.cc:492] T d87d3b4755344b82b0aec7ffd1ce785c P ed1efbce9ee3401e99a5260edeae4e7c: Bootstrap starting.
I20250109 08:58:25.149972 23451 raft_consensus.cc:2463] T 750dd1e1055f4570b0aecdda3ebe8772 P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed1efbce9ee3401e99a5260edeae4e7c in term 1.
I20250109 08:58:25.151820 23346 leader_election.cc:304] T 750dd1e1055f4570b0aecdda3ebe8772 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf55383bef6649eb9e78cb2d2512dedc, ed1efbce9ee3401e99a5260edeae4e7c; no voters:
I20250109 08:58:25.156311 23894 consensus_queue.cc:1035] T de98c70e25494d2a8e581443be836bd3 P cf55383bef6649eb9e78cb2d2512dedc [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20250109 08:58:25.160831 23569 tablet_bootstrap.cc:654] T 11aa76a3a30643708fececed660a5250 P 05a74e1eb2474b3d8b3a2da1ef127e01: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:25.161871 23877 leader_election.cc:290] T 750dd1e1055f4570b0aecdda3ebe8772 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 election: Requested vote from peers cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551), 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829)
I20250109 08:58:25.159987 23572 raft_consensus.cc:357] T d714271c7ea64c0b8b13fcf63591ec07 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:25.162995 23572 raft_consensus.cc:383] T d714271c7ea64c0b8b13fcf63591ec07 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:25.164455 23572 raft_consensus.cc:738] T d714271c7ea64c0b8b13fcf63591ec07 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf55383bef6649eb9e78cb2d2512dedc, State: Initialized, Role: FOLLOWER
I20250109 08:58:25.156613 23611 raft_consensus.cc:2798] T 750dd1e1055f4570b0aecdda3ebe8772 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:58:25.165308 23611 raft_consensus.cc:695] T 750dd1e1055f4570b0aecdda3ebe8772 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 LEADER]: Becoming Leader. State: Replica: ed1efbce9ee3401e99a5260edeae4e7c, State: Running, Role: LEADER
I20250109 08:58:25.165484 23572 consensus_queue.cc:260] T d714271c7ea64c0b8b13fcf63591ec07 P cf55383bef6649eb9e78cb2d2512dedc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:25.166141 23611 consensus_queue.cc:237] T 750dd1e1055f4570b0aecdda3ebe8772 P ed1efbce9ee3401e99a5260edeae4e7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:25.168068 23572 ts_tablet_manager.cc:1428] T d714271c7ea64c0b8b13fcf63591ec07 P cf55383bef6649eb9e78cb2d2512dedc: Time spent starting tablet: real 0.029s user 0.007s sys 0.000s
I20250109 08:58:25.169035 23572 tablet_bootstrap.cc:492] T 950e07e9cc794b8ab9ec5bd97d7e8e59 P cf55383bef6649eb9e78cb2d2512dedc: Bootstrap starting.
I20250109 08:58:25.169306 23658 raft_consensus.cc:491] T 1f9c6525267145d6bb73dde32ed9b7d4 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:25.169817 23658 raft_consensus.cc:513] T 1f9c6525267145d6bb73dde32ed9b7d4 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:25.157434 23528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "750dd1e1055f4570b0aecdda3ebe8772" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01"
I20250109 08:58:25.171082 23528 raft_consensus.cc:3054] T 750dd1e1055f4570b0aecdda3ebe8772 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:25.171746 23658 leader_election.cc:290] T 1f9c6525267145d6bb73dde32ed9b7d4 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551), 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829)
I20250109 08:58:25.174822 23527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9c6525267145d6bb73dde32ed9b7d4" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" is_pre_election: true
I20250109 08:58:25.180099 23572 tablet_bootstrap.cc:654] T 950e07e9cc794b8ab9ec5bd97d7e8e59 P cf55383bef6649eb9e78cb2d2512dedc: Neither blocks nor log segments found. Creating new log.
W20250109 08:58:25.181013 23344 leader_election.cc:343] T 1f9c6525267145d6bb73dde32ed9b7d4 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:25.154871 23580 tablet_bootstrap.cc:654] T d87d3b4755344b82b0aec7ffd1ce785c P ed1efbce9ee3401e99a5260edeae4e7c: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:25.187772 23574 tablet_bootstrap.cc:492] T 060daed1d3334c6188413dd381b82bf9 P 235ad6af04c34c88a5261daa3604fada: No bootstrap required, opened a new log
I20250109 08:58:25.188242 23574 ts_tablet_manager.cc:1397] T 060daed1d3334c6188413dd381b82bf9 P 235ad6af04c34c88a5261daa3604fada: Time spent bootstrapping tablet: real 0.097s user 0.009s sys 0.000s
I20250109 08:58:25.189121 23569 tablet_bootstrap.cc:492] T 11aa76a3a30643708fececed660a5250 P 05a74e1eb2474b3d8b3a2da1ef127e01: No bootstrap required, opened a new log
I20250109 08:58:25.182008 23203 catalog_manager.cc:5526] T f1f9274aeda748f493f5752814fb262d P 235ad6af04c34c88a5261daa3604fada reported cstate change: term changed from 0 to 1, leader changed from <none> to 235ad6af04c34c88a5261daa3604fada (127.19.226.193). New cstate: current_term: 1 leader_uuid: "235ad6af04c34c88a5261daa3604fada" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:25.190290 23569 ts_tablet_manager.cc:1397] T 11aa76a3a30643708fececed660a5250 P 05a74e1eb2474b3d8b3a2da1ef127e01: Time spent bootstrapping tablet: real 0.056s user 0.014s sys 0.002s
I20250109 08:58:25.156548 23378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ecc833ad1afa4079a75e41b1a807c923" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c"
I20250109 08:58:25.191013 23378 raft_consensus.cc:3054] T ecc833ad1afa4079a75e41b1a807c923 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:25.180009 23202 catalog_manager.cc:5526] T 0dab6c23ce444dda9427901a98f07b15 P 05a74e1eb2474b3d8b3a2da1ef127e01 reported cstate change: term changed from 0 to 1, leader changed from <none> to 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196). New cstate: current_term: 1 leader_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } health_report { overall_health: HEALTHY } } }
I20250109 08:58:25.182880 23528 raft_consensus.cc:2463] T 750dd1e1055f4570b0aecdda3ebe8772 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed1efbce9ee3401e99a5260edeae4e7c in term 1.
I20250109 08:58:25.195046 23569 raft_consensus.cc:357] T 11aa76a3a30643708fececed660a5250 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:25.195780 23569 raft_consensus.cc:383] T 11aa76a3a30643708fececed660a5250 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:25.196069 23569 raft_consensus.cc:738] T 11aa76a3a30643708fececed660a5250 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05a74e1eb2474b3d8b3a2da1ef127e01, State: Initialized, Role: FOLLOWER
I20250109 08:58:25.182989 23451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9c6525267145d6bb73dde32ed9b7d4" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc" is_pre_election: true
I20250109 08:58:25.196669 23569 consensus_queue.cc:260] T 11aa76a3a30643708fececed660a5250 P 05a74e1eb2474b3d8b3a2da1ef127e01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
W20250109 08:58:25.197576 23346 leader_election.cc:343] T 1f9c6525267145d6bb73dde32ed9b7d4 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:25.198050 23346 leader_election.cc:304] T 1f9c6525267145d6bb73dde32ed9b7d4 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ed1efbce9ee3401e99a5260edeae4e7c; no voters: 05a74e1eb2474b3d8b3a2da1ef127e01, cf55383bef6649eb9e78cb2d2512dedc
I20250109 08:58:25.199173 23611 raft_consensus.cc:2743] T 1f9c6525267145d6bb73dde32ed9b7d4 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:58:25.200867 23574 raft_consensus.cc:357] T 060daed1d3334c6188413dd381b82bf9 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:25.201558 23574 raft_consensus.cc:383] T 060daed1d3334c6188413dd381b82bf9 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:25.201819 23574 raft_consensus.cc:738] T 060daed1d3334c6188413dd381b82bf9 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Initialized, Role: FOLLOWER
I20250109 08:58:25.202440 23574 consensus_queue.cc:260] T 060daed1d3334c6188413dd381b82bf9 P 235ad6af04c34c88a5261daa3604fada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:25.204592 23574 ts_tablet_manager.cc:1428] T 060daed1d3334c6188413dd381b82bf9 P 235ad6af04c34c88a5261daa3604fada: Time spent starting tablet: real 0.016s user 0.006s sys 0.000s
I20250109 08:58:25.205368 23574 tablet_bootstrap.cc:492] T 5f0d500a91b54851bda52e5ccf741fd6 P 235ad6af04c34c88a5261daa3604fada: Bootstrap starting.
I20250109 08:58:25.209353 23569 ts_tablet_manager.cc:1428] T 11aa76a3a30643708fececed660a5250 P 05a74e1eb2474b3d8b3a2da1ef127e01: Time spent starting tablet: real 0.019s user 0.006s sys 0.000s
I20250109 08:58:25.210273 23569 tablet_bootstrap.cc:492] T 060daed1d3334c6188413dd381b82bf9 P 05a74e1eb2474b3d8b3a2da1ef127e01: Bootstrap starting.
I20250109 08:58:25.211656 23378 raft_consensus.cc:2463] T ecc833ad1afa4079a75e41b1a807c923 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05a74e1eb2474b3d8b3a2da1ef127e01 in term 1.
I20250109 08:58:25.213086 23494 leader_election.cc:304] T ecc833ad1afa4079a75e41b1a807c923 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01, ed1efbce9ee3401e99a5260edeae4e7c; no voters: cf55383bef6649eb9e78cb2d2512dedc
I20250109 08:58:25.213783 23634 raft_consensus.cc:2798] T ecc833ad1afa4079a75e41b1a807c923 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:58:25.214236 23634 raft_consensus.cc:695] T ecc833ad1afa4079a75e41b1a807c923 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 LEADER]: Becoming Leader. State: Replica: 05a74e1eb2474b3d8b3a2da1ef127e01, State: Running, Role: LEADER
I20250109 08:58:25.215248 23634 consensus_queue.cc:237] T ecc833ad1afa4079a75e41b1a807c923 P 05a74e1eb2474b3d8b3a2da1ef127e01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:25.218137 23574 tablet_bootstrap.cc:654] T 5f0d500a91b54851bda52e5ccf741fd6 P 235ad6af04c34c88a5261daa3604fada: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:25.220813 23580 tablet_bootstrap.cc:492] T d87d3b4755344b82b0aec7ffd1ce785c P ed1efbce9ee3401e99a5260edeae4e7c: No bootstrap required, opened a new log
I20250109 08:58:25.221282 23580 ts_tablet_manager.cc:1397] T d87d3b4755344b82b0aec7ffd1ce785c P ed1efbce9ee3401e99a5260edeae4e7c: Time spent bootstrapping tablet: real 0.073s user 0.010s sys 0.007s
I20250109 08:58:25.224648 23200 catalog_manager.cc:5526] T 750dd1e1055f4570b0aecdda3ebe8772 P ed1efbce9ee3401e99a5260edeae4e7c reported cstate change: term changed from 0 to 1, leader changed from <none> to ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194). New cstate: current_term: 1 leader_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:25.223850 23580 raft_consensus.cc:357] T d87d3b4755344b82b0aec7ffd1ce785c P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:25.232995 23580 raft_consensus.cc:383] T d87d3b4755344b82b0aec7ffd1ce785c P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:25.233294 23580 raft_consensus.cc:738] T d87d3b4755344b82b0aec7ffd1ce785c P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed1efbce9ee3401e99a5260edeae4e7c, State: Initialized, Role: FOLLOWER
I20250109 08:58:25.234329 23580 consensus_queue.cc:260] T d87d3b4755344b82b0aec7ffd1ce785c P ed1efbce9ee3401e99a5260edeae4e7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:25.239050 23572 tablet_bootstrap.cc:492] T 950e07e9cc794b8ab9ec5bd97d7e8e59 P cf55383bef6649eb9e78cb2d2512dedc: No bootstrap required, opened a new log
I20250109 08:58:25.239593 23572 ts_tablet_manager.cc:1397] T 950e07e9cc794b8ab9ec5bd97d7e8e59 P cf55383bef6649eb9e78cb2d2512dedc: Time spent bootstrapping tablet: real 0.071s user 0.007s sys 0.005s
I20250109 08:58:25.244078 23572 raft_consensus.cc:357] T 950e07e9cc794b8ab9ec5bd97d7e8e59 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:25.244823 23572 raft_consensus.cc:383] T 950e07e9cc794b8ab9ec5bd97d7e8e59 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:25.245082 23572 raft_consensus.cc:738] T 950e07e9cc794b8ab9ec5bd97d7e8e59 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf55383bef6649eb9e78cb2d2512dedc, State: Initialized, Role: FOLLOWER
I20250109 08:58:25.245687 23572 consensus_queue.cc:260] T 950e07e9cc794b8ab9ec5bd97d7e8e59 P cf55383bef6649eb9e78cb2d2512dedc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:25.250345 23580 ts_tablet_manager.cc:1428] T d87d3b4755344b82b0aec7ffd1ce785c P ed1efbce9ee3401e99a5260edeae4e7c: Time spent starting tablet: real 0.029s user 0.005s sys 0.002s
I20250109 08:58:25.251397 23580 tablet_bootstrap.cc:492] T 1ac4317290b24209a76d2e19e68ee45b P ed1efbce9ee3401e99a5260edeae4e7c: Bootstrap starting.
I20250109 08:58:25.256700 23569 tablet_bootstrap.cc:654] T 060daed1d3334c6188413dd381b82bf9 P 05a74e1eb2474b3d8b3a2da1ef127e01: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:25.257856 23574 tablet_bootstrap.cc:492] T 5f0d500a91b54851bda52e5ccf741fd6 P 235ad6af04c34c88a5261daa3604fada: No bootstrap required, opened a new log
I20250109 08:58:25.257161 23199 catalog_manager.cc:5526] T ecc833ad1afa4079a75e41b1a807c923 P 05a74e1eb2474b3d8b3a2da1ef127e01 reported cstate change: term changed from 0 to 1, leader changed from <none> to 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196). New cstate: current_term: 1 leader_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } health_report { overall_health: HEALTHY } } }
I20250109 08:58:25.258466 23574 ts_tablet_manager.cc:1397] T 5f0d500a91b54851bda52e5ccf741fd6 P 235ad6af04c34c88a5261daa3604fada: Time spent bootstrapping tablet: real 0.053s user 0.020s sys 0.024s
I20250109 08:58:25.258004 23202 catalog_manager.cc:5526] T fc1038255dd64cb5ba2425037d22f1f7 P 235ad6af04c34c88a5261daa3604fada reported cstate change: term changed from 0 to 1, leader changed from <none> to 235ad6af04c34c88a5261daa3604fada (127.19.226.193). New cstate: current_term: 1 leader_uuid: "235ad6af04c34c88a5261daa3604fada" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:25.259122 23572 ts_tablet_manager.cc:1428] T 950e07e9cc794b8ab9ec5bd97d7e8e59 P cf55383bef6649eb9e78cb2d2512dedc: Time spent starting tablet: real 0.019s user 0.008s sys 0.000s
I20250109 08:58:25.260031 23572 tablet_bootstrap.cc:492] T 1f9c6525267145d6bb73dde32ed9b7d4 P cf55383bef6649eb9e78cb2d2512dedc: Bootstrap starting.
I20250109 08:58:25.261278 23574 raft_consensus.cc:357] T 5f0d500a91b54851bda52e5ccf741fd6 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:25.262017 23574 raft_consensus.cc:383] T 5f0d500a91b54851bda52e5ccf741fd6 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:25.262303 23574 raft_consensus.cc:738] T 5f0d500a91b54851bda52e5ccf741fd6 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Initialized, Role: FOLLOWER
I20250109 08:58:25.264431 23574 consensus_queue.cc:260] T 5f0d500a91b54851bda52e5ccf741fd6 P 235ad6af04c34c88a5261daa3604fada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:25.266358 23580 tablet_bootstrap.cc:654] T 1ac4317290b24209a76d2e19e68ee45b P ed1efbce9ee3401e99a5260edeae4e7c: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:25.266908 23574 ts_tablet_manager.cc:1428] T 5f0d500a91b54851bda52e5ccf741fd6 P 235ad6af04c34c88a5261daa3604fada: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20250109 08:58:25.267825 23574 tablet_bootstrap.cc:492] T 77597d68423749e7b87f9f0db7a1ff3b P 235ad6af04c34c88a5261daa3604fada: Bootstrap starting.
I20250109 08:58:25.271284 23572 tablet_bootstrap.cc:654] T 1f9c6525267145d6bb73dde32ed9b7d4 P cf55383bef6649eb9e78cb2d2512dedc: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:25.273368 23574 tablet_bootstrap.cc:654] T 77597d68423749e7b87f9f0db7a1ff3b P 235ad6af04c34c88a5261daa3604fada: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:25.273702 23569 tablet_bootstrap.cc:492] T 060daed1d3334c6188413dd381b82bf9 P 05a74e1eb2474b3d8b3a2da1ef127e01: No bootstrap required, opened a new log
I20250109 08:58:25.274165 23569 ts_tablet_manager.cc:1397] T 060daed1d3334c6188413dd381b82bf9 P 05a74e1eb2474b3d8b3a2da1ef127e01: Time spent bootstrapping tablet: real 0.064s user 0.009s sys 0.000s
I20250109 08:58:25.276558 23569 raft_consensus.cc:357] T 060daed1d3334c6188413dd381b82bf9 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:25.277212 23569 raft_consensus.cc:383] T 060daed1d3334c6188413dd381b82bf9 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:25.277467 23569 raft_consensus.cc:738] T 060daed1d3334c6188413dd381b82bf9 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05a74e1eb2474b3d8b3a2da1ef127e01, State: Initialized, Role: FOLLOWER
I20250109 08:58:25.278580 23569 consensus_queue.cc:260] T 060daed1d3334c6188413dd381b82bf9 P 05a74e1eb2474b3d8b3a2da1ef127e01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:25.285034 23574 tablet_bootstrap.cc:492] T 77597d68423749e7b87f9f0db7a1ff3b P 235ad6af04c34c88a5261daa3604fada: No bootstrap required, opened a new log
I20250109 08:58:25.285506 23574 ts_tablet_manager.cc:1397] T 77597d68423749e7b87f9f0db7a1ff3b P 235ad6af04c34c88a5261daa3604fada: Time spent bootstrapping tablet: real 0.018s user 0.009s sys 0.006s
I20250109 08:58:25.285583 23569 ts_tablet_manager.cc:1428] T 060daed1d3334c6188413dd381b82bf9 P 05a74e1eb2474b3d8b3a2da1ef127e01: Time spent starting tablet: real 0.011s user 0.006s sys 0.000s
I20250109 08:58:25.286983 23569 tablet_bootstrap.cc:492] T 950e07e9cc794b8ab9ec5bd97d7e8e59 P 05a74e1eb2474b3d8b3a2da1ef127e01: Bootstrap starting.
I20250109 08:58:25.287882 23574 raft_consensus.cc:357] T 77597d68423749e7b87f9f0db7a1ff3b P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:25.288604 23574 raft_consensus.cc:383] T 77597d68423749e7b87f9f0db7a1ff3b P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:25.288852 23574 raft_consensus.cc:738] T 77597d68423749e7b87f9f0db7a1ff3b P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Initialized, Role: FOLLOWER
I20250109 08:58:25.289577 23378 raft_consensus.cc:3054] T d87d3b4755344b82b0aec7ffd1ce785c P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:25.290616 23574 consensus_queue.cc:260] T 77597d68423749e7b87f9f0db7a1ff3b P 235ad6af04c34c88a5261daa3604fada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:25.292037 23569 tablet_bootstrap.cc:654] T 950e07e9cc794b8ab9ec5bd97d7e8e59 P 05a74e1eb2474b3d8b3a2da1ef127e01: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:25.292701 23580 tablet_bootstrap.cc:492] T 1ac4317290b24209a76d2e19e68ee45b P ed1efbce9ee3401e99a5260edeae4e7c: No bootstrap required, opened a new log
I20250109 08:58:25.293068 23574 ts_tablet_manager.cc:1428] T 77597d68423749e7b87f9f0db7a1ff3b P 235ad6af04c34c88a5261daa3604fada: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20250109 08:58:25.293408 23580 ts_tablet_manager.cc:1397] T 1ac4317290b24209a76d2e19e68ee45b P ed1efbce9ee3401e99a5260edeae4e7c: Time spent bootstrapping tablet: real 0.042s user 0.011s sys 0.000s
I20250109 08:58:25.293946 23574 tablet_bootstrap.cc:492] T bfb140bb3a8b44bd8b4e1fcf0946162b P 235ad6af04c34c88a5261daa3604fada: Bootstrap starting.
I20250109 08:58:25.296003 23580 raft_consensus.cc:357] T 1ac4317290b24209a76d2e19e68ee45b P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:25.296801 23580 raft_consensus.cc:383] T 1ac4317290b24209a76d2e19e68ee45b P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:25.297063 23580 raft_consensus.cc:738] T 1ac4317290b24209a76d2e19e68ee45b P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed1efbce9ee3401e99a5260edeae4e7c, State: Initialized, Role: FOLLOWER
I20250109 08:58:25.297845 23580 consensus_queue.cc:260] T 1ac4317290b24209a76d2e19e68ee45b P ed1efbce9ee3401e99a5260edeae4e7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:25.299585 23574 tablet_bootstrap.cc:654] T bfb140bb3a8b44bd8b4e1fcf0946162b P 235ad6af04c34c88a5261daa3604fada: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:25.303468 23572 tablet_bootstrap.cc:492] T 1f9c6525267145d6bb73dde32ed9b7d4 P cf55383bef6649eb9e78cb2d2512dedc: No bootstrap required, opened a new log
I20250109 08:58:25.303576 23611 raft_consensus.cc:491] T 1365130ef97c4581b623c22c39753831 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:25.303915 23572 ts_tablet_manager.cc:1397] T 1f9c6525267145d6bb73dde32ed9b7d4 P cf55383bef6649eb9e78cb2d2512dedc: Time spent bootstrapping tablet: real 0.044s user 0.009s sys 0.000s
I20250109 08:58:25.304028 23611 raft_consensus.cc:513] T 1365130ef97c4581b623c22c39753831 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:25.304455 23580 ts_tablet_manager.cc:1428] T 1ac4317290b24209a76d2e19e68ee45b P ed1efbce9ee3401e99a5260edeae4e7c: Time spent starting tablet: real 0.011s user 0.003s sys 0.004s
I20250109 08:58:25.304656 23666 raft_consensus.cc:491] T 77597d68423749e7b87f9f0db7a1ff3b P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:25.305133 23666 raft_consensus.cc:513] T 77597d68423749e7b87f9f0db7a1ff3b P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:25.305570 23580 tablet_bootstrap.cc:492] T 11aa76a3a30643708fececed660a5250 P ed1efbce9ee3401e99a5260edeae4e7c: Bootstrap starting.
I20250109 08:58:25.306064 23611 leader_election.cc:290] T 1365130ef97c4581b623c22c39753831 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691), cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551)
I20250109 08:58:25.307020 23666 leader_election.cc:290] T 77597d68423749e7b87f9f0db7a1ff3b P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551), 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829)
I20250109 08:58:25.307829 23451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1365130ef97c4581b623c22c39753831" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc" is_pre_election: true
I20250109 08:58:25.308472 23451 raft_consensus.cc:2463] T 1365130ef97c4581b623c22c39753831 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed1efbce9ee3401e99a5260edeae4e7c in term 0.
I20250109 08:58:25.309173 23301 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1365130ef97c4581b623c22c39753831" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada" is_pre_election: true
I20250109 08:58:25.310297 23528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77597d68423749e7b87f9f0db7a1ff3b" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" is_pre_election: true
I20250109 08:58:25.311765 23580 tablet_bootstrap.cc:654] T 11aa76a3a30643708fececed660a5250 P ed1efbce9ee3401e99a5260edeae4e7c: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:25.312594 23346 leader_election.cc:304] T 1365130ef97c4581b623c22c39753831 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf55383bef6649eb9e78cb2d2512dedc, ed1efbce9ee3401e99a5260edeae4e7c; no voters:
W20250109 08:58:25.314100 23269 leader_election.cc:343] T 77597d68423749e7b87f9f0db7a1ff3b P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829): Illegal state: must be running to vote when last-logged opid is not known
W20250109 08:58:25.316447 23346 leader_election.cc:343] T 1365130ef97c4581b623c22c39753831 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:25.316963 23877 raft_consensus.cc:2798] T 1365130ef97c4581b623c22c39753831 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:58:25.317351 23877 raft_consensus.cc:491] T 1365130ef97c4581b623c22c39753831 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:58:25.317659 23877 raft_consensus.cc:3054] T 1365130ef97c4581b623c22c39753831 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:25.319518 23451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77597d68423749e7b87f9f0db7a1ff3b" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc" is_pre_election: true
I20250109 08:58:25.320159 23451 raft_consensus.cc:2463] T 77597d68423749e7b87f9f0db7a1ff3b P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 235ad6af04c34c88a5261daa3604fada in term 0.
I20250109 08:58:25.321331 23271 leader_election.cc:304] T 77597d68423749e7b87f9f0db7a1ff3b P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 235ad6af04c34c88a5261daa3604fada, cf55383bef6649eb9e78cb2d2512dedc; no voters: 05a74e1eb2474b3d8b3a2da1ef127e01
I20250109 08:58:25.322407 23902 raft_consensus.cc:2798] T 77597d68423749e7b87f9f0db7a1ff3b P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:58:25.322754 23902 raft_consensus.cc:491] T 77597d68423749e7b87f9f0db7a1ff3b P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:58:25.323078 23902 raft_consensus.cc:3054] T 77597d68423749e7b87f9f0db7a1ff3b P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:25.320184 23611 raft_consensus.cc:491] T d714271c7ea64c0b8b13fcf63591ec07 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:25.331476 23611 raft_consensus.cc:513] T d714271c7ea64c0b8b13fcf63591ec07 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:25.334498 23301 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d714271c7ea64c0b8b13fcf63591ec07" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada" is_pre_election: true
I20250109 08:58:25.335124 23301 raft_consensus.cc:2463] T d714271c7ea64c0b8b13fcf63591ec07 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed1efbce9ee3401e99a5260edeae4e7c in term 0.
I20250109 08:58:25.336472 23346 leader_election.cc:304] T d714271c7ea64c0b8b13fcf63591ec07 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 235ad6af04c34c88a5261daa3604fada, ed1efbce9ee3401e99a5260edeae4e7c; no voters:
I20250109 08:58:25.337250 23658 raft_consensus.cc:2798] T d714271c7ea64c0b8b13fcf63591ec07 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:58:25.337646 23658 raft_consensus.cc:491] T d714271c7ea64c0b8b13fcf63591ec07 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:58:25.338229 23658 raft_consensus.cc:3054] T d714271c7ea64c0b8b13fcf63591ec07 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:25.337592 23572 raft_consensus.cc:357] T 1f9c6525267145d6bb73dde32ed9b7d4 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:25.338913 23572 raft_consensus.cc:383] T 1f9c6525267145d6bb73dde32ed9b7d4 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:25.339169 23572 raft_consensus.cc:738] T 1f9c6525267145d6bb73dde32ed9b7d4 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf55383bef6649eb9e78cb2d2512dedc, State: Initialized, Role: FOLLOWER
I20250109 08:58:25.339747 23572 consensus_queue.cc:260] T 1f9c6525267145d6bb73dde32ed9b7d4 P cf55383bef6649eb9e78cb2d2512dedc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:25.341892 23572 ts_tablet_manager.cc:1428] T 1f9c6525267145d6bb73dde32ed9b7d4 P cf55383bef6649eb9e78cb2d2512dedc: Time spent starting tablet: real 0.038s user 0.002s sys 0.003s
I20250109 08:58:25.342705 23572 tablet_bootstrap.cc:492] T ecc833ad1afa4079a75e41b1a807c923 P cf55383bef6649eb9e78cb2d2512dedc: Bootstrap starting.
I20250109 08:58:25.345415 23583 raft_consensus.cc:491] T d714271c7ea64c0b8b13fcf63591ec07 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:25.345870 23583 raft_consensus.cc:513] T d714271c7ea64c0b8b13fcf63591ec07 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:25.347554 23583 leader_election.cc:290] T d714271c7ea64c0b8b13fcf63591ec07 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691), ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365)
I20250109 08:58:25.347889 23572 tablet_bootstrap.cc:654] T ecc833ad1afa4079a75e41b1a807c923 P cf55383bef6649eb9e78cb2d2512dedc: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:25.349349 23301 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d714271c7ea64c0b8b13fcf63591ec07" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada" is_pre_election: true
I20250109 08:58:25.350044 23301 raft_consensus.cc:2463] T d714271c7ea64c0b8b13fcf63591ec07 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf55383bef6649eb9e78cb2d2512dedc in term 0.
I20250109 08:58:25.350710 23574 tablet_bootstrap.cc:492] T bfb140bb3a8b44bd8b4e1fcf0946162b P 235ad6af04c34c88a5261daa3604fada: No bootstrap required, opened a new log
I20250109 08:58:25.351228 23574 ts_tablet_manager.cc:1397] T bfb140bb3a8b44bd8b4e1fcf0946162b P 235ad6af04c34c88a5261daa3604fada: Time spent bootstrapping tablet: real 0.057s user 0.010s sys 0.000s
I20250109 08:58:25.351404 23420 leader_election.cc:304] T d714271c7ea64c0b8b13fcf63591ec07 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 235ad6af04c34c88a5261daa3604fada, cf55383bef6649eb9e78cb2d2512dedc; no voters:
I20250109 08:58:25.353457 23379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d714271c7ea64c0b8b13fcf63591ec07" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" is_pre_election: true
I20250109 08:58:25.353683 23574 raft_consensus.cc:357] T bfb140bb3a8b44bd8b4e1fcf0946162b P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:25.354429 23574 raft_consensus.cc:383] T bfb140bb3a8b44bd8b4e1fcf0946162b P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:25.354552 23583 raft_consensus.cc:2798] T d714271c7ea64c0b8b13fcf63591ec07 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:58:25.355098 23583 raft_consensus.cc:491] T d714271c7ea64c0b8b13fcf63591ec07 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:58:25.354705 23574 raft_consensus.cc:738] T bfb140bb3a8b44bd8b4e1fcf0946162b P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Initialized, Role: FOLLOWER
I20250109 08:58:25.357254 23583 raft_consensus.cc:3054] T d714271c7ea64c0b8b13fcf63591ec07 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:25.357630 23574 consensus_queue.cc:260] T bfb140bb3a8b44bd8b4e1fcf0946162b P 235ad6af04c34c88a5261daa3604fada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:25.360159 23574 ts_tablet_manager.cc:1428] T bfb140bb3a8b44bd8b4e1fcf0946162b P 235ad6af04c34c88a5261daa3604fada: Time spent starting tablet: real 0.009s user 0.005s sys 0.001s
I20250109 08:58:25.360987 23574 tablet_bootstrap.cc:492] T ee703ea575234c659da04f21ef3cdd70 P 235ad6af04c34c88a5261daa3604fada: Bootstrap starting.
I20250109 08:58:25.376538 23572 tablet_bootstrap.cc:492] T ecc833ad1afa4079a75e41b1a807c923 P cf55383bef6649eb9e78cb2d2512dedc: No bootstrap required, opened a new log
I20250109 08:58:25.377046 23572 ts_tablet_manager.cc:1397] T ecc833ad1afa4079a75e41b1a807c923 P cf55383bef6649eb9e78cb2d2512dedc: Time spent bootstrapping tablet: real 0.035s user 0.010s sys 0.000s
I20250109 08:58:25.377468 23658 raft_consensus.cc:513] T d714271c7ea64c0b8b13fcf63591ec07 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:25.378931 23379 raft_consensus.cc:2388] T d714271c7ea64c0b8b13fcf63591ec07 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf55383bef6649eb9e78cb2d2512dedc in current term 1: Already voted for candidate ed1efbce9ee3401e99a5260edeae4e7c in this term.
I20250109 08:58:25.379314 23572 raft_consensus.cc:357] T ecc833ad1afa4079a75e41b1a807c923 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:25.380012 23572 raft_consensus.cc:383] T ecc833ad1afa4079a75e41b1a807c923 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:25.380254 23572 raft_consensus.cc:738] T ecc833ad1afa4079a75e41b1a807c923 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf55383bef6649eb9e78cb2d2512dedc, State: Initialized, Role: FOLLOWER
I20250109 08:58:25.382579 23572 consensus_queue.cc:260] T ecc833ad1afa4079a75e41b1a807c923 P cf55383bef6649eb9e78cb2d2512dedc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:25.384754 23572 ts_tablet_manager.cc:1428] T ecc833ad1afa4079a75e41b1a807c923 P cf55383bef6649eb9e78cb2d2512dedc: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20250109 08:58:25.385564 23572 tablet_bootstrap.cc:492] T 1ac4317290b24209a76d2e19e68ee45b P cf55383bef6649eb9e78cb2d2512dedc: Bootstrap starting.
I20250109 08:58:25.386785 23580 tablet_bootstrap.cc:492] T 11aa76a3a30643708fececed660a5250 P ed1efbce9ee3401e99a5260edeae4e7c: No bootstrap required, opened a new log
I20250109 08:58:25.386658 23666 consensus_queue.cc:1035] T f1f9274aeda748f493f5752814fb262d P 235ad6af04c34c88a5261daa3604fada [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:58:25.392779 23452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d714271c7ea64c0b8b13fcf63591ec07" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc"
I20250109 08:58:25.388809 23301 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d714271c7ea64c0b8b13fcf63591ec07" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada"
I20250109 08:58:25.393071 23902 raft_consensus.cc:513] T 77597d68423749e7b87f9f0db7a1ff3b P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:25.393792 23301 raft_consensus.cc:3054] T d714271c7ea64c0b8b13fcf63591ec07 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:25.389550 23894 raft_consensus.cc:491] T 950e07e9cc794b8ab9ec5bd97d7e8e59 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:25.390574 23572 tablet_bootstrap.cc:654] T 1ac4317290b24209a76d2e19e68ee45b P cf55383bef6649eb9e78cb2d2512dedc: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:25.390802 23574 tablet_bootstrap.cc:654] T ee703ea575234c659da04f21ef3cdd70 P 235ad6af04c34c88a5261daa3604fada: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:25.398805 23894 raft_consensus.cc:513] T 950e07e9cc794b8ab9ec5bd97d7e8e59 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:25.403321 23301 raft_consensus.cc:2463] T d714271c7ea64c0b8b13fcf63591ec07 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed1efbce9ee3401e99a5260edeae4e7c in term 1.
I20250109 08:58:25.404505 23346 leader_election.cc:304] T d714271c7ea64c0b8b13fcf63591ec07 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 235ad6af04c34c88a5261daa3604fada, ed1efbce9ee3401e99a5260edeae4e7c; no voters:
I20250109 08:58:25.413265 23934 raft_consensus.cc:2798] T d714271c7ea64c0b8b13fcf63591ec07 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:58:25.391512 23451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d714271c7ea64c0b8b13fcf63591ec07" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc" is_pre_election: true
I20250109 08:58:25.392115 23658 leader_election.cc:290] T d714271c7ea64c0b8b13fcf63591ec07 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 election: Requested vote from peers 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691), cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551)
I20250109 08:58:25.418156 23580 ts_tablet_manager.cc:1397] T 11aa76a3a30643708fececed660a5250 P ed1efbce9ee3401e99a5260edeae4e7c: Time spent bootstrapping tablet: real 0.113s user 0.018s sys 0.012s
I20250109 08:58:25.414919 23528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "950e07e9cc794b8ab9ec5bd97d7e8e59" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" is_pre_election: true
I20250109 08:58:25.414973 23305 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "950e07e9cc794b8ab9ec5bd97d7e8e59" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada" is_pre_election: true
W20250109 08:58:25.421059 23418 leader_election.cc:343] T 950e07e9cc794b8ab9ec5bd97d7e8e59 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829): Illegal state: must be running to vote when last-logged opid is not known
W20250109 08:58:25.421819 23420 leader_election.cc:343] T 950e07e9cc794b8ab9ec5bd97d7e8e59 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:25.391283 23611 leader_election.cc:290] T d714271c7ea64c0b8b13fcf63591ec07 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691), cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551)
I20250109 08:58:25.391299 23583 raft_consensus.cc:513] T d714271c7ea64c0b8b13fcf63591ec07 P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:25.423540 23420 leader_election.cc:304] T 950e07e9cc794b8ab9ec5bd97d7e8e59 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf55383bef6649eb9e78cb2d2512dedc; no voters: 05a74e1eb2474b3d8b3a2da1ef127e01, 235ad6af04c34c88a5261daa3604fada
I20250109 08:58:25.424746 23873 raft_consensus.cc:491] T 060daed1d3334c6188413dd381b82bf9 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:25.424861 23675 raft_consensus.cc:2743] T 950e07e9cc794b8ab9ec5bd97d7e8e59 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:58:25.413929 23894 leader_election.cc:290] T 950e07e9cc794b8ab9ec5bd97d7e8e59 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829), 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691)
I20250109 08:58:25.418215 23877 raft_consensus.cc:513] T 1365130ef97c4581b623c22c39753831 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:25.425268 23873 raft_consensus.cc:513] T 060daed1d3334c6188413dd381b82bf9 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:25.426885 23934 raft_consensus.cc:695] T d714271c7ea64c0b8b13fcf63591ec07 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 LEADER]: Becoming Leader. State: Replica: ed1efbce9ee3401e99a5260edeae4e7c, State: Running, Role: LEADER
I20250109 08:58:25.428593 23528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "060daed1d3334c6188413dd381b82bf9" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" is_pre_election: true
I20250109 08:58:25.429350 23528 raft_consensus.cc:2463] T 060daed1d3334c6188413dd381b82bf9 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 235ad6af04c34c88a5261daa3604fada in term 0.
I20250109 08:58:25.424909 23580 raft_consensus.cc:357] T 11aa76a3a30643708fececed660a5250 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
W20250109 08:58:25.432229 23494 consensus_peers.cc:487] T 0dab6c23ce444dda9427901a98f07b15 P 05a74e1eb2474b3d8b3a2da1ef127e01 -> Peer ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365): Couldn't send request to peer ed1efbce9ee3401e99a5260edeae4e7c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:25.430626 23269 leader_election.cc:304] T 060daed1d3334c6188413dd381b82bf9 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01, 235ad6af04c34c88a5261daa3604fada; no voters:
I20250109 08:58:25.433226 23586 raft_consensus.cc:2798] T 060daed1d3334c6188413dd381b82bf9 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:58:25.433583 23586 raft_consensus.cc:491] T 060daed1d3334c6188413dd381b82bf9 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:58:25.433861 23452 raft_consensus.cc:2388] T d714271c7ea64c0b8b13fcf63591ec07 P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate ed1efbce9ee3401e99a5260edeae4e7c in current term 1: Already voted for candidate cf55383bef6649eb9e78cb2d2512dedc in this term.
I20250109 08:58:25.436283 23583 leader_election.cc:290] T d714271c7ea64c0b8b13fcf63591ec07 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 election: Requested vote from peers 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691), ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365)
I20250109 08:58:25.438045 23305 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d714271c7ea64c0b8b13fcf63591ec07" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada"
I20250109 08:58:25.438884 23305 raft_consensus.cc:2388] T d714271c7ea64c0b8b13fcf63591ec07 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate cf55383bef6649eb9e78cb2d2512dedc in current term 1: Already voted for candidate ed1efbce9ee3401e99a5260edeae4e7c in this term.
I20250109 08:58:25.440088 23305 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1365130ef97c4581b623c22c39753831" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada"
I20250109 08:58:25.440474 23379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d714271c7ea64c0b8b13fcf63591ec07" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c"
I20250109 08:58:25.441967 23420 leader_election.cc:304] T d714271c7ea64c0b8b13fcf63591ec07 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf55383bef6649eb9e78cb2d2512dedc; no voters: 235ad6af04c34c88a5261daa3604fada, ed1efbce9ee3401e99a5260edeae4e7c
I20250109 08:58:25.432440 23580 raft_consensus.cc:383] T 11aa76a3a30643708fececed660a5250 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:25.442915 23580 raft_consensus.cc:738] T 11aa76a3a30643708fececed660a5250 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed1efbce9ee3401e99a5260edeae4e7c, State: Initialized, Role: FOLLOWER
I20250109 08:58:25.442103 23452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1365130ef97c4581b623c22c39753831" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc"
I20250109 08:58:25.444487 23452 raft_consensus.cc:3054] T 1365130ef97c4581b623c22c39753831 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:25.444161 23894 raft_consensus.cc:2743] T d714271c7ea64c0b8b13fcf63591ec07 P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20250109 08:58:25.446581 23301 raft_consensus.cc:1270] T cc9886333c0343798226d59758a1ea5e P 235ad6af04c34c88a5261daa3604fada [term 3 FOLLOWER]: Refusing update from remote peer 05a74e1eb2474b3d8b3a2da1ef127e01: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20250109 08:58:25.448082 23634 consensus_queue.cc:1035] T cc9886333c0343798226d59758a1ea5e P 05a74e1eb2474b3d8b3a2da1ef127e01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:58:25.443562 23580 consensus_queue.cc:260] T 11aa76a3a30643708fececed660a5250 P ed1efbce9ee3401e99a5260edeae4e7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:25.451750 23611 raft_consensus.cc:491] T 11aa76a3a30643708fececed660a5250 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:25.452459 23580 ts_tablet_manager.cc:1428] T 11aa76a3a30643708fececed660a5250 P ed1efbce9ee3401e99a5260edeae4e7c: Time spent starting tablet: real 0.032s user 0.003s sys 0.003s
I20250109 08:58:25.452211 23611 raft_consensus.cc:513] T 11aa76a3a30643708fececed660a5250 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:25.453402 23580 tablet_bootstrap.cc:492] T 0dab6c23ce444dda9427901a98f07b15 P ed1efbce9ee3401e99a5260edeae4e7c: Bootstrap starting.
I20250109 08:58:25.455195 23528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11aa76a3a30643708fececed660a5250" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" is_pre_election: true
I20250109 08:58:25.455876 23528 raft_consensus.cc:2463] T 11aa76a3a30643708fececed660a5250 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed1efbce9ee3401e99a5260edeae4e7c in term 0.
I20250109 08:58:25.457013 23344 leader_election.cc:304] T 11aa76a3a30643708fececed660a5250 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01, ed1efbce9ee3401e99a5260edeae4e7c; no voters:
I20250109 08:58:25.458602 23611 leader_election.cc:290] T 11aa76a3a30643708fececed660a5250 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551), 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829)
I20250109 08:58:25.459095 23611 raft_consensus.cc:2798] T 11aa76a3a30643708fececed660a5250 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:58:25.459442 23611 raft_consensus.cc:491] T 11aa76a3a30643708fececed660a5250 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:58:25.459481 23451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11aa76a3a30643708fececed660a5250" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc" is_pre_election: true
I20250109 08:58:25.459816 23611 raft_consensus.cc:3054] T 11aa76a3a30643708fececed660a5250 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:25.433890 23586 raft_consensus.cc:3054] T 060daed1d3334c6188413dd381b82bf9 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:25.460155 23451 raft_consensus.cc:2463] T 11aa76a3a30643708fececed660a5250 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed1efbce9ee3401e99a5260edeae4e7c in term 0.
I20250109 08:58:25.460659 23902 leader_election.cc:290] T 77597d68423749e7b87f9f0db7a1ff3b P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 election: Requested vote from peers cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551), 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829)
I20250109 08:58:25.433879 23877 leader_election.cc:290] T 1365130ef97c4581b623c22c39753831 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 election: Requested vote from peers 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691), cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551)
I20250109 08:58:25.464166 23873 leader_election.cc:290] T 060daed1d3334c6188413dd381b82bf9 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829), ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365)
W20250109 08:58:25.466286 23346 leader_election.cc:343] T 1365130ef97c4581b623c22c39753831 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:25.430470 23934 consensus_queue.cc:237] T d714271c7ea64c0b8b13fcf63591ec07 P ed1efbce9ee3401e99a5260edeae4e7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:25.469132 23586 raft_consensus.cc:513] T 060daed1d3334c6188413dd381b82bf9 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:25.471632 23528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77597d68423749e7b87f9f0db7a1ff3b" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01"
W20250109 08:58:25.472803 23269 leader_election.cc:343] T 77597d68423749e7b87f9f0db7a1ff3b P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:25.473197 23527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "060daed1d3334c6188413dd381b82bf9" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01"
I20250109 08:58:25.473775 23527 raft_consensus.cc:3054] T 060daed1d3334c6188413dd381b82bf9 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:25.474103 23586 leader_election.cc:290] T 060daed1d3334c6188413dd381b82bf9 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 election: Requested vote from peers 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829), ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365)
I20250109 08:58:25.459833 23452 raft_consensus.cc:2463] T 1365130ef97c4581b623c22c39753831 P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed1efbce9ee3401e99a5260edeae4e7c in term 1.
I20250109 08:58:25.479497 23346 leader_election.cc:304] T 1365130ef97c4581b623c22c39753831 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf55383bef6649eb9e78cb2d2512dedc, ed1efbce9ee3401e99a5260edeae4e7c; no voters: 235ad6af04c34c88a5261daa3604fada
I20250109 08:58:25.480136 23527 raft_consensus.cc:2463] T 060daed1d3334c6188413dd381b82bf9 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 235ad6af04c34c88a5261daa3604fada in term 1.
I20250109 08:58:25.496606 23877 raft_consensus.cc:2798] T 1365130ef97c4581b623c22c39753831 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:58:25.500200 23580 tablet_bootstrap.cc:654] T 0dab6c23ce444dda9427901a98f07b15 P ed1efbce9ee3401e99a5260edeae4e7c: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:25.501891 23572 tablet_bootstrap.cc:492] T 1ac4317290b24209a76d2e19e68ee45b P cf55383bef6649eb9e78cb2d2512dedc: No bootstrap required, opened a new log
I20250109 08:58:25.502502 23572 ts_tablet_manager.cc:1397] T 1ac4317290b24209a76d2e19e68ee45b P cf55383bef6649eb9e78cb2d2512dedc: Time spent bootstrapping tablet: real 0.117s user 0.008s sys 0.022s
I20250109 08:58:25.502236 23877 raft_consensus.cc:695] T 1365130ef97c4581b623c22c39753831 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 LEADER]: Becoming Leader. State: Replica: ed1efbce9ee3401e99a5260edeae4e7c, State: Running, Role: LEADER
I20250109 08:58:25.510036 23452 raft_consensus.cc:1270] T cc9886333c0343798226d59758a1ea5e P cf55383bef6649eb9e78cb2d2512dedc [term 3 FOLLOWER]: Refusing update from remote peer 05a74e1eb2474b3d8b3a2da1ef127e01: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20250109 08:58:25.502328 23269 leader_election.cc:304] T 060daed1d3334c6188413dd381b82bf9 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01, 235ad6af04c34c88a5261daa3604fada; no voters:
I20250109 08:58:25.512795 23893 consensus_queue.cc:1035] T cc9886333c0343798226d59758a1ea5e P 05a74e1eb2474b3d8b3a2da1ef127e01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:58:25.514050 23893 consensus_queue.cc:1035] T 0dab6c23ce444dda9427901a98f07b15 P 05a74e1eb2474b3d8b3a2da1ef127e01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:58:25.515242 23877 consensus_queue.cc:237] T 1365130ef97c4581b623c22c39753831 P ed1efbce9ee3401e99a5260edeae4e7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:25.516253 23451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77597d68423749e7b87f9f0db7a1ff3b" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc"
I20250109 08:58:25.516865 23451 raft_consensus.cc:3054] T 77597d68423749e7b87f9f0db7a1ff3b P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:25.516278 23586 raft_consensus.cc:2798] T 060daed1d3334c6188413dd381b82bf9 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:58:25.520911 23586 raft_consensus.cc:695] T 060daed1d3334c6188413dd381b82bf9 P 235ad6af04c34c88a5261daa3604fada [term 1 LEADER]: Becoming Leader. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Running, Role: LEADER
I20250109 08:58:25.521099 23611 raft_consensus.cc:513] T 11aa76a3a30643708fececed660a5250 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:25.521744 23586 consensus_queue.cc:237] T 060daed1d3334c6188413dd381b82bf9 P 235ad6af04c34c88a5261daa3604fada [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:25.527734 23611 leader_election.cc:290] T 11aa76a3a30643708fececed660a5250 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 election: Requested vote from peers cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551), 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829)
I20250109 08:58:25.542201 23574 tablet_bootstrap.cc:492] T ee703ea575234c659da04f21ef3cdd70 P 235ad6af04c34c88a5261daa3604fada: No bootstrap required, opened a new log
I20250109 08:58:25.542724 23574 ts_tablet_manager.cc:1397] T ee703ea575234c659da04f21ef3cdd70 P 235ad6af04c34c88a5261daa3604fada: Time spent bootstrapping tablet: real 0.182s user 0.027s sys 0.009s
I20250109 08:58:25.542934 23380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "060daed1d3334c6188413dd381b82bf9" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" is_pre_election: true
I20250109 08:58:25.548544 23574 raft_consensus.cc:357] T ee703ea575234c659da04f21ef3cdd70 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:25.549425 23574 raft_consensus.cc:383] T ee703ea575234c659da04f21ef3cdd70 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:25.549783 23574 raft_consensus.cc:738] T ee703ea575234c659da04f21ef3cdd70 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Initialized, Role: FOLLOWER
I20250109 08:58:25.550387 23574 consensus_queue.cc:260] T ee703ea575234c659da04f21ef3cdd70 P 235ad6af04c34c88a5261daa3604fada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:25.593393 23574 ts_tablet_manager.cc:1428] T ee703ea575234c659da04f21ef3cdd70 P 235ad6af04c34c88a5261daa3604fada: Time spent starting tablet: real 0.050s user 0.008s sys 0.000s
I20250109 08:58:25.594816 23574 tablet_bootstrap.cc:492] T 444507f7812441ceaeebc44bc8f06293 P 235ad6af04c34c88a5261daa3604fada: Bootstrap starting.
I20250109 08:58:25.595118 23527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11aa76a3a30643708fececed660a5250" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01"
I20250109 08:58:25.595798 23527 raft_consensus.cc:3054] T 11aa76a3a30643708fececed660a5250 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:25.598464 23878 raft_consensus.cc:491] T d87d3b4755344b82b0aec7ffd1ce785c P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 05a74e1eb2474b3d8b3a2da1ef127e01)
I20250109 08:58:25.598997 23878 raft_consensus.cc:513] T d87d3b4755344b82b0aec7ffd1ce785c P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:25.599251 23377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "060daed1d3334c6188413dd381b82bf9" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c"
I20250109 08:58:25.600283 23574 tablet_bootstrap.cc:654] T 444507f7812441ceaeebc44bc8f06293 P 235ad6af04c34c88a5261daa3604fada: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:25.603648 23527 raft_consensus.cc:2463] T 11aa76a3a30643708fececed660a5250 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed1efbce9ee3401e99a5260edeae4e7c in term 1.
I20250109 08:58:25.646653 23569 tablet_bootstrap.cc:492] T 950e07e9cc794b8ab9ec5bd97d7e8e59 P 05a74e1eb2474b3d8b3a2da1ef127e01: No bootstrap required, opened a new log
I20250109 08:58:25.647215 23569 ts_tablet_manager.cc:1397] T 950e07e9cc794b8ab9ec5bd97d7e8e59 P 05a74e1eb2474b3d8b3a2da1ef127e01: Time spent bootstrapping tablet: real 0.360s user 0.069s sys 0.080s
I20250109 08:58:25.649557 23666 raft_consensus.cc:491] T ee703ea575234c659da04f21ef3cdd70 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:25.650143 23666 raft_consensus.cc:513] T ee703ea575234c659da04f21ef3cdd70 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:25.651510 23569 raft_consensus.cc:357] T 950e07e9cc794b8ab9ec5bd97d7e8e59 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:25.652274 23569 raft_consensus.cc:383] T 950e07e9cc794b8ab9ec5bd97d7e8e59 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:25.655946 23569 raft_consensus.cc:738] T 950e07e9cc794b8ab9ec5bd97d7e8e59 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05a74e1eb2474b3d8b3a2da1ef127e01, State: Initialized, Role: FOLLOWER
I20250109 08:58:25.656594 23569 consensus_queue.cc:260] T 950e07e9cc794b8ab9ec5bd97d7e8e59 P 05a74e1eb2474b3d8b3a2da1ef127e01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:25.652211 23666 leader_election.cc:290] T ee703ea575234c659da04f21ef3cdd70 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829), ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365)
I20250109 08:58:25.681860 23580 tablet_bootstrap.cc:492] T 0dab6c23ce444dda9427901a98f07b15 P ed1efbce9ee3401e99a5260edeae4e7c: No bootstrap required, opened a new log
I20250109 08:58:25.682431 23580 ts_tablet_manager.cc:1397] T 0dab6c23ce444dda9427901a98f07b15 P ed1efbce9ee3401e99a5260edeae4e7c: Time spent bootstrapping tablet: real 0.229s user 0.023s sys 0.029s
I20250109 08:58:25.683823 23449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11aa76a3a30643708fececed660a5250" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc"
I20250109 08:58:25.672891 23344 leader_election.cc:304] T 11aa76a3a30643708fececed660a5250 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01, ed1efbce9ee3401e99a5260edeae4e7c; no voters:
I20250109 08:58:25.685715 23877 raft_consensus.cc:2798] T 11aa76a3a30643708fececed660a5250 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:58:25.686173 23877 raft_consensus.cc:695] T 11aa76a3a30643708fececed660a5250 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 LEADER]: Becoming Leader. State: Replica: ed1efbce9ee3401e99a5260edeae4e7c, State: Running, Role: LEADER
I20250109 08:58:25.687078 23878 leader_election.cc:290] T d87d3b4755344b82b0aec7ffd1ce785c P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829), 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691)
I20250109 08:58:25.687008 23877 consensus_queue.cc:237] T 11aa76a3a30643708fececed660a5250 P ed1efbce9ee3401e99a5260edeae4e7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:25.688740 23527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee703ea575234c659da04f21ef3cdd70" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" is_pre_election: true
W20250109 08:58:25.663756 23271 leader_election.cc:343] T 060daed1d3334c6188413dd381b82bf9 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:25.699573 23580 raft_consensus.cc:357] T 0dab6c23ce444dda9427901a98f07b15 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:25.701730 23580 raft_consensus.cc:383] T 0dab6c23ce444dda9427901a98f07b15 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:25.702028 23580 raft_consensus.cc:738] T 0dab6c23ce444dda9427901a98f07b15 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed1efbce9ee3401e99a5260edeae4e7c, State: Initialized, Role: FOLLOWER
I20250109 08:58:25.702653 23580 consensus_queue.cc:260] T 0dab6c23ce444dda9427901a98f07b15 P ed1efbce9ee3401e99a5260edeae4e7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:25.712239 23572 raft_consensus.cc:357] T 1ac4317290b24209a76d2e19e68ee45b P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:25.713016 23572 raft_consensus.cc:383] T 1ac4317290b24209a76d2e19e68ee45b P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:25.713572 23572 raft_consensus.cc:738] T 1ac4317290b24209a76d2e19e68ee45b P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf55383bef6649eb9e78cb2d2512dedc, State: Initialized, Role: FOLLOWER
W20250109 08:58:25.715036 23494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.226.193:42691, user_credentials={real_user=slave}} blocked reactor thread for 87691.2us
I20250109 08:58:25.714257 23572 consensus_queue.cc:260] T 1ac4317290b24209a76d2e19e68ee45b P cf55383bef6649eb9e78cb2d2512dedc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:25.700820 23449 raft_consensus.cc:3054] T 11aa76a3a30643708fececed660a5250 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:25.721329 23580 ts_tablet_manager.cc:1428] T 0dab6c23ce444dda9427901a98f07b15 P ed1efbce9ee3401e99a5260edeae4e7c: Time spent starting tablet: real 0.039s user 0.005s sys 0.001s
I20250109 08:58:25.722486 23580 tablet_bootstrap.cc:492] T 060daed1d3334c6188413dd381b82bf9 P ed1efbce9ee3401e99a5260edeae4e7c: Bootstrap starting.
I20250109 08:58:25.723971 23569 ts_tablet_manager.cc:1428] T 950e07e9cc794b8ab9ec5bd97d7e8e59 P 05a74e1eb2474b3d8b3a2da1ef127e01: Time spent starting tablet: real 0.076s user 0.012s sys 0.009s
W20250109 08:58:25.727485 23269 leader_election.cc:343] T ee703ea575234c659da04f21ef3cdd70 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:25.730695 23569 tablet_bootstrap.cc:492] T fc1038255dd64cb5ba2425037d22f1f7 P 05a74e1eb2474b3d8b3a2da1ef127e01: Bootstrap starting.
I20250109 08:58:25.732515 23451 raft_consensus.cc:2463] T 77597d68423749e7b87f9f0db7a1ff3b P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 235ad6af04c34c88a5261daa3604fada in term 1.
I20250109 08:58:25.735216 23658 raft_consensus.cc:491] T 1ac4317290b24209a76d2e19e68ee45b P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:25.735682 23658 raft_consensus.cc:513] T 1ac4317290b24209a76d2e19e68ee45b P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:25.708647 23937 raft_consensus.cc:491] T 5f0d500a91b54851bda52e5ccf741fd6 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20250109 08:58:25.722064 23271 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.226.194:41365, user_credentials={real_user=slave}} blocked reactor thread for 60744.7us
W20250109 08:58:25.754839 23271 leader_election.cc:343] T 060daed1d3334c6188413dd381b82bf9 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:25.756433 23658 leader_election.cc:290] T 1ac4317290b24209a76d2e19e68ee45b P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551), 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829)
I20250109 08:58:25.757257 23583 raft_consensus.cc:491] T 1f9c6525267145d6bb73dde32ed9b7d4 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:25.757751 23583 raft_consensus.cc:513] T 1f9c6525267145d6bb73dde32ed9b7d4 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:25.759310 23675 raft_consensus.cc:491] T ecc833ad1afa4079a75e41b1a807c923 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:25.765616 23675 raft_consensus.cc:513] T ecc833ad1afa4079a75e41b1a807c923 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:25.751554 23937 raft_consensus.cc:513] T 5f0d500a91b54851bda52e5ccf741fd6 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:25.767554 23675 leader_election.cc:290] T ecc833ad1afa4079a75e41b1a807c923 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365), 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829)
I20250109 08:58:25.775427 23583 leader_election.cc:290] T 1f9c6525267145d6bb73dde32ed9b7d4 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365), 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829)
I20250109 08:58:25.795648 23572 ts_tablet_manager.cc:1428] T 1ac4317290b24209a76d2e19e68ee45b P cf55383bef6649eb9e78cb2d2512dedc: Time spent starting tablet: real 0.292s user 0.007s sys 0.000s
I20250109 08:58:25.796820 23572 tablet_bootstrap.cc:492] T 8d3947300cfc4523acdab19fc992f26a P cf55383bef6649eb9e78cb2d2512dedc: Bootstrap starting.
I20250109 08:58:25.768707 23937 leader_election.cc:290] T 5f0d500a91b54851bda52e5ccf741fd6 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551), 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829)
I20250109 08:58:25.811477 23580 tablet_bootstrap.cc:654] T 060daed1d3334c6188413dd381b82bf9 P ed1efbce9ee3401e99a5260edeae4e7c: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:25.794409 23305 raft_consensus.cc:3054] T 5f0d500a91b54851bda52e5ccf741fd6 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:25.762310 23767 consensus_queue.cc:1035] T fc1038255dd64cb5ba2425037d22f1f7 P 235ad6af04c34c88a5261daa3604fada [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.051s
I20250109 08:58:25.838759 23528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d87d3b4755344b82b0aec7ffd1ce785c" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" is_pre_election: true
I20250109 08:58:25.840539 23936 raft_consensus.cc:491] T bfb140bb3a8b44bd8b4e1fcf0946162b P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:25.852742 23574 tablet_bootstrap.cc:492] T 444507f7812441ceaeebc44bc8f06293 P 235ad6af04c34c88a5261daa3604fada: No bootstrap required, opened a new log
I20250109 08:58:25.949386 23574 ts_tablet_manager.cc:1397] T 444507f7812441ceaeebc44bc8f06293 P 235ad6af04c34c88a5261daa3604fada: Time spent bootstrapping tablet: real 0.355s user 0.023s sys 0.000s
W20250109 08:58:25.851631 23269 consensus_peers.cc:487] T fc1038255dd64cb5ba2425037d22f1f7 P 235ad6af04c34c88a5261daa3604fada -> Peer 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829): Couldn't send request to peer 05a74e1eb2474b3d8b3a2da1ef127e01. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20250109 08:58:25.950485 23947 log.cc:926] Time spent T d714271c7ea64c0b8b13fcf63591ec07 P ed1efbce9ee3401e99a5260edeae4e7c: Append to log took a long time: real 0.145s user 0.009s sys 0.000s
I20250109 08:58:25.961059 23574 raft_consensus.cc:357] T 444507f7812441ceaeebc44bc8f06293 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:26.027199 23574 raft_consensus.cc:383] T 444507f7812441ceaeebc44bc8f06293 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:26.027562 23574 raft_consensus.cc:738] T 444507f7812441ceaeebc44bc8f06293 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Initialized, Role: FOLLOWER
I20250109 08:58:26.028209 23574 consensus_queue.cc:260] T 444507f7812441ceaeebc44bc8f06293 P 235ad6af04c34c88a5261daa3604fada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:26.030050 23574 ts_tablet_manager.cc:1428] T 444507f7812441ceaeebc44bc8f06293 P 235ad6af04c34c88a5261daa3604fada: Time spent starting tablet: real 0.080s user 0.006s sys 0.000s
I20250109 08:58:26.030462 23955 consensus_queue.cc:1035] T 750dd1e1055f4570b0aecdda3ebe8772 P ed1efbce9ee3401e99a5260edeae4e7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:58:26.031009 23574 tablet_bootstrap.cc:492] T 8d3947300cfc4523acdab19fc992f26a P 235ad6af04c34c88a5261daa3604fada: Bootstrap starting.
W20250109 08:58:25.962704 23954 log.cc:926] Time spent T 11aa76a3a30643708fececed660a5250 P ed1efbce9ee3401e99a5260edeae4e7c: Append to log took a long time: real 0.109s user 0.000s sys 0.000s
W20250109 08:58:25.963416 23950 log.cc:926] Time spent T 060daed1d3334c6188413dd381b82bf9 P 235ad6af04c34c88a5261daa3604fada: Append to log took a long time: real 0.188s user 0.001s sys 0.000s
I20250109 08:58:25.966692 23569 tablet_bootstrap.cc:654] T fc1038255dd64cb5ba2425037d22f1f7 P 05a74e1eb2474b3d8b3a2da1ef127e01: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:25.972555 23380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee703ea575234c659da04f21ef3cdd70" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" is_pre_election: true
I20250109 08:58:25.982899 23527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ecc833ad1afa4079a75e41b1a807c923" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" is_pre_election: true
I20250109 08:58:25.984602 23524 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f0d500a91b54851bda52e5ccf741fd6" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" is_pre_election: true
I20250109 08:58:25.984719 23522 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9c6525267145d6bb73dde32ed9b7d4" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" is_pre_election: true
W20250109 08:58:25.957737 23269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.226.196:41829, user_credentials={real_user=slave}} blocked reactor thread for 111961us
I20250109 08:58:26.043839 23574 tablet_bootstrap.cc:654] T 8d3947300cfc4523acdab19fc992f26a P 235ad6af04c34c88a5261daa3604fada: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:25.986398 23202 catalog_manager.cc:5526] T 060daed1d3334c6188413dd381b82bf9 P 235ad6af04c34c88a5261daa3604fada reported cstate change: term changed from 0 to 1, leader changed from <none> to 235ad6af04c34c88a5261daa3604fada (127.19.226.193). New cstate: current_term: 1 leader_uuid: "235ad6af04c34c88a5261daa3604fada" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:25.992053 23449 raft_consensus.cc:2463] T 11aa76a3a30643708fececed660a5250 P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed1efbce9ee3401e99a5260edeae4e7c in term 1.
I20250109 08:58:25.995486 23528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ac4317290b24209a76d2e19e68ee45b" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" is_pre_election: true
I20250109 08:58:25.998616 23767 consensus_queue.cc:1035] T f1f9274aeda748f493f5752814fb262d P 235ad6af04c34c88a5261daa3604fada [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:58:25.999604 23271 leader_election.cc:304] T 77597d68423749e7b87f9f0db7a1ff3b P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 235ad6af04c34c88a5261daa3604fada, cf55383bef6649eb9e78cb2d2512dedc; no voters: 05a74e1eb2474b3d8b3a2da1ef127e01
I20250109 08:58:26.000206 23203 catalog_manager.cc:5526] T d714271c7ea64c0b8b13fcf63591ec07 P ed1efbce9ee3401e99a5260edeae4e7c reported cstate change: term changed from 0 to 1, leader changed from <none> to ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194). New cstate: current_term: 1 leader_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } health_report { overall_health: UNKNOWN } } }
W20250109 08:58:26.056993 23271 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.226.195:35551, user_credentials={real_user=slave}} blocked reactor thread for 57482.8us
I20250109 08:58:26.057207 23203 catalog_manager.cc:5526] T 11aa76a3a30643708fececed660a5250 P ed1efbce9ee3401e99a5260edeae4e7c reported cstate change: term changed from 0 to 1, leader changed from <none> to ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194). New cstate: current_term: 1 leader_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:26.058915 23203 catalog_manager.cc:5526] T 1365130ef97c4581b623c22c39753831 P ed1efbce9ee3401e99a5260edeae4e7c reported cstate change: term changed from 0 to 1, leader changed from <none> to ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194). New cstate: current_term: 1 leader_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:26.060122 23666 raft_consensus.cc:2798] T 77597d68423749e7b87f9f0db7a1ff3b P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:58:26.003695 23572 tablet_bootstrap.cc:654] T 8d3947300cfc4523acdab19fc992f26a P cf55383bef6649eb9e78cb2d2512dedc: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:26.060743 23666 raft_consensus.cc:695] T 77597d68423749e7b87f9f0db7a1ff3b P 235ad6af04c34c88a5261daa3604fada [term 1 LEADER]: Becoming Leader. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Running, Role: LEADER
I20250109 08:58:26.061563 23666 consensus_queue.cc:237] T 77597d68423749e7b87f9f0db7a1ff3b P 235ad6af04c34c88a5261daa3604fada [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:26.007570 23377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9c6525267145d6bb73dde32ed9b7d4" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" is_pre_election: true
I20250109 08:58:26.009444 23379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ecc833ad1afa4079a75e41b1a807c923" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" is_pre_election: true
I20250109 08:58:26.063400 23377 raft_consensus.cc:2463] T 1f9c6525267145d6bb73dde32ed9b7d4 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf55383bef6649eb9e78cb2d2512dedc in term 0.
I20250109 08:58:26.063575 23379 raft_consensus.cc:2388] T ecc833ad1afa4079a75e41b1a807c923 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf55383bef6649eb9e78cb2d2512dedc in current term 1: Already voted for candidate 05a74e1eb2474b3d8b3a2da1ef127e01 in this term.
I20250109 08:58:25.952294 23936 raft_consensus.cc:513] T bfb140bb3a8b44bd8b4e1fcf0946162b P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:26.065738 23936 leader_election.cc:290] T bfb140bb3a8b44bd8b4e1fcf0946162b P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551), 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829)
I20250109 08:58:26.067446 23766 raft_consensus.cc:491] T 950e07e9cc794b8ab9ec5bd97d7e8e59 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:26.067915 23766 raft_consensus.cc:513] T 950e07e9cc794b8ab9ec5bd97d7e8e59 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:26.069718 23766 leader_election.cc:290] T 950e07e9cc794b8ab9ec5bd97d7e8e59 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551), 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691)
I20250109 08:58:26.073091 23448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ac4317290b24209a76d2e19e68ee45b" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc" is_pre_election: true
I20250109 08:58:26.073808 23448 raft_consensus.cc:2463] T 1ac4317290b24209a76d2e19e68ee45b P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed1efbce9ee3401e99a5260edeae4e7c in term 0.
I20250109 08:58:26.075168 23961 raft_consensus.cc:491] T 1ac4317290b24209a76d2e19e68ee45b P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:26.075722 23961 raft_consensus.cc:513] T 1ac4317290b24209a76d2e19e68ee45b P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:26.077842 23961 leader_election.cc:290] T 1ac4317290b24209a76d2e19e68ee45b P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365), 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829)
I20250109 08:58:26.081519 23580 tablet_bootstrap.cc:492] T 060daed1d3334c6188413dd381b82bf9 P ed1efbce9ee3401e99a5260edeae4e7c: No bootstrap required, opened a new log
I20250109 08:58:26.082029 23580 ts_tablet_manager.cc:1397] T 060daed1d3334c6188413dd381b82bf9 P ed1efbce9ee3401e99a5260edeae4e7c: Time spent bootstrapping tablet: real 0.360s user 0.006s sys 0.005s
I20250109 08:58:26.082578 23379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ac4317290b24209a76d2e19e68ee45b" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" is_pre_election: true
I20250109 08:58:26.083245 23379 raft_consensus.cc:2463] T 1ac4317290b24209a76d2e19e68ee45b P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf55383bef6649eb9e78cb2d2512dedc in term 0.
I20250109 08:58:26.085322 23580 raft_consensus.cc:357] T 060daed1d3334c6188413dd381b82bf9 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:26.086071 23580 raft_consensus.cc:383] T 060daed1d3334c6188413dd381b82bf9 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:26.086364 23580 raft_consensus.cc:738] T 060daed1d3334c6188413dd381b82bf9 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed1efbce9ee3401e99a5260edeae4e7c, State: Initialized, Role: FOLLOWER
I20250109 08:58:26.086959 23580 consensus_queue.cc:260] T 060daed1d3334c6188413dd381b82bf9 P ed1efbce9ee3401e99a5260edeae4e7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:26.089494 23580 ts_tablet_manager.cc:1428] T 060daed1d3334c6188413dd381b82bf9 P ed1efbce9ee3401e99a5260edeae4e7c: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20250109 08:58:26.090435 23580 tablet_bootstrap.cc:492] T 29828d9befb845d5bd49a80e018502e3 P ed1efbce9ee3401e99a5260edeae4e7c: Bootstrap starting.
I20250109 08:58:26.122002 23300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d87d3b4755344b82b0aec7ffd1ce785c" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada" is_pre_election: true
I20250109 08:58:26.125605 23454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f0d500a91b54851bda52e5ccf741fd6" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc" is_pre_election: true
W20250109 08:58:26.128441 23418 leader_election.cc:343] T 1f9c6525267145d6bb73dde32ed9b7d4 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:26.135375 23569 tablet_bootstrap.cc:492] T fc1038255dd64cb5ba2425037d22f1f7 P 05a74e1eb2474b3d8b3a2da1ef127e01: No bootstrap required, opened a new log
I20250109 08:58:26.136018 23569 ts_tablet_manager.cc:1397] T fc1038255dd64cb5ba2425037d22f1f7 P 05a74e1eb2474b3d8b3a2da1ef127e01: Time spent bootstrapping tablet: real 0.405s user 0.014s sys 0.013s
I20250109 08:58:26.137136 23658 consensus_queue.cc:1035] T 750dd1e1055f4570b0aecdda3ebe8772 P ed1efbce9ee3401e99a5260edeae4e7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:58:26.137470 23346 leader_election.cc:304] T 1ac4317290b24209a76d2e19e68ee45b P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf55383bef6649eb9e78cb2d2512dedc, ed1efbce9ee3401e99a5260edeae4e7c; no voters:
W20250109 08:58:26.146318 23251 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20250109 08:58:26.149271 23574 tablet_bootstrap.cc:492] T 8d3947300cfc4523acdab19fc992f26a P 235ad6af04c34c88a5261daa3604fada: No bootstrap required, opened a new log
I20250109 08:58:26.149869 23574 ts_tablet_manager.cc:1397] T 8d3947300cfc4523acdab19fc992f26a P 235ad6af04c34c88a5261daa3604fada: Time spent bootstrapping tablet: real 0.119s user 0.009s sys 0.007s
I20250109 08:58:26.142385 23934 raft_consensus.cc:2798] T 1ac4317290b24209a76d2e19e68ee45b P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Leader pre-election won for term 1
W20250109 08:58:26.158776 23494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.226.193:42691, user_credentials={real_user=slave}} blocked reactor thread for 411623us
W20250109 08:58:26.150770 23271 leader_election.cc:343] T ee703ea575234c659da04f21ef3cdd70 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:26.161609 23930 consensus_queue.cc:1035] T d714271c7ea64c0b8b13fcf63591ec07 P ed1efbce9ee3401e99a5260edeae4e7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.012s
I20250109 08:58:26.156203 23934 raft_consensus.cc:491] T 1ac4317290b24209a76d2e19e68ee45b P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:58:26.141491 23420 leader_election.cc:304] T 1f9c6525267145d6bb73dde32ed9b7d4 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf55383bef6649eb9e78cb2d2512dedc, ed1efbce9ee3401e99a5260edeae4e7c; no voters: 05a74e1eb2474b3d8b3a2da1ef127e01
W20250109 08:58:26.155999 23344 leader_election.cc:343] T 1ac4317290b24209a76d2e19e68ee45b P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:26.175443 23524 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ac4317290b24209a76d2e19e68ee45b" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" is_pre_election: true
I20250109 08:58:26.164477 23271 leader_election.cc:304] T ee703ea575234c659da04f21ef3cdd70 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 235ad6af04c34c88a5261daa3604fada; no voters: 05a74e1eb2474b3d8b3a2da1ef127e01, ed1efbce9ee3401e99a5260edeae4e7c
I20250109 08:58:26.176910 23574 raft_consensus.cc:357] T 8d3947300cfc4523acdab19fc992f26a P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:26.177609 23574 raft_consensus.cc:383] T 8d3947300cfc4523acdab19fc992f26a P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:26.177856 23574 raft_consensus.cc:738] T 8d3947300cfc4523acdab19fc992f26a P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Initialized, Role: FOLLOWER
I20250109 08:58:26.178503 23937 raft_consensus.cc:2743] T ee703ea575234c659da04f21ef3cdd70 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:58:26.180608 23453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfb140bb3a8b44bd8b4e1fcf0946162b" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc" is_pre_election: true
I20250109 08:58:26.169215 23934 raft_consensus.cc:3054] T 1ac4317290b24209a76d2e19e68ee45b P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:26.183904 23452 raft_consensus.cc:3054] T ecc833ad1afa4079a75e41b1a807c923 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:26.185118 23574 consensus_queue.cc:260] T 8d3947300cfc4523acdab19fc992f26a P 235ad6af04c34c88a5261daa3604fada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:26.186300 23580 tablet_bootstrap.cc:654] T 29828d9befb845d5bd49a80e018502e3 P ed1efbce9ee3401e99a5260edeae4e7c: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:26.189767 23963 raft_consensus.cc:2798] T 1f9c6525267145d6bb73dde32ed9b7d4 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:58:26.189847 23449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "950e07e9cc794b8ab9ec5bd97d7e8e59" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc" is_pre_election: true
I20250109 08:58:26.190263 23963 raft_consensus.cc:491] T 1f9c6525267145d6bb73dde32ed9b7d4 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:58:26.190593 23449 raft_consensus.cc:2463] T 950e07e9cc794b8ab9ec5bd97d7e8e59 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05a74e1eb2474b3d8b3a2da1ef127e01 in term 0.
I20250109 08:58:26.200897 23574 ts_tablet_manager.cc:1428] T 8d3947300cfc4523acdab19fc992f26a P 235ad6af04c34c88a5261daa3604fada: Time spent starting tablet: real 0.051s user 0.006s sys 0.001s
I20250109 08:58:26.220139 23572 tablet_bootstrap.cc:492] T 8d3947300cfc4523acdab19fc992f26a P cf55383bef6649eb9e78cb2d2512dedc: No bootstrap required, opened a new log
I20250109 08:58:26.220772 23572 ts_tablet_manager.cc:1397] T 8d3947300cfc4523acdab19fc992f26a P cf55383bef6649eb9e78cb2d2512dedc: Time spent bootstrapping tablet: real 0.424s user 0.013s sys 0.018s
W20250109 08:58:26.222136 23271 leader_election.cc:343] T bfb140bb3a8b44bd8b4e1fcf0946162b P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:26.168485 23932 consensus_queue.cc:1035] T 1365130ef97c4581b623c22c39753831 P ed1efbce9ee3401e99a5260edeae4e7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.044s
I20250109 08:58:26.190640 23963 raft_consensus.cc:3054] T 1f9c6525267145d6bb73dde32ed9b7d4 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:26.233196 23574 tablet_bootstrap.cc:492] T 950e07e9cc794b8ab9ec5bd97d7e8e59 P 235ad6af04c34c88a5261daa3604fada: Bootstrap starting.
I20250109 08:58:26.229410 23494 leader_election.cc:304] T 950e07e9cc794b8ab9ec5bd97d7e8e59 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01, cf55383bef6649eb9e78cb2d2512dedc; no voters:
I20250109 08:58:26.193266 23634 consensus_queue.cc:1035] T ecc833ad1afa4079a75e41b1a807c923 P 05a74e1eb2474b3d8b3a2da1ef127e01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.043s
W20250109 08:58:26.218900 23420 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.226.194:41365, user_credentials={real_user=slave}} blocked reactor thread for 86355.4us
W20250109 08:58:26.239897 23418 leader_election.cc:343] T 1ac4317290b24209a76d2e19e68ee45b P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:26.245257 23420 leader_election.cc:304] T ecc833ad1afa4079a75e41b1a807c923 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf55383bef6649eb9e78cb2d2512dedc; no voters: 05a74e1eb2474b3d8b3a2da1ef127e01, ed1efbce9ee3401e99a5260edeae4e7c
I20250109 08:58:26.247177 23420 leader_election.cc:304] T 1ac4317290b24209a76d2e19e68ee45b P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf55383bef6649eb9e78cb2d2512dedc, ed1efbce9ee3401e99a5260edeae4e7c; no voters: 05a74e1eb2474b3d8b3a2da1ef127e01
I20250109 08:58:26.249213 23960 raft_consensus.cc:2798] T 950e07e9cc794b8ab9ec5bd97d7e8e59 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:58:26.245410 23376 raft_consensus.cc:3054] T 0dab6c23ce444dda9427901a98f07b15 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:26.263257 23956 raft_consensus.cc:2798] T 1ac4317290b24209a76d2e19e68ee45b P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:58:26.263712 23956 raft_consensus.cc:491] T 1ac4317290b24209a76d2e19e68ee45b P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:58:26.264048 23956 raft_consensus.cc:3054] T 1ac4317290b24209a76d2e19e68ee45b P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:26.265568 23569 raft_consensus.cc:357] T fc1038255dd64cb5ba2425037d22f1f7 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:26.282485 23569 raft_consensus.cc:383] T fc1038255dd64cb5ba2425037d22f1f7 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:26.282833 23569 raft_consensus.cc:738] T fc1038255dd64cb5ba2425037d22f1f7 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05a74e1eb2474b3d8b3a2da1ef127e01, State: Initialized, Role: FOLLOWER
I20250109 08:58:26.283490 23569 consensus_queue.cc:260] T fc1038255dd64cb5ba2425037d22f1f7 P 05a74e1eb2474b3d8b3a2da1ef127e01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:26.261917 23960 raft_consensus.cc:491] T 950e07e9cc794b8ab9ec5bd97d7e8e59 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:58:26.307181 23572 raft_consensus.cc:357] T 8d3947300cfc4523acdab19fc992f26a P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:26.308060 23572 raft_consensus.cc:383] T 8d3947300cfc4523acdab19fc992f26a P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:26.308338 23572 raft_consensus.cc:738] T 8d3947300cfc4523acdab19fc992f26a P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf55383bef6649eb9e78cb2d2512dedc, State: Initialized, Role: FOLLOWER
I20250109 08:58:26.309046 23572 consensus_queue.cc:260] T 8d3947300cfc4523acdab19fc992f26a P cf55383bef6649eb9e78cb2d2512dedc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:26.311636 23569 ts_tablet_manager.cc:1428] T fc1038255dd64cb5ba2425037d22f1f7 P 05a74e1eb2474b3d8b3a2da1ef127e01: Time spent starting tablet: real 0.175s user 0.006s sys 0.001s
I20250109 08:58:26.316462 23569 tablet_bootstrap.cc:492] T 1f9c6525267145d6bb73dde32ed9b7d4 P 05a74e1eb2474b3d8b3a2da1ef127e01: Bootstrap starting.
I20250109 08:58:26.295459 23960 raft_consensus.cc:3054] T 950e07e9cc794b8ab9ec5bd97d7e8e59 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:26.323611 23346 leader_election.cc:304] T d87d3b4755344b82b0aec7ffd1ce785c P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ed1efbce9ee3401e99a5260edeae4e7c; no voters: 05a74e1eb2474b3d8b3a2da1ef127e01, 235ad6af04c34c88a5261daa3604fada
I20250109 08:58:26.345218 23269 leader_election.cc:304] T 5f0d500a91b54851bda52e5ccf741fd6 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 235ad6af04c34c88a5261daa3604fada; no voters: 05a74e1eb2474b3d8b3a2da1ef127e01, cf55383bef6649eb9e78cb2d2512dedc
I20250109 08:58:26.348246 23586 raft_consensus.cc:2743] T 5f0d500a91b54851bda52e5ccf741fd6 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:58:26.349303 23586 consensus_queue.cc:1035] T 060daed1d3334c6188413dd381b82bf9 P 235ad6af04c34c88a5261daa3604fada [LEADER]: Connected to new peer: Peer: permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:58:26.350194 23580 tablet_bootstrap.cc:492] T 29828d9befb845d5bd49a80e018502e3 P ed1efbce9ee3401e99a5260edeae4e7c: No bootstrap required, opened a new log
I20250109 08:58:26.350672 23580 ts_tablet_manager.cc:1397] T 29828d9befb845d5bd49a80e018502e3 P ed1efbce9ee3401e99a5260edeae4e7c: Time spent bootstrapping tablet: real 0.260s user 0.031s sys 0.008s
I20250109 08:58:26.355769 23572 ts_tablet_manager.cc:1428] T 8d3947300cfc4523acdab19fc992f26a P cf55383bef6649eb9e78cb2d2512dedc: Time spent starting tablet: real 0.135s user 0.006s sys 0.000s
I20250109 08:58:26.356941 23572 tablet_bootstrap.cc:492] T bfb140bb3a8b44bd8b4e1fcf0946162b P cf55383bef6649eb9e78cb2d2512dedc: Bootstrap starting.
I20250109 08:58:26.363128 23572 tablet_bootstrap.cc:654] T bfb140bb3a8b44bd8b4e1fcf0946162b P cf55383bef6649eb9e78cb2d2512dedc: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:26.378979 23527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfb140bb3a8b44bd8b4e1fcf0946162b" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" is_pre_election: true
I20250109 08:58:26.347940 23937 raft_consensus.cc:491] T 444507f7812441ceaeebc44bc8f06293 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:26.380671 23937 raft_consensus.cc:513] T 444507f7812441ceaeebc44bc8f06293 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:26.382678 23937 leader_election.cc:290] T 444507f7812441ceaeebc44bc8f06293 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829), ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365)
I20250109 08:58:26.346846 23932 raft_consensus.cc:2743] T d87d3b4755344b82b0aec7ffd1ce785c P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20250109 08:58:26.392902 23346 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.226.193:42691, user_credentials={real_user=slave}} blocked reactor thread for 45557.5us
I20250109 08:58:26.393661 23569 tablet_bootstrap.cc:654] T 1f9c6525267145d6bb73dde32ed9b7d4 P 05a74e1eb2474b3d8b3a2da1ef127e01: Neither blocks nor log segments found. Creating new log.
W20250109 08:58:26.393738 23346 consensus_peers.cc:487] T 1365130ef97c4581b623c22c39753831 P ed1efbce9ee3401e99a5260edeae4e7c -> Peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Couldn't send request to peer 235ad6af04c34c88a5261daa3604fada. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:26.397975 23658 consensus_queue.cc:1035] T d714271c7ea64c0b8b13fcf63591ec07 P ed1efbce9ee3401e99a5260edeae4e7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:58:26.399439 23934 raft_consensus.cc:513] T 1ac4317290b24209a76d2e19e68ee45b P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:26.414207 23934 leader_election.cc:290] T 1ac4317290b24209a76d2e19e68ee45b P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 election: Requested vote from peers cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551), 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829)
I20250109 08:58:26.416188 23675 raft_consensus.cc:2743] T ecc833ad1afa4079a75e41b1a807c923 P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:58:26.416441 23527 raft_consensus.cc:3054] T fc1038255dd64cb5ba2425037d22f1f7 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:26.485543 23448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ac4317290b24209a76d2e19e68ee45b" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc"
I20250109 08:58:26.497285 23580 raft_consensus.cc:357] T 29828d9befb845d5bd49a80e018502e3 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:26.514093 23580 raft_consensus.cc:383] T 29828d9befb845d5bd49a80e018502e3 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:26.514415 23580 raft_consensus.cc:738] T 29828d9befb845d5bd49a80e018502e3 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed1efbce9ee3401e99a5260edeae4e7c, State: Initialized, Role: FOLLOWER
I20250109 08:58:26.514974 23580 consensus_queue.cc:260] T 29828d9befb845d5bd49a80e018502e3 P ed1efbce9ee3401e99a5260edeae4e7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:26.443882 23944 consensus_queue.cc:1035] T ecc833ad1afa4079a75e41b1a807c923 P 05a74e1eb2474b3d8b3a2da1ef127e01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.044s
W20250109 08:58:26.517714 23269 leader_election.cc:343] T bfb140bb3a8b44bd8b4e1fcf0946162b P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:26.518970 23269 leader_election.cc:304] T bfb140bb3a8b44bd8b4e1fcf0946162b P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 235ad6af04c34c88a5261daa3604fada; no voters: 05a74e1eb2474b3d8b3a2da1ef127e01, cf55383bef6649eb9e78cb2d2512dedc
I20250109 08:58:26.519968 23992 consensus_queue.cc:1035] T 11aa76a3a30643708fececed660a5250 P ed1efbce9ee3401e99a5260edeae4e7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.035s
I20250109 08:58:26.533511 23572 tablet_bootstrap.cc:492] T bfb140bb3a8b44bd8b4e1fcf0946162b P cf55383bef6649eb9e78cb2d2512dedc: No bootstrap required, opened a new log
I20250109 08:58:26.541113 23572 ts_tablet_manager.cc:1397] T bfb140bb3a8b44bd8b4e1fcf0946162b P cf55383bef6649eb9e78cb2d2512dedc: Time spent bootstrapping tablet: real 0.184s user 0.015s sys 0.021s
I20250109 08:58:26.526849 23380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "444507f7812441ceaeebc44bc8f06293" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" is_pre_election: true
I20250109 08:58:26.541116 23930 raft_consensus.cc:491] T 0dab6c23ce444dda9427901a98f07b15 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 05a74e1eb2474b3d8b3a2da1ef127e01)
I20250109 08:58:26.547019 23930 raft_consensus.cc:513] T 0dab6c23ce444dda9427901a98f07b15 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:26.546048 23958 raft_consensus.cc:491] T 29828d9befb845d5bd49a80e018502e3 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:26.547904 23958 raft_consensus.cc:513] T 29828d9befb845d5bd49a80e018502e3 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
W20250109 08:58:26.548527 23987 log.cc:926] Time spent T 750dd1e1055f4570b0aecdda3ebe8772 P ed1efbce9ee3401e99a5260edeae4e7c: Append to log took a long time: real 0.127s user 0.001s sys 0.000s
I20250109 08:58:26.549980 23958 leader_election.cc:290] T 29828d9befb845d5bd49a80e018502e3 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829), 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691)
I20250109 08:58:26.545815 23580 ts_tablet_manager.cc:1428] T 29828d9befb845d5bd49a80e018502e3 P ed1efbce9ee3401e99a5260edeae4e7c: Time spent starting tablet: real 0.195s user 0.005s sys 0.002s
I20250109 08:58:26.517187 23894 raft_consensus.cc:491] T 8d3947300cfc4523acdab19fc992f26a P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:26.557579 23580 tablet_bootstrap.cc:492] T 444507f7812441ceaeebc44bc8f06293 P ed1efbce9ee3401e99a5260edeae4e7c: Bootstrap starting.
I20250109 08:58:26.558653 23574 tablet_bootstrap.cc:654] T 950e07e9cc794b8ab9ec5bd97d7e8e59 P 235ad6af04c34c88a5261daa3604fada: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:26.559545 23994 raft_consensus.cc:491] T 060daed1d3334c6188413dd381b82bf9 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:26.560127 23994 raft_consensus.cc:513] T 060daed1d3334c6188413dd381b82bf9 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:26.575971 23994 leader_election.cc:290] T 060daed1d3334c6188413dd381b82bf9 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829), 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691)
I20250109 08:58:26.541522 23522 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "444507f7812441ceaeebc44bc8f06293" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" is_pre_election: true
I20250109 08:58:26.496398 23963 raft_consensus.cc:513] T 1f9c6525267145d6bb73dde32ed9b7d4 P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:26.587659 23521 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ac4317290b24209a76d2e19e68ee45b" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01"
I20250109 08:58:26.592355 23524 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29828d9befb845d5bd49a80e018502e3" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" is_pre_election: true
I20250109 08:58:26.586133 23522 raft_consensus.cc:2463] T 444507f7812441ceaeebc44bc8f06293 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 235ad6af04c34c88a5261daa3604fada in term 0.
I20250109 08:58:26.528913 23296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "950e07e9cc794b8ab9ec5bd97d7e8e59" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada" is_pre_election: true
I20250109 08:58:26.609863 23963 leader_election.cc:290] T 1f9c6525267145d6bb73dde32ed9b7d4 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 election: Requested vote from peers ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365), 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829)
I20250109 08:58:26.610266 23572 raft_consensus.cc:357] T bfb140bb3a8b44bd8b4e1fcf0946162b P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:26.616331 23572 raft_consensus.cc:383] T bfb140bb3a8b44bd8b4e1fcf0946162b P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:26.616665 23572 raft_consensus.cc:738] T bfb140bb3a8b44bd8b4e1fcf0946162b P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf55383bef6649eb9e78cb2d2512dedc, State: Initialized, Role: FOLLOWER
I20250109 08:58:26.617362 23572 consensus_queue.cc:260] T bfb140bb3a8b44bd8b4e1fcf0946162b P cf55383bef6649eb9e78cb2d2512dedc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:26.570852 23894 raft_consensus.cc:513] T 8d3947300cfc4523acdab19fc992f26a P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:26.620442 23376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9c6525267145d6bb73dde32ed9b7d4" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c"
W20250109 08:58:26.621563 23344 leader_election.cc:343] T 1ac4317290b24209a76d2e19e68ee45b P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829): Illegal state: must be running to vote when last-logged opid is not known
W20250109 08:58:26.622290 23344 leader_election.cc:343] T 29828d9befb845d5bd49a80e018502e3 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:26.621558 23376 raft_consensus.cc:3054] T 1f9c6525267145d6bb73dde32ed9b7d4 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:26.632259 23572 ts_tablet_manager.cc:1428] T bfb140bb3a8b44bd8b4e1fcf0946162b P cf55383bef6649eb9e78cb2d2512dedc: Time spent starting tablet: real 0.091s user 0.000s sys 0.006s
I20250109 08:58:26.633685 23572 tablet_bootstrap.cc:492] T 94f2be307f704c75bceb213e52307cfb P cf55383bef6649eb9e78cb2d2512dedc: Bootstrap starting.
I20250109 08:58:26.605059 23956 raft_consensus.cc:513] T 1ac4317290b24209a76d2e19e68ee45b P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:26.636469 23580 tablet_bootstrap.cc:654] T 444507f7812441ceaeebc44bc8f06293 P ed1efbce9ee3401e99a5260edeae4e7c: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:26.637154 23957 raft_consensus.cc:491] T bfb140bb3a8b44bd8b4e1fcf0946162b P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:26.637672 23957 raft_consensus.cc:513] T bfb140bb3a8b44bd8b4e1fcf0946162b P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:26.645690 23956 leader_election.cc:290] T 1ac4317290b24209a76d2e19e68ee45b P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 election: Requested vote from peers ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365), 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829)
I20250109 08:58:26.637039 23520 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9c6525267145d6bb73dde32ed9b7d4" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01"
I20250109 08:58:26.648358 23448 raft_consensus.cc:2388] T 1ac4317290b24209a76d2e19e68ee45b P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate ed1efbce9ee3401e99a5260edeae4e7c in current term 1: Already voted for candidate cf55383bef6649eb9e78cb2d2512dedc in this term.
I20250109 08:58:26.649375 23894 leader_election.cc:290] T 8d3947300cfc4523acdab19fc992f26a P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691), ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365)
W20250109 08:58:26.655767 23418 leader_election.cc:343] T 1f9c6525267145d6bb73dde32ed9b7d4 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:26.656991 23346 leader_election.cc:304] T 1ac4317290b24209a76d2e19e68ee45b P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ed1efbce9ee3401e99a5260edeae4e7c; no voters: 05a74e1eb2474b3d8b3a2da1ef127e01, cf55383bef6649eb9e78cb2d2512dedc
I20250109 08:58:26.658104 23375 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ac4317290b24209a76d2e19e68ee45b" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c"
I20250109 08:58:26.659938 23380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d3947300cfc4523acdab19fc992f26a" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" is_pre_election: true
I20250109 08:58:26.623754 23989 consensus_queue.cc:1035] T 11aa76a3a30643708fececed660a5250 P ed1efbce9ee3401e99a5260edeae4e7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.013s
I20250109 08:58:26.660225 23375 raft_consensus.cc:2388] T 1ac4317290b24209a76d2e19e68ee45b P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate cf55383bef6649eb9e78cb2d2512dedc in current term 1: Already voted for candidate ed1efbce9ee3401e99a5260edeae4e7c in this term.
W20250109 08:58:26.666412 23420 leader_election.cc:343] T 8d3947300cfc4523acdab19fc992f26a P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:26.677351 23960 raft_consensus.cc:513] T 950e07e9cc794b8ab9ec5bd97d7e8e59 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:26.677913 23930 leader_election.cc:290] T 0dab6c23ce444dda9427901a98f07b15 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551), 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829)
I20250109 08:58:26.631392 23521 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "060daed1d3334c6188413dd381b82bf9" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" is_pre_election: true
I20250109 08:58:26.639634 23957 leader_election.cc:290] T bfb140bb3a8b44bd8b4e1fcf0946162b P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829), 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691)
W20250109 08:58:26.694862 23269 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.226.196:41829, user_credentials={real_user=slave}} blocked reactor thread for 177180us
I20250109 08:58:26.701920 23569 tablet_bootstrap.cc:492] T 1f9c6525267145d6bb73dde32ed9b7d4 P 05a74e1eb2474b3d8b3a2da1ef127e01: No bootstrap required, opened a new log
I20250109 08:58:26.702484 23569 ts_tablet_manager.cc:1397] T 1f9c6525267145d6bb73dde32ed9b7d4 P 05a74e1eb2474b3d8b3a2da1ef127e01: Time spent bootstrapping tablet: real 0.386s user 0.037s sys 0.029s
I20250109 08:58:26.683297 23994 raft_consensus.cc:2743] T 1ac4317290b24209a76d2e19e68ee45b P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20250109 08:58:26.706586 23269 leader_election.cc:304] T 444507f7812441ceaeebc44bc8f06293 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01, 235ad6af04c34c88a5261daa3604fada; no voters:
W20250109 08:58:26.694294 23271 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.226.194:41365, user_credentials={real_user=slave}} blocked reactor thread for 245417us
I20250109 08:58:26.701252 23666 raft_consensus.cc:2743] T bfb140bb3a8b44bd8b4e1fcf0946162b P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:58:26.720360 23572 tablet_bootstrap.cc:654] T 94f2be307f704c75bceb213e52307cfb P cf55383bef6649eb9e78cb2d2512dedc: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:26.718992 23960 leader_election.cc:290] T 950e07e9cc794b8ab9ec5bd97d7e8e59 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 election: Requested vote from peers cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551), 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691)
I20250109 08:58:26.731552 23522 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0dab6c23ce444dda9427901a98f07b15" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" is_pre_election: true
W20250109 08:58:26.731554 24003 log.cc:926] Time spent T fc1038255dd64cb5ba2425037d22f1f7 P 05a74e1eb2474b3d8b3a2da1ef127e01: Append to log took a long time: real 0.105s user 0.001s sys 0.000s
I20250109 08:58:26.706965 23448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0dab6c23ce444dda9427901a98f07b15" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc" is_pre_election: true
I20250109 08:58:26.709759 23873 raft_consensus.cc:2798] T 444507f7812441ceaeebc44bc8f06293 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:58:26.746843 23569 raft_consensus.cc:357] T 1f9c6525267145d6bb73dde32ed9b7d4 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:26.747591 23569 raft_consensus.cc:383] T 1f9c6525267145d6bb73dde32ed9b7d4 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:26.747836 23569 raft_consensus.cc:738] T 1f9c6525267145d6bb73dde32ed9b7d4 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05a74e1eb2474b3d8b3a2da1ef127e01, State: Initialized, Role: FOLLOWER
W20250109 08:58:26.748004 23494 leader_election.cc:343] T 950e07e9cc794b8ab9ec5bd97d7e8e59 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:26.748479 23569 consensus_queue.cc:260] T 1f9c6525267145d6bb73dde32ed9b7d4 P 05a74e1eb2474b3d8b3a2da1ef127e01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:26.715184 23304 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29828d9befb845d5bd49a80e018502e3" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada" is_pre_election: true
I20250109 08:58:26.729020 23450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "950e07e9cc794b8ab9ec5bd97d7e8e59" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc"
I20250109 08:58:26.759097 23450 raft_consensus.cc:3054] T 950e07e9cc794b8ab9ec5bd97d7e8e59 P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:26.745723 23873 raft_consensus.cc:491] T 444507f7812441ceaeebc44bc8f06293 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:58:26.763198 23873 raft_consensus.cc:3054] T 444507f7812441ceaeebc44bc8f06293 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:26.746174 23666 raft_consensus.cc:491] T 8d3947300cfc4523acdab19fc992f26a P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:26.770567 23666 raft_consensus.cc:513] T 8d3947300cfc4523acdab19fc992f26a P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:26.772653 23666 leader_election.cc:290] T 8d3947300cfc4523acdab19fc992f26a P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365), cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551)
I20250109 08:58:26.773319 23300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "060daed1d3334c6188413dd381b82bf9" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada" is_pre_election: true
I20250109 08:58:26.791395 23346 leader_election.cc:304] T 060daed1d3334c6188413dd381b82bf9 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ed1efbce9ee3401e99a5260edeae4e7c; no voters: 05a74e1eb2474b3d8b3a2da1ef127e01, 235ad6af04c34c88a5261daa3604fada
I20250109 08:58:26.792560 23989 raft_consensus.cc:2743] T 060daed1d3334c6188413dd381b82bf9 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:58:26.797333 23574 tablet_bootstrap.cc:492] T 950e07e9cc794b8ab9ec5bd97d7e8e59 P 235ad6af04c34c88a5261daa3604fada: No bootstrap required, opened a new log
I20250109 08:58:26.797950 23574 ts_tablet_manager.cc:1397] T 950e07e9cc794b8ab9ec5bd97d7e8e59 P 235ad6af04c34c88a5261daa3604fada: Time spent bootstrapping tablet: real 0.565s user 0.023s sys 0.017s
I20250109 08:58:26.800859 23574 raft_consensus.cc:357] T 950e07e9cc794b8ab9ec5bd97d7e8e59 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:26.801616 23574 raft_consensus.cc:383] T 950e07e9cc794b8ab9ec5bd97d7e8e59 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:26.801879 23574 raft_consensus.cc:738] T 950e07e9cc794b8ab9ec5bd97d7e8e59 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Initialized, Role: FOLLOWER
I20250109 08:58:26.804643 23574 consensus_queue.cc:260] T 950e07e9cc794b8ab9ec5bd97d7e8e59 P 235ad6af04c34c88a5261daa3604fada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:26.805094 23569 ts_tablet_manager.cc:1428] T 1f9c6525267145d6bb73dde32ed9b7d4 P 05a74e1eb2474b3d8b3a2da1ef127e01: Time spent starting tablet: real 0.102s user 0.006s sys 0.001s
W20250109 08:58:26.808472 23346 leader_election.cc:343] T 29828d9befb845d5bd49a80e018502e3 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:26.809012 23346 leader_election.cc:304] T 29828d9befb845d5bd49a80e018502e3 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ed1efbce9ee3401e99a5260edeae4e7c; no voters: 05a74e1eb2474b3d8b3a2da1ef127e01, 235ad6af04c34c88a5261daa3604fada
I20250109 08:58:26.823354 23992 raft_consensus.cc:2743] T 29828d9befb845d5bd49a80e018502e3 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:58:26.838310 23299 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d3947300cfc4523acdab19fc992f26a" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada" is_pre_election: true
I20250109 08:58:26.840233 23299 raft_consensus.cc:2463] T 8d3947300cfc4523acdab19fc992f26a P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf55383bef6649eb9e78cb2d2512dedc in term 0.
I20250109 08:58:26.842738 23420 leader_election.cc:304] T 8d3947300cfc4523acdab19fc992f26a P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 235ad6af04c34c88a5261daa3604fada, cf55383bef6649eb9e78cb2d2512dedc; no voters: ed1efbce9ee3401e99a5260edeae4e7c
I20250109 08:58:26.881858 24004 raft_consensus.cc:2798] T 8d3947300cfc4523acdab19fc992f26a P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:58:26.832612 23344 leader_election.cc:304] T 0dab6c23ce444dda9427901a98f07b15 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ed1efbce9ee3401e99a5260edeae4e7c; no voters: 05a74e1eb2474b3d8b3a2da1ef127e01, cf55383bef6649eb9e78cb2d2512dedc
I20250109 08:58:26.882285 24004 raft_consensus.cc:491] T 8d3947300cfc4523acdab19fc992f26a P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:58:26.885583 23574 ts_tablet_manager.cc:1428] T 950e07e9cc794b8ab9ec5bd97d7e8e59 P 235ad6af04c34c88a5261daa3604fada: Time spent starting tablet: real 0.054s user 0.007s sys 0.000s
I20250109 08:58:26.889199 23574 tablet_bootstrap.cc:492] T 1365130ef97c4581b623c22c39753831 P 235ad6af04c34c88a5261daa3604fada: Bootstrap starting.
I20250109 08:58:26.885398 23296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfb140bb3a8b44bd8b4e1fcf0946162b" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada" is_pre_election: true
I20250109 08:58:26.889848 24004 raft_consensus.cc:3054] T 8d3947300cfc4523acdab19fc992f26a P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:26.898137 23296 raft_consensus.cc:2463] T bfb140bb3a8b44bd8b4e1fcf0946162b P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf55383bef6649eb9e78cb2d2512dedc in term 0.
I20250109 08:58:26.885056 23873 raft_consensus.cc:513] T 444507f7812441ceaeebc44bc8f06293 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:26.892444 23989 raft_consensus.cc:2743] T 0dab6c23ce444dda9427901a98f07b15 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20250109 08:58:26.898303 24010 log.cc:926] Time spent T ecc833ad1afa4079a75e41b1a807c923 P ed1efbce9ee3401e99a5260edeae4e7c: Append to log took a long time: real 0.063s user 0.001s sys 0.000s
I20250109 08:58:26.905826 23420 leader_election.cc:304] T bfb140bb3a8b44bd8b4e1fcf0946162b P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 235ad6af04c34c88a5261daa3604fada, cf55383bef6649eb9e78cb2d2512dedc; no voters:
W20250109 08:58:26.897855 23344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.226.196:41829, user_credentials={real_user=slave}} blocked reactor thread for 70041us
I20250109 08:58:26.910310 24001 raft_consensus.cc:2798] T bfb140bb3a8b44bd8b4e1fcf0946162b P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:58:26.917804 24001 raft_consensus.cc:491] T bfb140bb3a8b44bd8b4e1fcf0946162b P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:58:26.919085 24001 raft_consensus.cc:3054] T bfb140bb3a8b44bd8b4e1fcf0946162b P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:26.912804 23378 raft_consensus.cc:3054] T 060daed1d3334c6188413dd381b82bf9 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:26.960088 23520 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ac4317290b24209a76d2e19e68ee45b" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01"
I20250109 08:58:26.961644 23203 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:34144 (request call id 70) took 1136 ms. Trace:
W20250109 08:58:26.967519 23418 leader_election.cc:343] T 1ac4317290b24209a76d2e19e68ee45b P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:26.968037 23418 leader_election.cc:304] T 1ac4317290b24209a76d2e19e68ee45b P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf55383bef6649eb9e78cb2d2512dedc; no voters: 05a74e1eb2474b3d8b3a2da1ef127e01, ed1efbce9ee3401e99a5260edeae4e7c
I20250109 08:58:26.975934 23450 raft_consensus.cc:2463] T 950e07e9cc794b8ab9ec5bd97d7e8e59 P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05a74e1eb2474b3d8b3a2da1ef127e01 in term 1.
I20250109 08:58:26.978793 23494 leader_election.cc:304] T 950e07e9cc794b8ab9ec5bd97d7e8e59 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01, cf55383bef6649eb9e78cb2d2512dedc; no voters:
W20250109 08:58:26.979820 23269 consensus_peers.cc:487] T 77597d68423749e7b87f9f0db7a1ff3b P 235ad6af04c34c88a5261daa3604fada -> Peer 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829): Couldn't send request to peer 05a74e1eb2474b3d8b3a2da1ef127e01. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20250109 08:58:26.983418 23346 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.226.195:35551, user_credentials={real_user=slave}} blocked reactor thread for 59282.8us
I20250109 08:58:26.995500 23945 raft_consensus.cc:2798] T 950e07e9cc794b8ab9ec5bd97d7e8e59 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:58:26.996071 23945 raft_consensus.cc:695] T 950e07e9cc794b8ab9ec5bd97d7e8e59 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 LEADER]: Becoming Leader. State: Replica: 05a74e1eb2474b3d8b3a2da1ef127e01, State: Running, Role: LEADER
I20250109 08:58:26.996809 23945 consensus_queue.cc:237] T 950e07e9cc794b8ab9ec5bd97d7e8e59 P 05a74e1eb2474b3d8b3a2da1ef127e01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:26.962064 23203 rpcz_store.cc:276] 0109 08:58:25.834193 (+ 0us) service_pool.cc:168] Inserting onto call queue
0109 08:58:25.835688 (+ 1495us) service_pool.cc:225] Handling call
0109 08:58:26.961553 (+1125865us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0109 08:58:26.171636 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0109 08:58:26.172189 (+ 553us) write_op.cc:432] Acquiring schema lock in shared mode
0109 08:58:26.172219 (+ 30us) write_op.cc:435] Acquired schema lock
0109 08:58:26.172236 (+ 17us) tablet.cc:659] Decoding operations
0109 08:58:26.172800 (+ 564us) write_op.cc:620] Acquiring the partition lock for write op
0109 08:58:26.172845 (+ 45us) write_op.cc:641] Partition lock acquired for write op
0109 08:58:26.172875 (+ 30us) tablet.cc:682] Acquiring locks for 3 operations
0109 08:58:26.173292 (+ 417us) tablet.cc:698] Row locks acquired
0109 08:58:26.173308 (+ 16us) write_op.cc:260] PREPARE: finished
0109 08:58:26.199670 (+ 26362us) write_op.cc:270] Start()
0109 08:58:26.219740 (+ 20070us) write_op.cc:276] Timestamp: P: 1736413106193530 usec, L: 0
0109 08:58:26.226816 (+ 7076us) op_driver.cc:348] REPLICATION: starting
0109 08:58:26.246887 (+ 20071us) log.cc:844] Serialized 1325 byte log entry
0109 08:58:26.338163 (+ 91276us) op_driver.cc:464] REPLICATION: finished
0109 08:58:26.340021 (+ 1858us) write_op.cc:301] APPLY: starting
0109 08:58:26.340080 (+ 59us) tablet.cc:1364] starting BulkCheckPresence
0109 08:58:26.340204 (+ 124us) tablet.cc:1367] finished BulkCheckPresence
0109 08:58:26.340309 (+ 105us) tablet.cc:1369] starting ApplyRowOperation cycle
0109 08:58:26.340712 (+ 403us) tablet.cc:1380] finished ApplyRowOperation cycle
0109 08:58:26.340864 (+ 152us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0109 08:58:26.340910 (+ 46us) write_op.cc:312] APPLY: finished
0109 08:58:26.341106 (+ 196us) log.cc:844] Serialized 34 byte log entry
0109 08:58:26.341247 (+ 141us) write_op.cc:489] Releasing partition, row and schema locks
0109 08:58:26.341322 (+ 75us) write_op.cc:454] Released schema lock
0109 08:58:26.341355 (+ 33us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":461705,"reported_tablets":6,"child_traces":[["op",{"apply.queue_time_us":1685,"apply.run_cpu_time_us":1558,"apply.run_wall_time_us":1566,"num_ops":3,"prepare.queue_time_us":108373,"prepare.run_cpu_time_us":2969,"prepare.run_wall_time_us":75506,"replication_time_us":109102,"thread_start_us":64747,"threads_started":1}]]}
I20250109 08:58:26.992291 23376 raft_consensus.cc:2463] T 1f9c6525267145d6bb73dde32ed9b7d4 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf55383bef6649eb9e78cb2d2512dedc in term 1.
I20250109 08:58:27.002280 23963 raft_consensus.cc:2743] T 1ac4317290b24209a76d2e19e68ee45b P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20250109 08:58:27.017746 23572 tablet_bootstrap.cc:492] T 94f2be307f704c75bceb213e52307cfb P cf55383bef6649eb9e78cb2d2512dedc: No bootstrap required, opened a new log
I20250109 08:58:27.021875 23300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "950e07e9cc794b8ab9ec5bd97d7e8e59" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada"
I20250109 08:58:27.024945 23375 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d3947300cfc4523acdab19fc992f26a" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" is_pre_election: true
I20250109 08:58:27.024783 24004 raft_consensus.cc:513] T 8d3947300cfc4523acdab19fc992f26a P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
W20250109 08:58:27.012061 23271 leader_election.cc:343] T 444507f7812441ceaeebc44bc8f06293 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:27.036267 23574 tablet_bootstrap.cc:654] T 1365130ef97c4581b623c22c39753831 P 235ad6af04c34c88a5261daa3604fada: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:27.040620 23527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "444507f7812441ceaeebc44bc8f06293" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01"
I20250109 08:58:27.074970 23527 raft_consensus.cc:3054] T 444507f7812441ceaeebc44bc8f06293 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:27.065994 23572 ts_tablet_manager.cc:1397] T 94f2be307f704c75bceb213e52307cfb P cf55383bef6649eb9e78cb2d2512dedc: Time spent bootstrapping tablet: real 0.432s user 0.024s sys 0.031s
I20250109 08:58:27.082785 23580 tablet_bootstrap.cc:492] T 444507f7812441ceaeebc44bc8f06293 P ed1efbce9ee3401e99a5260edeae4e7c: No bootstrap required, opened a new log
I20250109 08:58:27.083235 23580 ts_tablet_manager.cc:1397] T 444507f7812441ceaeebc44bc8f06293 P ed1efbce9ee3401e99a5260edeae4e7c: Time spent bootstrapping tablet: real 0.526s user 0.042s sys 0.034s
I20250109 08:58:27.074160 23300 raft_consensus.cc:3054] T 950e07e9cc794b8ab9ec5bd97d7e8e59 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:27.084757 23420 leader_election.cc:304] T 1f9c6525267145d6bb73dde32ed9b7d4 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf55383bef6649eb9e78cb2d2512dedc, ed1efbce9ee3401e99a5260edeae4e7c; no voters: 05a74e1eb2474b3d8b3a2da1ef127e01
I20250109 08:58:27.075181 23453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d3947300cfc4523acdab19fc992f26a" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc" is_pre_election: true
I20250109 08:58:27.098044 23521 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfb140bb3a8b44bd8b4e1fcf0946162b" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" is_pre_election: true
W20250109 08:58:27.083701 23271 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.226.194:41365, user_credentials={real_user=slave}} blocked reactor thread for 71681.6us
I20250109 08:58:27.090001 23963 raft_consensus.cc:2798] T 1f9c6525267145d6bb73dde32ed9b7d4 P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:58:27.104044 23569 tablet_bootstrap.cc:492] T 29828d9befb845d5bd49a80e018502e3 P 05a74e1eb2474b3d8b3a2da1ef127e01: Bootstrap starting.
I20250109 08:58:27.105647 23945 raft_consensus.cc:491] T 1f9c6525267145d6bb73dde32ed9b7d4 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:27.091212 23527 raft_consensus.cc:2463] T 444507f7812441ceaeebc44bc8f06293 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 235ad6af04c34c88a5261daa3604fada in term 1.
I20250109 08:58:27.108582 23269 leader_election.cc:304] T 444507f7812441ceaeebc44bc8f06293 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01, 235ad6af04c34c88a5261daa3604fada; no voters:
I20250109 08:58:27.073895 23873 leader_election.cc:290] T 444507f7812441ceaeebc44bc8f06293 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 election: Requested vote from peers 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829), ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365)
I20250109 08:58:27.111230 23586 raft_consensus.cc:2798] T 444507f7812441ceaeebc44bc8f06293 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:58:27.112620 23963 raft_consensus.cc:695] T 1f9c6525267145d6bb73dde32ed9b7d4 P cf55383bef6649eb9e78cb2d2512dedc [term 1 LEADER]: Becoming Leader. State: Replica: cf55383bef6649eb9e78cb2d2512dedc, State: Running, Role: LEADER
I20250109 08:58:27.106146 23945 raft_consensus.cc:513] T 1f9c6525267145d6bb73dde32ed9b7d4 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:27.114441 23572 raft_consensus.cc:357] T 94f2be307f704c75bceb213e52307cfb P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:27.115651 24004 leader_election.cc:290] T 8d3947300cfc4523acdab19fc992f26a P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 election: Requested vote from peers 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691), ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365)
I20250109 08:58:27.120499 23375 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "444507f7812441ceaeebc44bc8f06293" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c"
I20250109 08:58:27.101783 23993 consensus_queue.cc:1035] T 77597d68423749e7b87f9f0db7a1ff3b P 235ad6af04c34c88a5261daa3604fada [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.024s
I20250109 08:58:27.129931 23580 raft_consensus.cc:357] T 444507f7812441ceaeebc44bc8f06293 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:27.131220 23453 raft_consensus.cc:2388] T 8d3947300cfc4523acdab19fc992f26a P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 235ad6af04c34c88a5261daa3604fada in current term 1: Already voted for candidate cf55383bef6649eb9e78cb2d2512dedc in this term.
W20250109 08:58:27.101722 23271 leader_election.cc:343] T 8d3947300cfc4523acdab19fc992f26a P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:27.155038 23580 raft_consensus.cc:383] T 444507f7812441ceaeebc44bc8f06293 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:27.155417 23580 raft_consensus.cc:738] T 444507f7812441ceaeebc44bc8f06293 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed1efbce9ee3401e99a5260edeae4e7c, State: Initialized, Role: FOLLOWER
I20250109 08:58:27.156051 23580 consensus_queue.cc:260] T 444507f7812441ceaeebc44bc8f06293 P ed1efbce9ee3401e99a5260edeae4e7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:27.132184 23303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d3947300cfc4523acdab19fc992f26a" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada"
I20250109 08:58:27.108099 24001 raft_consensus.cc:513] T bfb140bb3a8b44bd8b4e1fcf0946162b P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:27.137800 23586 raft_consensus.cc:695] T 444507f7812441ceaeebc44bc8f06293 P 235ad6af04c34c88a5261daa3604fada [term 1 LEADER]: Becoming Leader. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Running, Role: LEADER
I20250109 08:58:27.157192 23303 raft_consensus.cc:3054] T 8d3947300cfc4523acdab19fc992f26a P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:27.157624 23586 consensus_queue.cc:237] T 444507f7812441ceaeebc44bc8f06293 P 235ad6af04c34c88a5261daa3604fada [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:27.158468 24001 leader_election.cc:290] T bfb140bb3a8b44bd8b4e1fcf0946162b P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 election: Requested vote from peers 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829), 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691)
W20250109 08:58:27.108295 23418 leader_election.cc:343] T bfb140bb3a8b44bd8b4e1fcf0946162b P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:27.141119 23572 raft_consensus.cc:383] T 94f2be307f704c75bceb213e52307cfb P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:27.160075 23299 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfb140bb3a8b44bd8b4e1fcf0946162b" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada"
I20250109 08:58:27.160476 23572 raft_consensus.cc:738] T 94f2be307f704c75bceb213e52307cfb P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf55383bef6649eb9e78cb2d2512dedc, State: Initialized, Role: FOLLOWER
W20250109 08:58:27.160717 23418 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.226.196:41829, user_credentials={real_user=slave}} blocked reactor thread for 52465.4us
I20250109 08:58:27.161152 23572 consensus_queue.cc:260] T 94f2be307f704c75bceb213e52307cfb P cf55383bef6649eb9e78cb2d2512dedc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:27.169494 23572 ts_tablet_manager.cc:1428] T 94f2be307f704c75bceb213e52307cfb P cf55383bef6649eb9e78cb2d2512dedc: Time spent starting tablet: real 0.093s user 0.005s sys 0.001s
W20250109 08:58:27.169996 23271 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.226.194:41365, user_credentials={real_user=slave}} blocked reactor thread for 68308.5us
I20250109 08:58:27.160751 23299 raft_consensus.cc:3054] T bfb140bb3a8b44bd8b4e1fcf0946162b P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:27.176050 23526 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfb140bb3a8b44bd8b4e1fcf0946162b" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01"
I20250109 08:58:27.179706 23767 consensus_queue.cc:1035] T 060daed1d3334c6188413dd381b82bf9 P 235ad6af04c34c88a5261daa3604fada [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:58:27.181099 23945 leader_election.cc:290] T 1f9c6525267145d6bb73dde32ed9b7d4 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365), cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551)
I20250109 08:58:27.182324 23379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d3947300cfc4523acdab19fc992f26a" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c"
I20250109 08:58:27.184410 23580 ts_tablet_manager.cc:1428] T 444507f7812441ceaeebc44bc8f06293 P ed1efbce9ee3401e99a5260edeae4e7c: Time spent starting tablet: real 0.101s user 0.006s sys 0.001s
I20250109 08:58:27.185318 23580 tablet_bootstrap.cc:492] T 8d3947300cfc4523acdab19fc992f26a P ed1efbce9ee3401e99a5260edeae4e7c: Bootstrap starting.
W20250109 08:58:27.187681 23420 leader_election.cc:343] T 8d3947300cfc4523acdab19fc992f26a P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365): Illegal state: must be running to vote when last-logged opid is not known
W20250109 08:58:27.192909 23418 leader_election.cc:343] T bfb140bb3a8b44bd8b4e1fcf0946162b P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:27.183574 23299 raft_consensus.cc:2463] T bfb140bb3a8b44bd8b4e1fcf0946162b P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf55383bef6649eb9e78cb2d2512dedc in term 1.
I20250109 08:58:27.232425 23420 leader_election.cc:304] T bfb140bb3a8b44bd8b4e1fcf0946162b P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 235ad6af04c34c88a5261daa3604fada, cf55383bef6649eb9e78cb2d2512dedc; no voters: 05a74e1eb2474b3d8b3a2da1ef127e01
I20250109 08:58:27.233461 24002 raft_consensus.cc:2798] T bfb140bb3a8b44bd8b4e1fcf0946162b P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:58:27.187076 23375 raft_consensus.cc:3054] T 444507f7812441ceaeebc44bc8f06293 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:27.233920 24002 raft_consensus.cc:695] T bfb140bb3a8b44bd8b4e1fcf0946162b P cf55383bef6649eb9e78cb2d2512dedc [term 1 LEADER]: Becoming Leader. State: Replica: cf55383bef6649eb9e78cb2d2512dedc, State: Running, Role: LEADER
W20250109 08:58:27.234156 23418 consensus_peers.cc:487] T a7d2ab61d238471c9f287ac1d5db4ed0 P cf55383bef6649eb9e78cb2d2512dedc -> Peer 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829): Couldn't send request to peer 05a74e1eb2474b3d8b3a2da1ef127e01. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20250109 08:58:27.236136 24002 consensus_queue.cc:237] T bfb140bb3a8b44bd8b4e1fcf0946162b P cf55383bef6649eb9e78cb2d2512dedc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
W20250109 08:58:27.242051 23251 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20250109 08:58:27.252045 23580 tablet_bootstrap.cc:654] T 8d3947300cfc4523acdab19fc992f26a P ed1efbce9ee3401e99a5260edeae4e7c: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:27.258208 23569 tablet_bootstrap.cc:654] T 29828d9befb845d5bd49a80e018502e3 P 05a74e1eb2474b3d8b3a2da1ef127e01: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:27.179718 23963 consensus_queue.cc:237] T 1f9c6525267145d6bb73dde32ed9b7d4 P cf55383bef6649eb9e78cb2d2512dedc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:27.254460 23300 raft_consensus.cc:2463] T 950e07e9cc794b8ab9ec5bd97d7e8e59 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05a74e1eb2474b3d8b3a2da1ef127e01 in term 1.
I20250109 08:58:27.269234 23448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9c6525267145d6bb73dde32ed9b7d4" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc" is_pre_election: true
I20250109 08:58:27.270649 23271 leader_election.cc:304] T 8d3947300cfc4523acdab19fc992f26a P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 235ad6af04c34c88a5261daa3604fada; no voters: cf55383bef6649eb9e78cb2d2512dedc, ed1efbce9ee3401e99a5260edeae4e7c
I20250109 08:58:27.256026 23303 raft_consensus.cc:2463] T 8d3947300cfc4523acdab19fc992f26a P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf55383bef6649eb9e78cb2d2512dedc in term 1.
I20250109 08:58:27.294180 23420 leader_election.cc:304] T 8d3947300cfc4523acdab19fc992f26a P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 235ad6af04c34c88a5261daa3604fada, cf55383bef6649eb9e78cb2d2512dedc; no voters: ed1efbce9ee3401e99a5260edeae4e7c
W20250109 08:58:27.295977 23494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.226.193:42691, user_credentials={real_user=slave}} blocked reactor thread for 106003us
I20250109 08:58:27.293675 23574 tablet_bootstrap.cc:492] T 1365130ef97c4581b623c22c39753831 P 235ad6af04c34c88a5261daa3604fada: No bootstrap required, opened a new log
I20250109 08:58:27.296861 23574 ts_tablet_manager.cc:1397] T 1365130ef97c4581b623c22c39753831 P 235ad6af04c34c88a5261daa3604fada: Time spent bootstrapping tablet: real 0.408s user 0.044s sys 0.036s
I20250109 08:58:27.292685 23993 raft_consensus.cc:2743] T 8d3947300cfc4523acdab19fc992f26a P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:58:27.298622 23378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9c6525267145d6bb73dde32ed9b7d4" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" is_pre_election: true
I20250109 08:58:27.305318 23378 raft_consensus.cc:2388] T 1f9c6525267145d6bb73dde32ed9b7d4 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 05a74e1eb2474b3d8b3a2da1ef127e01 in current term 1: Already voted for candidate cf55383bef6649eb9e78cb2d2512dedc in this term.
I20250109 08:58:27.300810 24001 raft_consensus.cc:2798] T 8d3947300cfc4523acdab19fc992f26a P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:58:27.293390 23375 raft_consensus.cc:2463] T 444507f7812441ceaeebc44bc8f06293 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 235ad6af04c34c88a5261daa3604fada in term 1.
I20250109 08:58:27.308326 24001 raft_consensus.cc:695] T 8d3947300cfc4523acdab19fc992f26a P cf55383bef6649eb9e78cb2d2512dedc [term 1 LEADER]: Becoming Leader. State: Replica: cf55383bef6649eb9e78cb2d2512dedc, State: Running, Role: LEADER
I20250109 08:58:27.309311 24001 consensus_queue.cc:237] T 8d3947300cfc4523acdab19fc992f26a P cf55383bef6649eb9e78cb2d2512dedc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:27.338238 23894 raft_consensus.cc:491] T 94f2be307f704c75bceb213e52307cfb P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:27.346325 23203 catalog_manager.cc:5526] T 950e07e9cc794b8ab9ec5bd97d7e8e59 P 05a74e1eb2474b3d8b3a2da1ef127e01 reported cstate change: term changed from 0 to 1, leader changed from <none> to 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196). New cstate: current_term: 1 leader_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:27.346375 23894 raft_consensus.cc:513] T 94f2be307f704c75bceb213e52307cfb P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:27.350397 23894 leader_election.cc:290] T 94f2be307f704c75bceb213e52307cfb P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829), 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691)
I20250109 08:58:27.353652 23521 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94f2be307f704c75bceb213e52307cfb" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" is_pre_election: true
W20250109 08:58:27.380336 23494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.226.193:42691, user_credentials={real_user=slave}} blocked reactor thread for 44302.7us
I20250109 08:58:27.384629 23574 raft_consensus.cc:357] T 1365130ef97c4581b623c22c39753831 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:27.385217 23494 leader_election.cc:304] T 1f9c6525267145d6bb73dde32ed9b7d4 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01; no voters: cf55383bef6649eb9e78cb2d2512dedc, ed1efbce9ee3401e99a5260edeae4e7c
I20250109 08:58:27.385488 23574 raft_consensus.cc:383] T 1365130ef97c4581b623c22c39753831 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:27.385865 23574 raft_consensus.cc:738] T 1365130ef97c4581b623c22c39753831 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Initialized, Role: FOLLOWER
I20250109 08:58:27.387120 23574 consensus_queue.cc:260] T 1365130ef97c4581b623c22c39753831 P 235ad6af04c34c88a5261daa3604fada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:27.379864 23303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94f2be307f704c75bceb213e52307cfb" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada" is_pre_election: true
I20250109 08:58:27.389429 23945 raft_consensus.cc:3054] T 1f9c6525267145d6bb73dde32ed9b7d4 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:27.398842 23574 ts_tablet_manager.cc:1428] T 1365130ef97c4581b623c22c39753831 P 235ad6af04c34c88a5261daa3604fada: Time spent starting tablet: real 0.101s user 0.005s sys 0.001s
I20250109 08:58:27.399603 23574 tablet_bootstrap.cc:492] T 29828d9befb845d5bd49a80e018502e3 P 235ad6af04c34c88a5261daa3604fada: Bootstrap starting.
I20250109 08:58:27.406695 23580 tablet_bootstrap.cc:492] T 8d3947300cfc4523acdab19fc992f26a P ed1efbce9ee3401e99a5260edeae4e7c: No bootstrap required, opened a new log
I20250109 08:58:27.407178 23580 ts_tablet_manager.cc:1397] T 8d3947300cfc4523acdab19fc992f26a P ed1efbce9ee3401e99a5260edeae4e7c: Time spent bootstrapping tablet: real 0.222s user 0.003s sys 0.011s
I20250109 08:58:27.414636 23569 tablet_bootstrap.cc:492] T 29828d9befb845d5bd49a80e018502e3 P 05a74e1eb2474b3d8b3a2da1ef127e01: No bootstrap required, opened a new log
I20250109 08:58:27.415158 23569 ts_tablet_manager.cc:1397] T 29828d9befb845d5bd49a80e018502e3 P 05a74e1eb2474b3d8b3a2da1ef127e01: Time spent bootstrapping tablet: real 0.311s user 0.019s sys 0.003s
I20250109 08:58:27.419469 23420 leader_election.cc:304] T 94f2be307f704c75bceb213e52307cfb P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf55383bef6649eb9e78cb2d2512dedc; no voters: 05a74e1eb2474b3d8b3a2da1ef127e01, 235ad6af04c34c88a5261daa3604fada
I20250109 08:58:27.423218 23569 raft_consensus.cc:357] T 29828d9befb845d5bd49a80e018502e3 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:27.429410 23569 raft_consensus.cc:383] T 29828d9befb845d5bd49a80e018502e3 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:27.430197 23569 raft_consensus.cc:738] T 29828d9befb845d5bd49a80e018502e3 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05a74e1eb2474b3d8b3a2da1ef127e01, State: Initialized, Role: FOLLOWER
I20250109 08:58:27.430895 23569 consensus_queue.cc:260] T 29828d9befb845d5bd49a80e018502e3 P 05a74e1eb2474b3d8b3a2da1ef127e01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:27.429533 24002 raft_consensus.cc:2743] T 94f2be307f704c75bceb213e52307cfb P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:58:27.432333 23454 raft_consensus.cc:3054] T 94f2be307f704c75bceb213e52307cfb P cf55383bef6649eb9e78cb2d2512dedc [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:27.433290 23569 ts_tablet_manager.cc:1428] T 29828d9befb845d5bd49a80e018502e3 P 05a74e1eb2474b3d8b3a2da1ef127e01: Time spent starting tablet: real 0.018s user 0.006s sys 0.001s
I20250109 08:58:27.434195 23569 tablet_bootstrap.cc:492] T a7d2ab61d238471c9f287ac1d5db4ed0 P 05a74e1eb2474b3d8b3a2da1ef127e01: Bootstrap starting.
I20250109 08:58:27.428675 23574 tablet_bootstrap.cc:654] T 29828d9befb845d5bd49a80e018502e3 P 235ad6af04c34c88a5261daa3604fada: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:27.437018 23580 raft_consensus.cc:357] T 8d3947300cfc4523acdab19fc992f26a P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:27.437775 23580 raft_consensus.cc:383] T 8d3947300cfc4523acdab19fc992f26a P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:27.438031 23580 raft_consensus.cc:738] T 8d3947300cfc4523acdab19fc992f26a P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed1efbce9ee3401e99a5260edeae4e7c, State: Initialized, Role: FOLLOWER
I20250109 08:58:27.438640 23580 consensus_queue.cc:260] T 8d3947300cfc4523acdab19fc992f26a P ed1efbce9ee3401e99a5260edeae4e7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:27.442167 23580 ts_tablet_manager.cc:1428] T 8d3947300cfc4523acdab19fc992f26a P ed1efbce9ee3401e99a5260edeae4e7c: Time spent starting tablet: real 0.035s user 0.004s sys 0.002s
I20250109 08:58:27.443051 23580 tablet_bootstrap.cc:492] T ee703ea575234c659da04f21ef3cdd70 P ed1efbce9ee3401e99a5260edeae4e7c: Bootstrap starting.
I20250109 08:58:27.444206 23965 consensus_queue.cc:1035] T 950e07e9cc794b8ab9ec5bd97d7e8e59 P 05a74e1eb2474b3d8b3a2da1ef127e01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:58:27.451786 23945 raft_consensus.cc:2743] T 1f9c6525267145d6bb73dde32ed9b7d4 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:58:27.453246 23580 tablet_bootstrap.cc:654] T ee703ea575234c659da04f21ef3cdd70 P ed1efbce9ee3401e99a5260edeae4e7c: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:27.476810 23569 tablet_bootstrap.cc:654] T a7d2ab61d238471c9f287ac1d5db4ed0 P 05a74e1eb2474b3d8b3a2da1ef127e01: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:27.480314 23574 tablet_bootstrap.cc:492] T 29828d9befb845d5bd49a80e018502e3 P 235ad6af04c34c88a5261daa3604fada: No bootstrap required, opened a new log
I20250109 08:58:27.480831 23574 ts_tablet_manager.cc:1397] T 29828d9befb845d5bd49a80e018502e3 P 235ad6af04c34c88a5261daa3604fada: Time spent bootstrapping tablet: real 0.081s user 0.015s sys 0.008s
I20250109 08:58:27.481732 23580 tablet_bootstrap.cc:492] T ee703ea575234c659da04f21ef3cdd70 P ed1efbce9ee3401e99a5260edeae4e7c: No bootstrap required, opened a new log
I20250109 08:58:27.482198 23580 ts_tablet_manager.cc:1397] T ee703ea575234c659da04f21ef3cdd70 P ed1efbce9ee3401e99a5260edeae4e7c: Time spent bootstrapping tablet: real 0.039s user 0.007s sys 0.009s
I20250109 08:58:27.483050 23574 raft_consensus.cc:357] T 29828d9befb845d5bd49a80e018502e3 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:27.483749 23574 raft_consensus.cc:383] T 29828d9befb845d5bd49a80e018502e3 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:27.483950 23574 raft_consensus.cc:738] T 29828d9befb845d5bd49a80e018502e3 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Initialized, Role: FOLLOWER
I20250109 08:58:27.484483 23574 consensus_queue.cc:260] T 29828d9befb845d5bd49a80e018502e3 P 235ad6af04c34c88a5261daa3604fada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:27.484669 23580 raft_consensus.cc:357] T ee703ea575234c659da04f21ef3cdd70 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:27.485397 23580 raft_consensus.cc:383] T ee703ea575234c659da04f21ef3cdd70 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:27.485668 23580 raft_consensus.cc:738] T ee703ea575234c659da04f21ef3cdd70 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed1efbce9ee3401e99a5260edeae4e7c, State: Initialized, Role: FOLLOWER
I20250109 08:58:27.486129 23574 ts_tablet_manager.cc:1428] T 29828d9befb845d5bd49a80e018502e3 P 235ad6af04c34c88a5261daa3604fada: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20250109 08:58:27.486325 23580 consensus_queue.cc:260] T ee703ea575234c659da04f21ef3cdd70 P ed1efbce9ee3401e99a5260edeae4e7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:27.493510 23569 tablet_bootstrap.cc:492] T a7d2ab61d238471c9f287ac1d5db4ed0 P 05a74e1eb2474b3d8b3a2da1ef127e01: No bootstrap required, opened a new log
I20250109 08:58:27.493177 23943 consensus_queue.cc:1035] T 950e07e9cc794b8ab9ec5bd97d7e8e59 P 05a74e1eb2474b3d8b3a2da1ef127e01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:58:27.497164 23569 ts_tablet_manager.cc:1397] T a7d2ab61d238471c9f287ac1d5db4ed0 P 05a74e1eb2474b3d8b3a2da1ef127e01: Time spent bootstrapping tablet: real 0.063s user 0.006s sys 0.006s
I20250109 08:58:27.500684 23580 ts_tablet_manager.cc:1428] T ee703ea575234c659da04f21ef3cdd70 P ed1efbce9ee3401e99a5260edeae4e7c: Time spent starting tablet: real 0.018s user 0.007s sys 0.000s
I20250109 08:58:27.499842 23569 raft_consensus.cc:357] T a7d2ab61d238471c9f287ac1d5db4ed0 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:27.504005 23569 raft_consensus.cc:383] T a7d2ab61d238471c9f287ac1d5db4ed0 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:27.504283 23569 raft_consensus.cc:738] T a7d2ab61d238471c9f287ac1d5db4ed0 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05a74e1eb2474b3d8b3a2da1ef127e01, State: Initialized, Role: FOLLOWER
I20250109 08:58:27.504997 23569 consensus_queue.cc:260] T a7d2ab61d238471c9f287ac1d5db4ed0 P 05a74e1eb2474b3d8b3a2da1ef127e01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:27.507311 23200 catalog_manager.cc:5526] T 77597d68423749e7b87f9f0db7a1ff3b P 235ad6af04c34c88a5261daa3604fada reported cstate change: term changed from 0 to 1, leader changed from <none> to 235ad6af04c34c88a5261daa3604fada (127.19.226.193). New cstate: current_term: 1 leader_uuid: "235ad6af04c34c88a5261daa3604fada" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } health_report { overall_health: HEALTHY } } }
I20250109 08:58:27.508410 23569 ts_tablet_manager.cc:1428] T a7d2ab61d238471c9f287ac1d5db4ed0 P 05a74e1eb2474b3d8b3a2da1ef127e01: Time spent starting tablet: real 0.011s user 0.007s sys 0.000s
I20250109 08:58:27.509323 23569 tablet_bootstrap.cc:492] T ee703ea575234c659da04f21ef3cdd70 P 05a74e1eb2474b3d8b3a2da1ef127e01: Bootstrap starting.
I20250109 08:58:27.516299 23569 tablet_bootstrap.cc:654] T ee703ea575234c659da04f21ef3cdd70 P 05a74e1eb2474b3d8b3a2da1ef127e01: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:27.558609 23202 catalog_manager.cc:5526] T 8d3947300cfc4523acdab19fc992f26a P cf55383bef6649eb9e78cb2d2512dedc reported cstate change: term changed from 0 to 1, leader changed from <none> to cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195). New cstate: current_term: 1 leader_uuid: "cf55383bef6649eb9e78cb2d2512dedc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } health_report { overall_health: HEALTHY } } }
I20250109 08:58:27.566329 23202 catalog_manager.cc:5526] T 1f9c6525267145d6bb73dde32ed9b7d4 P cf55383bef6649eb9e78cb2d2512dedc reported cstate change: term changed from 0 to 1, leader changed from <none> to cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195). New cstate: current_term: 1 leader_uuid: "cf55383bef6649eb9e78cb2d2512dedc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:27.587980 23611 raft_consensus.cc:491] T ee703ea575234c659da04f21ef3cdd70 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:27.588555 23611 raft_consensus.cc:513] T ee703ea575234c659da04f21ef3cdd70 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:27.590268 23611 leader_election.cc:290] T ee703ea575234c659da04f21ef3cdd70 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829), 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691)
I20250109 08:58:27.606189 23302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee703ea575234c659da04f21ef3cdd70" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada" is_pre_election: true
I20250109 08:58:27.606963 23302 raft_consensus.cc:2463] T ee703ea575234c659da04f21ef3cdd70 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed1efbce9ee3401e99a5260edeae4e7c in term 0.
I20250109 08:58:27.607950 23346 leader_election.cc:304] T ee703ea575234c659da04f21ef3cdd70 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 235ad6af04c34c88a5261daa3604fada, ed1efbce9ee3401e99a5260edeae4e7c; no voters:
I20250109 08:58:27.609231 24024 raft_consensus.cc:2798] T ee703ea575234c659da04f21ef3cdd70 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:58:27.609624 24024 raft_consensus.cc:491] T ee703ea575234c659da04f21ef3cdd70 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:58:27.609923 24024 raft_consensus.cc:3054] T ee703ea575234c659da04f21ef3cdd70 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:27.601348 23873 consensus_queue.cc:1035] T 444507f7812441ceaeebc44bc8f06293 P 235ad6af04c34c88a5261daa3604fada [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.015s
I20250109 08:58:27.616631 24024 raft_consensus.cc:513] T ee703ea575234c659da04f21ef3cdd70 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:27.625777 24024 leader_election.cc:290] T ee703ea575234c659da04f21ef3cdd70 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 election: Requested vote from peers 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829), 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691)
I20250109 08:58:27.625766 23304 raft_consensus.cc:3054] T 1365130ef97c4581b623c22c39753831 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:27.633399 23521 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee703ea575234c659da04f21ef3cdd70" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" is_pre_election: true
I20250109 08:58:27.638818 23943 raft_consensus.cc:491] T 29828d9befb845d5bd49a80e018502e3 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:27.641117 23943 raft_consensus.cc:513] T 29828d9befb845d5bd49a80e018502e3 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
W20250109 08:58:27.644865 23344 leader_election.cc:343] T ee703ea575234c659da04f21ef3cdd70 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:27.649097 23569 tablet_bootstrap.cc:492] T ee703ea575234c659da04f21ef3cdd70 P 05a74e1eb2474b3d8b3a2da1ef127e01: No bootstrap required, opened a new log
I20250109 08:58:27.649643 23569 ts_tablet_manager.cc:1397] T ee703ea575234c659da04f21ef3cdd70 P 05a74e1eb2474b3d8b3a2da1ef127e01: Time spent bootstrapping tablet: real 0.140s user 0.056s sys 0.041s
I20250109 08:58:27.643051 23302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee703ea575234c659da04f21ef3cdd70" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada"
I20250109 08:58:27.650774 23302 raft_consensus.cc:3054] T ee703ea575234c659da04f21ef3cdd70 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:27.650170 23296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29828d9befb845d5bd49a80e018502e3" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada" is_pre_election: true
I20250109 08:58:27.652128 23296 raft_consensus.cc:2463] T 29828d9befb845d5bd49a80e018502e3 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05a74e1eb2474b3d8b3a2da1ef127e01 in term 0.
I20250109 08:58:27.653337 23494 leader_election.cc:304] T 29828d9befb845d5bd49a80e018502e3 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01, 235ad6af04c34c88a5261daa3604fada; no voters:
I20250109 08:58:27.653903 23569 raft_consensus.cc:357] T ee703ea575234c659da04f21ef3cdd70 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:27.654618 23569 raft_consensus.cc:383] T ee703ea575234c659da04f21ef3cdd70 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:27.654876 23569 raft_consensus.cc:738] T ee703ea575234c659da04f21ef3cdd70 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05a74e1eb2474b3d8b3a2da1ef127e01, State: Initialized, Role: FOLLOWER
I20250109 08:58:27.648936 23943 leader_election.cc:290] T 29828d9befb845d5bd49a80e018502e3 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691), ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365)
I20250109 08:58:27.655474 23569 consensus_queue.cc:260] T ee703ea575234c659da04f21ef3cdd70 P 05a74e1eb2474b3d8b3a2da1ef127e01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:27.656164 24031 raft_consensus.cc:2798] T 29828d9befb845d5bd49a80e018502e3 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:58:27.657982 24031 raft_consensus.cc:491] T 29828d9befb845d5bd49a80e018502e3 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:58:27.658286 24031 raft_consensus.cc:3054] T 29828d9befb845d5bd49a80e018502e3 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:27.660487 23376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29828d9befb845d5bd49a80e018502e3" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" is_pre_election: true
I20250109 08:58:27.661505 23569 ts_tablet_manager.cc:1428] T ee703ea575234c659da04f21ef3cdd70 P 05a74e1eb2474b3d8b3a2da1ef127e01: Time spent starting tablet: real 0.012s user 0.004s sys 0.001s
I20250109 08:58:27.660838 23201 catalog_manager.cc:5526] T 444507f7812441ceaeebc44bc8f06293 P 235ad6af04c34c88a5261daa3604fada reported cstate change: term changed from 0 to 1, leader changed from <none> to 235ad6af04c34c88a5261daa3604fada (127.19.226.193). New cstate: current_term: 1 leader_uuid: "235ad6af04c34c88a5261daa3604fada" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:27.662426 23569 tablet_bootstrap.cc:492] T 77597d68423749e7b87f9f0db7a1ff3b P 05a74e1eb2474b3d8b3a2da1ef127e01: Bootstrap starting.
I20250109 08:58:27.664775 23527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee703ea575234c659da04f21ef3cdd70" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01"
I20250109 08:58:27.665323 23527 raft_consensus.cc:3054] T ee703ea575234c659da04f21ef3cdd70 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:27.669915 23569 tablet_bootstrap.cc:654] T 77597d68423749e7b87f9f0db7a1ff3b P 05a74e1eb2474b3d8b3a2da1ef127e01: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:27.674201 23527 raft_consensus.cc:2463] T ee703ea575234c659da04f21ef3cdd70 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed1efbce9ee3401e99a5260edeae4e7c in term 1.
I20250109 08:58:27.676136 23344 leader_election.cc:304] T ee703ea575234c659da04f21ef3cdd70 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01, ed1efbce9ee3401e99a5260edeae4e7c; no voters:
I20250109 08:58:27.680059 24000 raft_consensus.cc:2798] T ee703ea575234c659da04f21ef3cdd70 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:58:27.679687 23376 raft_consensus.cc:2463] T 29828d9befb845d5bd49a80e018502e3 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05a74e1eb2474b3d8b3a2da1ef127e01 in term 0.
I20250109 08:58:27.689267 24000 raft_consensus.cc:695] T ee703ea575234c659da04f21ef3cdd70 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 LEADER]: Becoming Leader. State: Replica: ed1efbce9ee3401e99a5260edeae4e7c, State: Running, Role: LEADER
I20250109 08:58:27.703488 24000 consensus_queue.cc:237] T ee703ea575234c659da04f21ef3cdd70 P ed1efbce9ee3401e99a5260edeae4e7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
W20250109 08:58:27.709431 23418 consensus_peers.cc:487] T bfb140bb3a8b44bd8b4e1fcf0946162b P cf55383bef6649eb9e78cb2d2512dedc -> Peer 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829): Couldn't send request to peer 05a74e1eb2474b3d8b3a2da1ef127e01. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:27.713253 23873 consensus_queue.cc:1035] T 444507f7812441ceaeebc44bc8f06293 P 235ad6af04c34c88a5261daa3604fada [LEADER]: Connected to new peer: Peer: permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:58:27.697306 24004 consensus_queue.cc:1035] T 1f9c6525267145d6bb73dde32ed9b7d4 P cf55383bef6649eb9e78cb2d2512dedc [LEADER]: Connected to new peer: Peer: permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.018s
I20250109 08:58:27.724987 23378 raft_consensus.cc:3054] T 8d3947300cfc4523acdab19fc992f26a P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:27.732729 23202 catalog_manager.cc:5526] T bfb140bb3a8b44bd8b4e1fcf0946162b P cf55383bef6649eb9e78cb2d2512dedc reported cstate change: term changed from 0 to 1, leader changed from <none> to cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195). New cstate: current_term: 1 leader_uuid: "cf55383bef6649eb9e78cb2d2512dedc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:27.762712 23894 consensus_queue.cc:1035] T bfb140bb3a8b44bd8b4e1fcf0946162b P cf55383bef6649eb9e78cb2d2512dedc [LEADER]: Connected to new peer: Peer: permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:58:27.767351 23894 consensus_queue.cc:1035] T 8d3947300cfc4523acdab19fc992f26a P cf55383bef6649eb9e78cb2d2512dedc [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:58:27.769387 23201 catalog_manager.cc:5526] T ee703ea575234c659da04f21ef3cdd70 P ed1efbce9ee3401e99a5260edeae4e7c reported cstate change: term changed from 0 to 1, leader changed from <none> to ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194). New cstate: current_term: 1 leader_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } health_report { overall_health: HEALTHY } } }
I20250109 08:58:27.780680 24031 raft_consensus.cc:513] T 29828d9befb845d5bd49a80e018502e3 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:27.782684 24031 leader_election.cc:290] T 29828d9befb845d5bd49a80e018502e3 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 election: Requested vote from peers 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691), ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365)
I20250109 08:58:27.784924 23300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29828d9befb845d5bd49a80e018502e3" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada"
I20250109 08:58:27.785473 23375 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29828d9befb845d5bd49a80e018502e3" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c"
I20250109 08:58:27.786206 23375 raft_consensus.cc:3054] T 29828d9befb845d5bd49a80e018502e3 P ed1efbce9ee3401e99a5260edeae4e7c [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:27.788012 24002 consensus_queue.cc:1035] T 1f9c6525267145d6bb73dde32ed9b7d4 P cf55383bef6649eb9e78cb2d2512dedc [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:58:27.785578 23300 raft_consensus.cc:3054] T 29828d9befb845d5bd49a80e018502e3 P 235ad6af04c34c88a5261daa3604fada [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:27.794375 23302 raft_consensus.cc:2463] T ee703ea575234c659da04f21ef3cdd70 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed1efbce9ee3401e99a5260edeae4e7c in term 1.
W20250109 08:58:27.795687 24053 log.cc:926] Time spent T 1365130ef97c4581b623c22c39753831 P 235ad6af04c34c88a5261daa3604fada: Append to log took a long time: real 0.065s user 0.001s sys 0.000s
I20250109 08:58:27.796128 23375 raft_consensus.cc:2463] T 29828d9befb845d5bd49a80e018502e3 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05a74e1eb2474b3d8b3a2da1ef127e01 in term 1.
I20250109 08:58:27.797861 23521 raft_consensus.cc:3054] T a7d2ab61d238471c9f287ac1d5db4ed0 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:27.806617 23300 raft_consensus.cc:2463] T 29828d9befb845d5bd49a80e018502e3 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05a74e1eb2474b3d8b3a2da1ef127e01 in term 1.
I20250109 08:58:27.807812 23494 leader_election.cc:304] T 29828d9befb845d5bd49a80e018502e3 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01, 235ad6af04c34c88a5261daa3604fada; no voters:
I20250109 08:58:27.808597 23943 raft_consensus.cc:2798] T 29828d9befb845d5bd49a80e018502e3 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:58:27.809185 23943 raft_consensus.cc:695] T 29828d9befb845d5bd49a80e018502e3 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 LEADER]: Becoming Leader. State: Replica: 05a74e1eb2474b3d8b3a2da1ef127e01, State: Running, Role: LEADER
I20250109 08:58:27.809988 23943 consensus_queue.cc:237] T 29828d9befb845d5bd49a80e018502e3 P 05a74e1eb2474b3d8b3a2da1ef127e01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:27.825134 24001 consensus_queue.cc:1035] T 8d3947300cfc4523acdab19fc992f26a P cf55383bef6649eb9e78cb2d2512dedc [LEADER]: Connected to new peer: Peer: permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:58:27.827258 23569 tablet_bootstrap.cc:492] T 77597d68423749e7b87f9f0db7a1ff3b P 05a74e1eb2474b3d8b3a2da1ef127e01: No bootstrap required, opened a new log
I20250109 08:58:27.827781 23569 ts_tablet_manager.cc:1397] T 77597d68423749e7b87f9f0db7a1ff3b P 05a74e1eb2474b3d8b3a2da1ef127e01: Time spent bootstrapping tablet: real 0.166s user 0.009s sys 0.003s
I20250109 08:58:27.830158 23569 raft_consensus.cc:357] T 77597d68423749e7b87f9f0db7a1ff3b P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:27.830828 23569 raft_consensus.cc:383] T 77597d68423749e7b87f9f0db7a1ff3b P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:27.831068 23569 raft_consensus.cc:738] T 77597d68423749e7b87f9f0db7a1ff3b P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05a74e1eb2474b3d8b3a2da1ef127e01, State: Initialized, Role: FOLLOWER
I20250109 08:58:27.831615 23569 consensus_queue.cc:260] T 77597d68423749e7b87f9f0db7a1ff3b P 05a74e1eb2474b3d8b3a2da1ef127e01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:27.846392 23569 ts_tablet_manager.cc:1428] T 77597d68423749e7b87f9f0db7a1ff3b P 05a74e1eb2474b3d8b3a2da1ef127e01: Time spent starting tablet: real 0.018s user 0.004s sys 0.000s
I20250109 08:58:27.847303 23569 tablet_bootstrap.cc:492] T 1ac4317290b24209a76d2e19e68ee45b P 05a74e1eb2474b3d8b3a2da1ef127e01: Bootstrap starting.
I20250109 08:58:27.847389 23526 raft_consensus.cc:3054] T 77597d68423749e7b87f9f0db7a1ff3b P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:27.871045 23569 tablet_bootstrap.cc:654] T 1ac4317290b24209a76d2e19e68ee45b P 05a74e1eb2474b3d8b3a2da1ef127e01: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:27.915153 23569 tablet_bootstrap.cc:492] T 1ac4317290b24209a76d2e19e68ee45b P 05a74e1eb2474b3d8b3a2da1ef127e01: No bootstrap required, opened a new log
I20250109 08:58:27.915602 23569 ts_tablet_manager.cc:1397] T 1ac4317290b24209a76d2e19e68ee45b P 05a74e1eb2474b3d8b3a2da1ef127e01: Time spent bootstrapping tablet: real 0.068s user 0.008s sys 0.004s
I20250109 08:58:27.917959 23569 raft_consensus.cc:357] T 1ac4317290b24209a76d2e19e68ee45b P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:27.918639 23569 raft_consensus.cc:383] T 1ac4317290b24209a76d2e19e68ee45b P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:27.918910 23569 raft_consensus.cc:738] T 1ac4317290b24209a76d2e19e68ee45b P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05a74e1eb2474b3d8b3a2da1ef127e01, State: Initialized, Role: FOLLOWER
I20250109 08:58:27.919493 23569 consensus_queue.cc:260] T 1ac4317290b24209a76d2e19e68ee45b P 05a74e1eb2474b3d8b3a2da1ef127e01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:27.921847 23569 ts_tablet_manager.cc:1428] T 1ac4317290b24209a76d2e19e68ee45b P 05a74e1eb2474b3d8b3a2da1ef127e01: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
I20250109 08:58:27.922999 23569 tablet_bootstrap.cc:492] T bfb140bb3a8b44bd8b4e1fcf0946162b P 05a74e1eb2474b3d8b3a2da1ef127e01: Bootstrap starting.
I20250109 08:58:27.929584 23569 tablet_bootstrap.cc:654] T bfb140bb3a8b44bd8b4e1fcf0946162b P 05a74e1eb2474b3d8b3a2da1ef127e01: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:27.932132 23200 catalog_manager.cc:5526] T 29828d9befb845d5bd49a80e018502e3 P 05a74e1eb2474b3d8b3a2da1ef127e01 reported cstate change: term changed from 0 to 1, leader changed from <none> to 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196). New cstate: current_term: 1 leader_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:27.938163 23569 tablet_bootstrap.cc:492] T bfb140bb3a8b44bd8b4e1fcf0946162b P 05a74e1eb2474b3d8b3a2da1ef127e01: No bootstrap required, opened a new log
I20250109 08:58:27.938684 23569 ts_tablet_manager.cc:1397] T bfb140bb3a8b44bd8b4e1fcf0946162b P 05a74e1eb2474b3d8b3a2da1ef127e01: Time spent bootstrapping tablet: real 0.016s user 0.008s sys 0.000s
I20250109 08:58:27.941320 23569 raft_consensus.cc:357] T bfb140bb3a8b44bd8b4e1fcf0946162b P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:27.942274 23569 raft_consensus.cc:383] T bfb140bb3a8b44bd8b4e1fcf0946162b P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:27.942557 23569 raft_consensus.cc:738] T bfb140bb3a8b44bd8b4e1fcf0946162b P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05a74e1eb2474b3d8b3a2da1ef127e01, State: Initialized, Role: FOLLOWER
I20250109 08:58:27.943065 23569 consensus_queue.cc:260] T bfb140bb3a8b44bd8b4e1fcf0946162b P 05a74e1eb2474b3d8b3a2da1ef127e01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:27.944864 23569 ts_tablet_manager.cc:1428] T bfb140bb3a8b44bd8b4e1fcf0946162b P 05a74e1eb2474b3d8b3a2da1ef127e01: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20250109 08:58:28.000486 23943 raft_consensus.cc:491] T bfb140bb3a8b44bd8b4e1fcf0946162b P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:28.000973 23943 raft_consensus.cc:513] T bfb140bb3a8b44bd8b4e1fcf0946162b P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:28.002805 23943 leader_election.cc:290] T bfb140bb3a8b44bd8b4e1fcf0946162b P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551), 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691)
I20250109 08:58:28.003484 23445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfb140bb3a8b44bd8b4e1fcf0946162b" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc" is_pre_election: true
I20250109 08:58:28.004019 23302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfb140bb3a8b44bd8b4e1fcf0946162b" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada" is_pre_election: true
I20250109 08:58:28.005339 23494 leader_election.cc:304] T bfb140bb3a8b44bd8b4e1fcf0946162b P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01; no voters: 235ad6af04c34c88a5261daa3604fada, cf55383bef6649eb9e78cb2d2512dedc
I20250109 08:58:28.005930 23943 raft_consensus.cc:2743] T bfb140bb3a8b44bd8b4e1fcf0946162b P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:58:28.189155 24024 consensus_queue.cc:1035] T ee703ea575234c659da04f21ef3cdd70 P ed1efbce9ee3401e99a5260edeae4e7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:58:28.198711 24000 consensus_queue.cc:1035] T ee703ea575234c659da04f21ef3cdd70 P ed1efbce9ee3401e99a5260edeae4e7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:58:28.230325 24024 raft_consensus.cc:491] T 1ac4317290b24209a76d2e19e68ee45b P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:28.230790 24024 raft_consensus.cc:513] T 1ac4317290b24209a76d2e19e68ee45b P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:28.232149 23521 raft_consensus.cc:3054] T bfb140bb3a8b44bd8b4e1fcf0946162b P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:28.232636 24024 leader_election.cc:290] T 1ac4317290b24209a76d2e19e68ee45b P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551), 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829)
I20250109 08:58:28.233218 23445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ac4317290b24209a76d2e19e68ee45b" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc" is_pre_election: true
I20250109 08:58:28.233842 23445 raft_consensus.cc:2463] T 1ac4317290b24209a76d2e19e68ee45b P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed1efbce9ee3401e99a5260edeae4e7c in term 1.
I20250109 08:58:28.233836 23526 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ac4317290b24209a76d2e19e68ee45b" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" is_pre_election: true
I20250109 08:58:28.234635 23526 raft_consensus.cc:2463] T 1ac4317290b24209a76d2e19e68ee45b P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed1efbce9ee3401e99a5260edeae4e7c in term 0.
I20250109 08:58:28.234953 23346 leader_election.cc:304] T 1ac4317290b24209a76d2e19e68ee45b P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf55383bef6649eb9e78cb2d2512dedc, ed1efbce9ee3401e99a5260edeae4e7c; no voters:
I20250109 08:58:28.235644 24024 raft_consensus.cc:2798] T 1ac4317290b24209a76d2e19e68ee45b P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20250109 08:58:28.235985 24024 raft_consensus.cc:491] T 1ac4317290b24209a76d2e19e68ee45b P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:58:28.236308 24024 raft_consensus.cc:3054] T 1ac4317290b24209a76d2e19e68ee45b P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:28.242355 24024 raft_consensus.cc:513] T 1ac4317290b24209a76d2e19e68ee45b P ed1efbce9ee3401e99a5260edeae4e7c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:28.244575 24024 leader_election.cc:290] T 1ac4317290b24209a76d2e19e68ee45b P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 2 election: Requested vote from peers cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551), 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829)
I20250109 08:58:28.245730 23521 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ac4317290b24209a76d2e19e68ee45b" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01"
I20250109 08:58:28.246282 23521 raft_consensus.cc:3054] T 1ac4317290b24209a76d2e19e68ee45b P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 0 FOLLOWER]: Advancing to term 2
I20250109 08:58:28.246301 23445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ac4317290b24209a76d2e19e68ee45b" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc"
I20250109 08:58:28.247253 23445 raft_consensus.cc:3054] T 1ac4317290b24209a76d2e19e68ee45b P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:28.252277 23521 raft_consensus.cc:2463] T 1ac4317290b24209a76d2e19e68ee45b P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed1efbce9ee3401e99a5260edeae4e7c in term 2.
I20250109 08:58:28.253281 23344 leader_election.cc:304] T 1ac4317290b24209a76d2e19e68ee45b P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01, ed1efbce9ee3401e99a5260edeae4e7c; no voters:
I20250109 08:58:28.253647 23445 raft_consensus.cc:2463] T 1ac4317290b24209a76d2e19e68ee45b P cf55383bef6649eb9e78cb2d2512dedc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed1efbce9ee3401e99a5260edeae4e7c in term 2.
I20250109 08:58:28.253850 24024 raft_consensus.cc:2798] T 1ac4317290b24209a76d2e19e68ee45b P ed1efbce9ee3401e99a5260edeae4e7c [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:58:28.254372 24024 raft_consensus.cc:695] T 1ac4317290b24209a76d2e19e68ee45b P ed1efbce9ee3401e99a5260edeae4e7c [term 2 LEADER]: Becoming Leader. State: Replica: ed1efbce9ee3401e99a5260edeae4e7c, State: Running, Role: LEADER
I20250109 08:58:28.255120 24024 consensus_queue.cc:237] T 1ac4317290b24209a76d2e19e68ee45b P ed1efbce9ee3401e99a5260edeae4e7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:28.266573 23200 catalog_manager.cc:5526] T 1ac4317290b24209a76d2e19e68ee45b P ed1efbce9ee3401e99a5260edeae4e7c reported cstate change: term changed from 0 to 2, leader changed from <none> to ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194). New cstate: current_term: 2 leader_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:28.325995 24031 consensus_queue.cc:1035] T 29828d9befb845d5bd49a80e018502e3 P 05a74e1eb2474b3d8b3a2da1ef127e01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:58:28.348403 23943 consensus_queue.cc:1035] T 29828d9befb845d5bd49a80e018502e3 P 05a74e1eb2474b3d8b3a2da1ef127e01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:58:28.684864 23526 raft_consensus.cc:1270] T 1ac4317290b24209a76d2e19e68ee45b P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 FOLLOWER]: Refusing update from remote peer ed1efbce9ee3401e99a5260edeae4e7c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20250109 08:58:28.686553 24024 consensus_queue.cc:1035] T 1ac4317290b24209a76d2e19e68ee45b P ed1efbce9ee3401e99a5260edeae4e7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:58:28.704411 23445 raft_consensus.cc:1270] T 1ac4317290b24209a76d2e19e68ee45b P cf55383bef6649eb9e78cb2d2512dedc [term 2 FOLLOWER]: Refusing update from remote peer ed1efbce9ee3401e99a5260edeae4e7c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20250109 08:58:28.705785 24024 consensus_queue.cc:1035] T 1ac4317290b24209a76d2e19e68ee45b P ed1efbce9ee3401e99a5260edeae4e7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:58:29.386126 20363 tablet_server.cc:178] TabletServer@127.19.226.193:0 shutting down...
W20250109 08:58:29.395856 23346 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 506 similar messages]
W20250109 08:58:29.525009 23346 consensus_peers.cc:487] T d714271c7ea64c0b8b13fcf63591ec07 P ed1efbce9ee3401e99a5260edeae4e7c -> Peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Couldn't send request to peer 235ad6af04c34c88a5261daa3604fada. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20250109 08:58:29.541108 23346 consensus_peers.cc:487] T 6eef9bb3c645428d9f1cb843bae9a264 P ed1efbce9ee3401e99a5260edeae4e7c -> Peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Couldn't send request to peer 235ad6af04c34c88a5261daa3604fada. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20250109 08:58:29.558948 23420 consensus_peers.cc:487] T 2e3d34c0d09843d2be54e47f3ccc16d8 P cf55383bef6649eb9e78cb2d2512dedc -> Peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Couldn't send request to peer 235ad6af04c34c88a5261daa3604fada. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20250109 08:58:29.558359 23346 consensus_peers.cc:487] T 1365130ef97c4581b623c22c39753831 P ed1efbce9ee3401e99a5260edeae4e7c -> Peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Couldn't send request to peer 235ad6af04c34c88a5261daa3604fada. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20250109 08:58:29.562079 23420 consensus_peers.cc:487] T 8d3947300cfc4523acdab19fc992f26a P cf55383bef6649eb9e78cb2d2512dedc -> Peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Couldn't send request to peer 235ad6af04c34c88a5261daa3604fada. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20250109 08:58:29.563891 23420 consensus_peers.cc:487] T e6b9aaeda9a947d4a87fedd774301cc0 P cf55383bef6649eb9e78cb2d2512dedc -> Peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Couldn't send request to peer 235ad6af04c34c88a5261daa3604fada. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20250109 08:58:29.565505 23420 consensus_peers.cc:487] T e5251927614642639bd46ecd5373d2a4 P cf55383bef6649eb9e78cb2d2512dedc -> Peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Couldn't send request to peer 235ad6af04c34c88a5261daa3604fada. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20250109 08:58:29.578678 23420 consensus_peers.cc:487] T 5ccc0d42d4a148d7819366cea0b754a7 P cf55383bef6649eb9e78cb2d2512dedc -> Peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Couldn't send request to peer 235ad6af04c34c88a5261daa3604fada. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20250109 08:58:29.585038 23494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.226.193:42691, user_credentials={real_user=slave}} blocked reactor thread for 125874us
W20250109 08:58:29.589435 23420 consensus_peers.cc:487] T a09d8dffc2e446f1acd3c7f6cc1705bb P cf55383bef6649eb9e78cb2d2512dedc -> Peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Couldn't send request to peer 235ad6af04c34c88a5261daa3604fada. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20250109 08:58:29.614431 23494 consensus_peers.cc:487] T d87d3b4755344b82b0aec7ffd1ce785c P 05a74e1eb2474b3d8b3a2da1ef127e01 -> Peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Couldn't send request to peer 235ad6af04c34c88a5261daa3604fada. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20250109 08:58:29.626679 23346 consensus_peers.cc:487] T a54ca9f0d6e844e9a8b1d9cd0bb2a026 P ed1efbce9ee3401e99a5260edeae4e7c -> Peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Couldn't send request to peer 235ad6af04c34c88a5261daa3604fada. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20250109 08:58:29.630262 23420 consensus_peers.cc:487] T de98c70e25494d2a8e581443be836bd3 P cf55383bef6649eb9e78cb2d2512dedc -> Peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Couldn't send request to peer 235ad6af04c34c88a5261daa3604fada. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20250109 08:58:29.636853 23346 consensus_peers.cc:487] T 732e4aa69aae44e9b1153ac330a01a96 P ed1efbce9ee3401e99a5260edeae4e7c -> Peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Couldn't send request to peer 235ad6af04c34c88a5261daa3604fada. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20250109 08:58:29.644649 23494 consensus_peers.cc:487] T cc9886333c0343798226d59758a1ea5e P 05a74e1eb2474b3d8b3a2da1ef127e01 -> Peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Couldn't send request to peer 235ad6af04c34c88a5261daa3604fada. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20250109 08:58:29.646209 23494 consensus_peers.cc:487] T 970a82e754db47b6b1a19551d4ba43d8 P 05a74e1eb2474b3d8b3a2da1ef127e01 -> Peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Couldn't send request to peer 235ad6af04c34c88a5261daa3604fada. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20250109 08:58:29.645116 23346 consensus_peers.cc:487] T ed79047393d2425f8ecff31ea08b9cad P ed1efbce9ee3401e99a5260edeae4e7c -> Peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Couldn't send request to peer 235ad6af04c34c88a5261daa3604fada. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20250109 08:58:29.647416 23494 consensus_peers.cc:487] T 41e7f0cc38c4414ab88c92c1cdeb3984 P 05a74e1eb2474b3d8b3a2da1ef127e01 -> Peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Couldn't send request to peer 235ad6af04c34c88a5261daa3604fada. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20250109 08:58:29.649703 23494 consensus_peers.cc:487] T 5f0d500a91b54851bda52e5ccf741fd6 P 05a74e1eb2474b3d8b3a2da1ef127e01 -> Peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Couldn't send request to peer 235ad6af04c34c88a5261daa3604fada. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20250109 08:58:29.656203 23346 consensus_peers.cc:487] T 86b82e19759c4fd0b471a0dbff839aa5 P ed1efbce9ee3401e99a5260edeae4e7c -> Peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Couldn't send request to peer 235ad6af04c34c88a5261daa3604fada. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20250109 08:58:29.659739 23420 consensus_peers.cc:487] T 5c37c58abaa54656b31c72160b4dec9e P cf55383bef6649eb9e78cb2d2512dedc -> Peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Couldn't send request to peer 235ad6af04c34c88a5261daa3604fada. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20250109 08:58:29.660218 23494 consensus_peers.cc:487] T 29828d9befb845d5bd49a80e018502e3 P 05a74e1eb2474b3d8b3a2da1ef127e01 -> Peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Couldn't send request to peer 235ad6af04c34c88a5261daa3604fada. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20250109 08:58:29.662278 23346 consensus_peers.cc:487] T 1c5b5beea4a840eb9677097cacee3408 P ed1efbce9ee3401e99a5260edeae4e7c -> Peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Couldn't send request to peer 235ad6af04c34c88a5261daa3604fada. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:29.683192 20363 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250109 08:58:29.684116 20363 tablet_replica.cc:331] T 29828d9befb845d5bd49a80e018502e3 P 235ad6af04c34c88a5261daa3604fada: stopping tablet replica
I20250109 08:58:29.685149 20363 raft_consensus.cc:2238] T 29828d9befb845d5bd49a80e018502e3 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:29.685782 20363 raft_consensus.cc:2267] T 29828d9befb845d5bd49a80e018502e3 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:29.687981 20363 tablet_replica.cc:331] T 1365130ef97c4581b623c22c39753831 P 235ad6af04c34c88a5261daa3604fada: stopping tablet replica
I20250109 08:58:29.688802 20363 raft_consensus.cc:2238] T 1365130ef97c4581b623c22c39753831 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:29.689383 20363 raft_consensus.cc:2267] T 1365130ef97c4581b623c22c39753831 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:29.691915 20363 tablet_replica.cc:331] T 950e07e9cc794b8ab9ec5bd97d7e8e59 P 235ad6af04c34c88a5261daa3604fada: stopping tablet replica
I20250109 08:58:29.692536 20363 raft_consensus.cc:2238] T 950e07e9cc794b8ab9ec5bd97d7e8e59 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:29.692942 20363 raft_consensus.cc:2267] T 950e07e9cc794b8ab9ec5bd97d7e8e59 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:29.695518 20363 tablet_replica.cc:331] T 060daed1d3334c6188413dd381b82bf9 P 235ad6af04c34c88a5261daa3604fada: stopping tablet replica
I20250109 08:58:29.696314 20363 raft_consensus.cc:2238] T 060daed1d3334c6188413dd381b82bf9 P 235ad6af04c34c88a5261daa3604fada [term 1 LEADER]: Raft consensus shutting down.
I20250109 08:58:29.697194 20363 raft_consensus.cc:2267] T 060daed1d3334c6188413dd381b82bf9 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:29.699173 20363 tablet_replica.cc:331] T ee703ea575234c659da04f21ef3cdd70 P 235ad6af04c34c88a5261daa3604fada: stopping tablet replica
I20250109 08:58:29.699705 20363 raft_consensus.cc:2238] T ee703ea575234c659da04f21ef3cdd70 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:29.700089 20363 raft_consensus.cc:2267] T ee703ea575234c659da04f21ef3cdd70 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:29.702420 20363 tablet_replica.cc:331] T 86b82e19759c4fd0b471a0dbff839aa5 P 235ad6af04c34c88a5261daa3604fada: stopping tablet replica
I20250109 08:58:29.703327 20363 raft_consensus.cc:2238] T 86b82e19759c4fd0b471a0dbff839aa5 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:29.704048 20363 raft_consensus.cc:2267] T 86b82e19759c4fd0b471a0dbff839aa5 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:29.707424 20363 tablet_replica.cc:331] T a7d2ab61d238471c9f287ac1d5db4ed0 P 235ad6af04c34c88a5261daa3604fada: stopping tablet replica
W20250109 08:58:29.708334 23346 consensus_peers.cc:487] T 08214ee9c4c14bda92bdc6fa8231c313 P ed1efbce9ee3401e99a5260edeae4e7c -> Peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Couldn't send request to peer 235ad6af04c34c88a5261daa3604fada. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:29.708544 20363 raft_consensus.cc:2238] T a7d2ab61d238471c9f287ac1d5db4ed0 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:29.709170 20363 raft_consensus.cc:2267] T a7d2ab61d238471c9f287ac1d5db4ed0 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus is shut down!
W20250109 08:58:29.710966 23420 consensus_peers.cc:487] T a7d2ab61d238471c9f287ac1d5db4ed0 P cf55383bef6649eb9e78cb2d2512dedc -> Peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Couldn't send request to peer 235ad6af04c34c88a5261daa3604fada. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:29.712050 20363 tablet_replica.cc:331] T 86d4074509744900abe94402467a7d85 P 235ad6af04c34c88a5261daa3604fada: stopping tablet replica
I20250109 08:58:29.712720 20363 raft_consensus.cc:2238] T 86d4074509744900abe94402467a7d85 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:29.713086 20363 raft_consensus.cc:2267] T 86d4074509744900abe94402467a7d85 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:29.715104 20363 tablet_replica.cc:331] T 08214ee9c4c14bda92bdc6fa8231c313 P 235ad6af04c34c88a5261daa3604fada: stopping tablet replica
I20250109 08:58:29.715735 20363 raft_consensus.cc:2238] T 08214ee9c4c14bda92bdc6fa8231c313 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:29.716228 20363 raft_consensus.cc:2267] T 08214ee9c4c14bda92bdc6fa8231c313 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:29.719056 20363 tablet_replica.cc:331] T 5f0d500a91b54851bda52e5ccf741fd6 P 235ad6af04c34c88a5261daa3604fada: stopping tablet replica
I20250109 08:58:29.720096 20363 raft_consensus.cc:2238] T 5f0d500a91b54851bda52e5ccf741fd6 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:29.720630 20363 raft_consensus.cc:2267] T 5f0d500a91b54851bda52e5ccf741fd6 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:29.723104 20363 tablet_replica.cc:331] T a09d8dffc2e446f1acd3c7f6cc1705bb P 235ad6af04c34c88a5261daa3604fada: stopping tablet replica
I20250109 08:58:29.723784 20363 raft_consensus.cc:2238] T a09d8dffc2e446f1acd3c7f6cc1705bb P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:29.724249 20363 raft_consensus.cc:2267] T a09d8dffc2e446f1acd3c7f6cc1705bb P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:29.726683 20363 tablet_replica.cc:331] T 1c5b5beea4a840eb9677097cacee3408 P 235ad6af04c34c88a5261daa3604fada: stopping tablet replica
I20250109 08:58:29.727273 20363 raft_consensus.cc:2238] T 1c5b5beea4a840eb9677097cacee3408 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:29.727701 20363 raft_consensus.cc:2267] T 1c5b5beea4a840eb9677097cacee3408 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:29.729794 20363 tablet_replica.cc:331] T 94f2be307f704c75bceb213e52307cfb P 235ad6af04c34c88a5261daa3604fada: stopping tablet replica
I20250109 08:58:29.730343 20363 raft_consensus.cc:2238] T 94f2be307f704c75bceb213e52307cfb P 235ad6af04c34c88a5261daa3604fada [term 1 LEADER]: Raft consensus shutting down.
I20250109 08:58:29.731107 20363 raft_consensus.cc:2267] T 94f2be307f704c75bceb213e52307cfb P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:29.732945 20363 tablet_replica.cc:331] T 41e7f0cc38c4414ab88c92c1cdeb3984 P 235ad6af04c34c88a5261daa3604fada: stopping tablet replica
I20250109 08:58:29.733482 20363 raft_consensus.cc:2238] T 41e7f0cc38c4414ab88c92c1cdeb3984 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:29.733891 20363 raft_consensus.cc:2267] T 41e7f0cc38c4414ab88c92c1cdeb3984 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:29.735808 20363 tablet_replica.cc:331] T e5251927614642639bd46ecd5373d2a4 P 235ad6af04c34c88a5261daa3604fada: stopping tablet replica
I20250109 08:58:29.736399 20363 raft_consensus.cc:2238] T e5251927614642639bd46ecd5373d2a4 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:29.736855 20363 raft_consensus.cc:2267] T e5251927614642639bd46ecd5373d2a4 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:29.738678 20363 tablet_replica.cc:331] T ea2b0762940e44dbbf69b9a163ace846 P 235ad6af04c34c88a5261daa3604fada: stopping tablet replica
I20250109 08:58:29.739214 20363 raft_consensus.cc:2238] T ea2b0762940e44dbbf69b9a163ace846 P 235ad6af04c34c88a5261daa3604fada [term 1 LEADER]: Raft consensus shutting down.
I20250109 08:58:29.740139 20363 raft_consensus.cc:2267] T ea2b0762940e44dbbf69b9a163ace846 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:29.743306 20363 tablet_replica.cc:331] T 24ce131f7bb143bea1581d5b0c2a7109 P 235ad6af04c34c88a5261daa3604fada: stopping tablet replica
I20250109 08:58:29.744072 20363 raft_consensus.cc:2238] T 24ce131f7bb143bea1581d5b0c2a7109 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:29.744731 20363 raft_consensus.cc:2267] T 24ce131f7bb143bea1581d5b0c2a7109 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:29.747822 20363 tablet_replica.cc:331] T 732e4aa69aae44e9b1153ac330a01a96 P 235ad6af04c34c88a5261daa3604fada: stopping tablet replica
I20250109 08:58:29.748704 20363 raft_consensus.cc:2238] T 732e4aa69aae44e9b1153ac330a01a96 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:29.749420 20363 raft_consensus.cc:2267] T 732e4aa69aae44e9b1153ac330a01a96 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:29.752434 20363 tablet_replica.cc:331] T 5ccc0d42d4a148d7819366cea0b754a7 P 235ad6af04c34c88a5261daa3604fada: stopping tablet replica
I20250109 08:58:29.752988 20363 raft_consensus.cc:2238] T 5ccc0d42d4a148d7819366cea0b754a7 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:29.753414 20363 raft_consensus.cc:2267] T 5ccc0d42d4a148d7819366cea0b754a7 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:29.755652 20363 tablet_replica.cc:331] T e6b9aaeda9a947d4a87fedd774301cc0 P 235ad6af04c34c88a5261daa3604fada: stopping tablet replica
I20250109 08:58:29.756440 20363 raft_consensus.cc:2238] T e6b9aaeda9a947d4a87fedd774301cc0 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:29.757046 20363 raft_consensus.cc:2267] T e6b9aaeda9a947d4a87fedd774301cc0 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:29.759848 20363 tablet_replica.cc:331] T 77597d68423749e7b87f9f0db7a1ff3b P 235ad6af04c34c88a5261daa3604fada: stopping tablet replica
I20250109 08:58:29.760605 20363 raft_consensus.cc:2238] T 77597d68423749e7b87f9f0db7a1ff3b P 235ad6af04c34c88a5261daa3604fada [term 1 LEADER]: Raft consensus shutting down.
I20250109 08:58:29.761471 20363 raft_consensus.cc:2267] T 77597d68423749e7b87f9f0db7a1ff3b P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:29.763355 20363 tablet_replica.cc:331] T d714271c7ea64c0b8b13fcf63591ec07 P 235ad6af04c34c88a5261daa3604fada: stopping tablet replica
I20250109 08:58:29.763883 20363 raft_consensus.cc:2238] T d714271c7ea64c0b8b13fcf63591ec07 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:29.764283 20363 raft_consensus.cc:2267] T d714271c7ea64c0b8b13fcf63591ec07 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:29.766217 20363 tablet_replica.cc:331] T db111e41c1be497e8e673a2c42b9ba66 P 235ad6af04c34c88a5261daa3604fada: stopping tablet replica
I20250109 08:58:29.766753 20363 raft_consensus.cc:2238] T db111e41c1be497e8e673a2c42b9ba66 P 235ad6af04c34c88a5261daa3604fada [term 1 LEADER]: Raft consensus shutting down.
I20250109 08:58:29.767634 20363 raft_consensus.cc:2267] T db111e41c1be497e8e673a2c42b9ba66 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:29.770287 20363 tablet_replica.cc:331] T 6eef9bb3c645428d9f1cb843bae9a264 P 235ad6af04c34c88a5261daa3604fada: stopping tablet replica
I20250109 08:58:29.771034 20363 raft_consensus.cc:2238] T 6eef9bb3c645428d9f1cb843bae9a264 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:29.771523 20363 raft_consensus.cc:2267] T 6eef9bb3c645428d9f1cb843bae9a264 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:29.773981 20363 tablet_replica.cc:331] T a54ca9f0d6e844e9a8b1d9cd0bb2a026 P 235ad6af04c34c88a5261daa3604fada: stopping tablet replica
I20250109 08:58:29.774691 20363 raft_consensus.cc:2238] T a54ca9f0d6e844e9a8b1d9cd0bb2a026 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:29.775291 20363 raft_consensus.cc:2267] T a54ca9f0d6e844e9a8b1d9cd0bb2a026 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:29.778185 20363 tablet_replica.cc:331] T f1f9274aeda748f493f5752814fb262d P 235ad6af04c34c88a5261daa3604fada: stopping tablet replica
W20250109 08:58:29.778131 23494 consensus_peers.cc:487] T 82ea7dc042dd4d05b260d508f4422c3c P 05a74e1eb2474b3d8b3a2da1ef127e01 -> Peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Couldn't send request to peer 235ad6af04c34c88a5261daa3604fada. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:29.779072 20363 raft_consensus.cc:2238] T f1f9274aeda748f493f5752814fb262d P 235ad6af04c34c88a5261daa3604fada [term 1 LEADER]: Raft consensus shutting down.
I20250109 08:58:29.780023 20363 raft_consensus.cc:2267] T f1f9274aeda748f493f5752814fb262d P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus is shut down!
W20250109 08:58:29.781368 23346 consensus_peers.cc:487] T ee703ea575234c659da04f21ef3cdd70 P ed1efbce9ee3401e99a5260edeae4e7c -> Peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Couldn't send request to peer 235ad6af04c34c88a5261daa3604fada. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:29.782179 20363 tablet_replica.cc:331] T 1340cf8ee2cb423f875276137da17e8b P 235ad6af04c34c88a5261daa3604fada: stopping tablet replica
I20250109 08:58:29.782799 20363 raft_consensus.cc:2238] T 1340cf8ee2cb423f875276137da17e8b P 235ad6af04c34c88a5261daa3604fada [term 1 LEADER]: Raft consensus shutting down.
I20250109 08:58:29.783565 20363 raft_consensus.cc:2267] T 1340cf8ee2cb423f875276137da17e8b P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:29.785590 20363 tablet_replica.cc:331] T 4df64aae69ca41dc91286cae1603000f P 235ad6af04c34c88a5261daa3604fada: stopping tablet replica
I20250109 08:58:29.786096 20363 raft_consensus.cc:2238] T 4df64aae69ca41dc91286cae1603000f P 235ad6af04c34c88a5261daa3604fada [term 1 LEADER]: Raft consensus shutting down.
I20250109 08:58:29.786864 20363 raft_consensus.cc:2267] T 4df64aae69ca41dc91286cae1603000f P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:29.788756 20363 tablet_replica.cc:331] T b572a6a3dfc04ea199bc703ddad8af67 P 235ad6af04c34c88a5261daa3604fada: stopping tablet replica
I20250109 08:58:29.789239 20363 raft_consensus.cc:2238] T b572a6a3dfc04ea199bc703ddad8af67 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:29.789606 20363 raft_consensus.cc:2267] T b572a6a3dfc04ea199bc703ddad8af67 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:29.791604 20363 tablet_replica.cc:331] T 444507f7812441ceaeebc44bc8f06293 P 235ad6af04c34c88a5261daa3604fada: stopping tablet replica
I20250109 08:58:29.792222 20363 raft_consensus.cc:2238] T 444507f7812441ceaeebc44bc8f06293 P 235ad6af04c34c88a5261daa3604fada [term 1 LEADER]: Raft consensus shutting down.
I20250109 08:58:29.793300 20363 raft_consensus.cc:2267] T 444507f7812441ceaeebc44bc8f06293 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus is shut down!
W20250109 08:58:29.794338 23420 consensus_peers.cc:487] T bd6fcd838b0249cabd5f1f95e5538ab6 P cf55383bef6649eb9e78cb2d2512dedc -> Peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Couldn't send request to peer 235ad6af04c34c88a5261daa3604fada. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:29.795785 20363 tablet_replica.cc:331] T 5c37c58abaa54656b31c72160b4dec9e P 235ad6af04c34c88a5261daa3604fada: stopping tablet replica
I20250109 08:58:29.796648 20363 raft_consensus.cc:2238] T 5c37c58abaa54656b31c72160b4dec9e P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:29.797144 20363 raft_consensus.cc:2267] T 5c37c58abaa54656b31c72160b4dec9e P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:29.799010 20363 tablet_replica.cc:331] T 3208a99957554f6db341654856980c8a P 235ad6af04c34c88a5261daa3604fada: stopping tablet replica
I20250109 08:58:29.799721 20363 raft_consensus.cc:2238] T 3208a99957554f6db341654856980c8a P 235ad6af04c34c88a5261daa3604fada [term 1 LEADER]: Raft consensus shutting down.
I20250109 08:58:29.800563 20363 raft_consensus.cc:2267] T 3208a99957554f6db341654856980c8a P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:29.802695 20363 tablet_replica.cc:331] T de98c70e25494d2a8e581443be836bd3 P 235ad6af04c34c88a5261daa3604fada: stopping tablet replica
I20250109 08:58:29.803182 20363 raft_consensus.cc:2238] T de98c70e25494d2a8e581443be836bd3 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:29.803545 20363 raft_consensus.cc:2267] T de98c70e25494d2a8e581443be836bd3 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:29.805449 20363 tablet_replica.cc:331] T fc1038255dd64cb5ba2425037d22f1f7 P 235ad6af04c34c88a5261daa3604fada: stopping tablet replica
I20250109 08:58:29.806069 20363 raft_consensus.cc:2238] T fc1038255dd64cb5ba2425037d22f1f7 P 235ad6af04c34c88a5261daa3604fada [term 1 LEADER]: Raft consensus shutting down.
I20250109 08:58:29.806947 20363 raft_consensus.cc:2267] T fc1038255dd64cb5ba2425037d22f1f7 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:29.808832 20363 tablet_replica.cc:331] T 9b245cb3666c421c884cf20161a22831 P 235ad6af04c34c88a5261daa3604fada: stopping tablet replica
I20250109 08:58:29.809264 20363 raft_consensus.cc:2238] T 9b245cb3666c421c884cf20161a22831 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:29.809624 20363 raft_consensus.cc:2267] T 9b245cb3666c421c884cf20161a22831 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:29.811344 20363 tablet_replica.cc:331] T bfb140bb3a8b44bd8b4e1fcf0946162b P 235ad6af04c34c88a5261daa3604fada: stopping tablet replica
I20250109 08:58:29.811790 20363 raft_consensus.cc:2238] T bfb140bb3a8b44bd8b4e1fcf0946162b P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:29.812141 20363 raft_consensus.cc:2267] T bfb140bb3a8b44bd8b4e1fcf0946162b P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:29.813822 20363 tablet_replica.cc:331] T ed79047393d2425f8ecff31ea08b9cad P 235ad6af04c34c88a5261daa3604fada: stopping tablet replica
I20250109 08:58:29.814287 20363 raft_consensus.cc:2238] T ed79047393d2425f8ecff31ea08b9cad P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:29.814651 20363 raft_consensus.cc:2267] T ed79047393d2425f8ecff31ea08b9cad P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:29.816610 20363 tablet_replica.cc:331] T 2ab70546dda143b5b5db15c3875aa348 P 235ad6af04c34c88a5261daa3604fada: stopping tablet replica
I20250109 08:58:29.817153 20363 raft_consensus.cc:2238] T 2ab70546dda143b5b5db15c3875aa348 P 235ad6af04c34c88a5261daa3604fada [term 1 LEADER]: Raft consensus shutting down.
I20250109 08:58:29.818164 20363 raft_consensus.cc:2267] T 2ab70546dda143b5b5db15c3875aa348 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:29.820825 20363 tablet_replica.cc:331] T 8d3947300cfc4523acdab19fc992f26a P 235ad6af04c34c88a5261daa3604fada: stopping tablet replica
I20250109 08:58:29.821547 20363 raft_consensus.cc:2238] T 8d3947300cfc4523acdab19fc992f26a P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:29.822081 20363 raft_consensus.cc:2267] T 8d3947300cfc4523acdab19fc992f26a P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:29.824728 20363 tablet_replica.cc:331] T d87d3b4755344b82b0aec7ffd1ce785c P 235ad6af04c34c88a5261daa3604fada: stopping tablet replica
I20250109 08:58:29.825413 20363 raft_consensus.cc:2238] T d87d3b4755344b82b0aec7ffd1ce785c P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:29.826017 20363 raft_consensus.cc:2267] T d87d3b4755344b82b0aec7ffd1ce785c P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:29.828900 20363 tablet_replica.cc:331] T cc9886333c0343798226d59758a1ea5e P 235ad6af04c34c88a5261daa3604fada: stopping tablet replica
I20250109 08:58:29.829603 20363 raft_consensus.cc:2238] T cc9886333c0343798226d59758a1ea5e P 235ad6af04c34c88a5261daa3604fada [term 3 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:29.830140 20363 raft_consensus.cc:2267] T cc9886333c0343798226d59758a1ea5e P 235ad6af04c34c88a5261daa3604fada [term 3 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:29.832299 20363 tablet_replica.cc:331] T 82ea7dc042dd4d05b260d508f4422c3c P 235ad6af04c34c88a5261daa3604fada: stopping tablet replica
I20250109 08:58:29.832835 20363 raft_consensus.cc:2238] T 82ea7dc042dd4d05b260d508f4422c3c P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:29.833257 20363 raft_consensus.cc:2267] T 82ea7dc042dd4d05b260d508f4422c3c P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:29.835489 20363 tablet_replica.cc:331] T 970a82e754db47b6b1a19551d4ba43d8 P 235ad6af04c34c88a5261daa3604fada: stopping tablet replica
I20250109 08:58:29.835996 20363 raft_consensus.cc:2238] T 970a82e754db47b6b1a19551d4ba43d8 P 235ad6af04c34c88a5261daa3604fada [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:29.836472 20363 raft_consensus.cc:2267] T 970a82e754db47b6b1a19551d4ba43d8 P 235ad6af04c34c88a5261daa3604fada [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:29.838479 20363 tablet_replica.cc:331] T bd6fcd838b0249cabd5f1f95e5538ab6 P 235ad6af04c34c88a5261daa3604fada: stopping tablet replica
I20250109 08:58:29.838943 20363 raft_consensus.cc:2238] T bd6fcd838b0249cabd5f1f95e5538ab6 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:29.839351 20363 raft_consensus.cc:2267] T bd6fcd838b0249cabd5f1f95e5538ab6 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:29.841629 20363 tablet_replica.cc:331] T 2e3d34c0d09843d2be54e47f3ccc16d8 P 235ad6af04c34c88a5261daa3604fada: stopping tablet replica
I20250109 08:58:29.842185 20363 raft_consensus.cc:2238] T 2e3d34c0d09843d2be54e47f3ccc16d8 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:29.842770 20363 raft_consensus.cc:2267] T 2e3d34c0d09843d2be54e47f3ccc16d8 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus is shut down!
W20250109 08:58:29.845194 23494 consensus_peers.cc:487] T 9b245cb3666c421c884cf20161a22831 P 05a74e1eb2474b3d8b3a2da1ef127e01 -> Peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Couldn't send request to peer 235ad6af04c34c88a5261daa3604fada. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20250109 08:58:29.867177 23494 consensus_peers.cc:487] T 950e07e9cc794b8ab9ec5bd97d7e8e59 P 05a74e1eb2474b3d8b3a2da1ef127e01 -> Peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Couldn't send request to peer 235ad6af04c34c88a5261daa3604fada. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20250109 08:58:29.887015 23346 consensus_peers.cc:487] T 86d4074509744900abe94402467a7d85 P ed1efbce9ee3401e99a5260edeae4e7c -> Peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Couldn't send request to peer 235ad6af04c34c88a5261daa3604fada. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20250109 08:58:29.893996 23494 consensus_peers.cc:487] T 24ce131f7bb143bea1581d5b0c2a7109 P 05a74e1eb2474b3d8b3a2da1ef127e01 -> Peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Couldn't send request to peer 235ad6af04c34c88a5261daa3604fada. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:29.917554 20363 tablet_server.cc:195] TabletServer@127.19.226.193:0 shutdown complete.
W20250109 08:58:29.925815 23420 consensus_peers.cc:487] T bfb140bb3a8b44bd8b4e1fcf0946162b P cf55383bef6649eb9e78cb2d2512dedc -> Peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Couldn't send request to peer 235ad6af04c34c88a5261daa3604fada. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.193:42691: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20250109 08:58:29.960659 23494 consensus_peers.cc:487] T b572a6a3dfc04ea199bc703ddad8af67 P 05a74e1eb2474b3d8b3a2da1ef127e01 -> Peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Couldn't send request to peer 235ad6af04c34c88a5261daa3604fada. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.193:42691: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:30.012120 20363 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250109 08:58:30.024921 24101 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250109 08:58:30.025671 24100 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250109 08:58:30.048010 24103 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250109 08:58:30.054167 20363 server_base.cc:1034] running on GCE node
I20250109 08:58:30.060177 20363 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250109 08:58:30.060575 20363 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250109 08:58:30.060779 20363 hybrid_clock.cc:648] HybridClock initialized: now 1736413110060761 us; error 0 us; skew 500 ppm
I20250109 08:58:30.061285 20363 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250109 08:58:30.064563 20363 webserver.cc:458] Webserver started at http://127.19.226.193:38143/ using document root <none> and password file <none>
I20250109 08:58:30.065058 20363 fs_manager.cc:362] Metadata directory not provided
I20250109 08:58:30.065243 20363 fs_manager.cc:365] Using existing metadata directory in first data directory
I20250109 08:58:30.068977 20363 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20250109 08:58:30.076360 24108 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250109 08:58:30.077235 20363 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20250109 08:58:30.077606 20363 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1736412987538469-20363-0/minicluster-data/ts-0-root
uuid: "235ad6af04c34c88a5261daa3604fada"
format_stamp: "Formatted at 2025-01-09 08:58:18 on dist-test-slave-g8ds"
I20250109 08:58:30.078187 20363 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1736412987538469-20363-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1736412987538469-20363-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1736412987538469-20363-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250109 08:58:30.092778 20363 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250109 08:58:30.094074 20363 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250109 08:58:30.095499 20363 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250109 08:58:30.099840 24115 ts_tablet_manager.cc:542] Loading tablet metadata (0/45 complete)
I20250109 08:58:30.346446 20363 ts_tablet_manager.cc:579] Loaded tablet metadata (45 total tablets, 45 live tablets)
I20250109 08:58:30.346803 20363 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.248s user 0.008s sys 0.001s
I20250109 08:58:30.347141 20363 ts_tablet_manager.cc:594] Registering tablets (0/45 complete)
I20250109 08:58:30.352723 24115 tablet_bootstrap.cc:492] T a54ca9f0d6e844e9a8b1d9cd0bb2a026 P 235ad6af04c34c88a5261daa3604fada: Bootstrap starting.
I20250109 08:58:30.380002 24115 tablet_bootstrap.cc:492] T a54ca9f0d6e844e9a8b1d9cd0bb2a026 P 235ad6af04c34c88a5261daa3604fada: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250109 08:58:30.380843 24115 tablet_bootstrap.cc:492] T a54ca9f0d6e844e9a8b1d9cd0bb2a026 P 235ad6af04c34c88a5261daa3604fada: Bootstrap complete.
I20250109 08:58:30.381461 24115 ts_tablet_manager.cc:1397] T a54ca9f0d6e844e9a8b1d9cd0bb2a026 P 235ad6af04c34c88a5261daa3604fada: Time spent bootstrapping tablet: real 0.029s user 0.009s sys 0.019s
I20250109 08:58:30.384089 24115 raft_consensus.cc:357] T a54ca9f0d6e844e9a8b1d9cd0bb2a026 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:30.385149 24115 raft_consensus.cc:738] T a54ca9f0d6e844e9a8b1d9cd0bb2a026 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Initialized, Role: FOLLOWER
I20250109 08:58:30.385929 24115 consensus_queue.cc:260] T a54ca9f0d6e844e9a8b1d9cd0bb2a026 P 235ad6af04c34c88a5261daa3604fada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:30.395085 24115 ts_tablet_manager.cc:1428] T a54ca9f0d6e844e9a8b1d9cd0bb2a026 P 235ad6af04c34c88a5261daa3604fada: Time spent starting tablet: real 0.013s user 0.009s sys 0.005s
I20250109 08:58:30.395932 24115 tablet_bootstrap.cc:492] T 950e07e9cc794b8ab9ec5bd97d7e8e59 P 235ad6af04c34c88a5261daa3604fada: Bootstrap starting.
I20250109 08:58:30.407887 24115 tablet_bootstrap.cc:492] T 950e07e9cc794b8ab9ec5bd97d7e8e59 P 235ad6af04c34c88a5261daa3604fada: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250109 08:58:30.408574 24115 tablet_bootstrap.cc:492] T 950e07e9cc794b8ab9ec5bd97d7e8e59 P 235ad6af04c34c88a5261daa3604fada: Bootstrap complete.
I20250109 08:58:30.409040 24115 ts_tablet_manager.cc:1397] T 950e07e9cc794b8ab9ec5bd97d7e8e59 P 235ad6af04c34c88a5261daa3604fada: Time spent bootstrapping tablet: real 0.013s user 0.012s sys 0.000s
I20250109 08:58:30.410734 24115 raft_consensus.cc:357] T 950e07e9cc794b8ab9ec5bd97d7e8e59 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:30.411203 24115 raft_consensus.cc:738] T 950e07e9cc794b8ab9ec5bd97d7e8e59 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Initialized, Role: FOLLOWER
I20250109 08:58:30.411677 24115 consensus_queue.cc:260] T 950e07e9cc794b8ab9ec5bd97d7e8e59 P 235ad6af04c34c88a5261daa3604fada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:30.413100 24115 ts_tablet_manager.cc:1428] T 950e07e9cc794b8ab9ec5bd97d7e8e59 P 235ad6af04c34c88a5261daa3604fada: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20250109 08:58:30.413692 24115 tablet_bootstrap.cc:492] T db111e41c1be497e8e673a2c42b9ba66 P 235ad6af04c34c88a5261daa3604fada: Bootstrap starting.
I20250109 08:58:30.426484 24115 tablet_bootstrap.cc:492] T db111e41c1be497e8e673a2c42b9ba66 P 235ad6af04c34c88a5261daa3604fada: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250109 08:58:30.428473 24115 tablet_bootstrap.cc:492] T db111e41c1be497e8e673a2c42b9ba66 P 235ad6af04c34c88a5261daa3604fada: Bootstrap complete.
I20250109 08:58:30.429095 24115 ts_tablet_manager.cc:1397] T db111e41c1be497e8e673a2c42b9ba66 P 235ad6af04c34c88a5261daa3604fada: Time spent bootstrapping tablet: real 0.016s user 0.009s sys 0.003s
I20250109 08:58:30.431376 24115 raft_consensus.cc:357] T db111e41c1be497e8e673a2c42b9ba66 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:30.432077 24115 raft_consensus.cc:738] T db111e41c1be497e8e673a2c42b9ba66 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Initialized, Role: FOLLOWER
I20250109 08:58:30.433584 24115 consensus_queue.cc:260] T db111e41c1be497e8e673a2c42b9ba66 P 235ad6af04c34c88a5261daa3604fada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:30.435485 24115 ts_tablet_manager.cc:1428] T db111e41c1be497e8e673a2c42b9ba66 P 235ad6af04c34c88a5261daa3604fada: Time spent starting tablet: real 0.006s user 0.003s sys 0.001s
I20250109 08:58:30.436249 24115 tablet_bootstrap.cc:492] T 732e4aa69aae44e9b1153ac330a01a96 P 235ad6af04c34c88a5261daa3604fada: Bootstrap starting.
I20250109 08:58:30.451215 24115 tablet_bootstrap.cc:492] T 732e4aa69aae44e9b1153ac330a01a96 P 235ad6af04c34c88a5261daa3604fada: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250109 08:58:30.452011 24115 tablet_bootstrap.cc:492] T 732e4aa69aae44e9b1153ac330a01a96 P 235ad6af04c34c88a5261daa3604fada: Bootstrap complete.
I20250109 08:58:30.452538 24115 ts_tablet_manager.cc:1397] T 732e4aa69aae44e9b1153ac330a01a96 P 235ad6af04c34c88a5261daa3604fada: Time spent bootstrapping tablet: real 0.016s user 0.015s sys 0.000s
I20250109 08:58:30.454211 24115 raft_consensus.cc:357] T 732e4aa69aae44e9b1153ac330a01a96 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:30.454763 24115 raft_consensus.cc:738] T 732e4aa69aae44e9b1153ac330a01a96 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Initialized, Role: FOLLOWER
I20250109 08:58:30.455348 24115 consensus_queue.cc:260] T 732e4aa69aae44e9b1153ac330a01a96 P 235ad6af04c34c88a5261daa3604fada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:30.457458 24115 ts_tablet_manager.cc:1428] T 732e4aa69aae44e9b1153ac330a01a96 P 235ad6af04c34c88a5261daa3604fada: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20250109 08:58:30.458217 24115 tablet_bootstrap.cc:492] T 86b82e19759c4fd0b471a0dbff839aa5 P 235ad6af04c34c88a5261daa3604fada: Bootstrap starting.
I20250109 08:58:30.470467 24115 tablet_bootstrap.cc:492] T 86b82e19759c4fd0b471a0dbff839aa5 P 235ad6af04c34c88a5261daa3604fada: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250109 08:58:30.471289 24115 tablet_bootstrap.cc:492] T 86b82e19759c4fd0b471a0dbff839aa5 P 235ad6af04c34c88a5261daa3604fada: Bootstrap complete.
I20250109 08:58:30.471880 24115 ts_tablet_manager.cc:1397] T 86b82e19759c4fd0b471a0dbff839aa5 P 235ad6af04c34c88a5261daa3604fada: Time spent bootstrapping tablet: real 0.014s user 0.010s sys 0.000s
I20250109 08:58:30.474167 24115 raft_consensus.cc:357] T 86b82e19759c4fd0b471a0dbff839aa5 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:30.474829 24115 raft_consensus.cc:738] T 86b82e19759c4fd0b471a0dbff839aa5 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Initialized, Role: FOLLOWER
I20250109 08:58:30.475617 24115 consensus_queue.cc:260] T 86b82e19759c4fd0b471a0dbff839aa5 P 235ad6af04c34c88a5261daa3604fada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:30.477545 24115 ts_tablet_manager.cc:1428] T 86b82e19759c4fd0b471a0dbff839aa5 P 235ad6af04c34c88a5261daa3604fada: Time spent starting tablet: real 0.005s user 0.008s sys 0.000s
I20250109 08:58:30.478283 24115 tablet_bootstrap.cc:492] T b572a6a3dfc04ea199bc703ddad8af67 P 235ad6af04c34c88a5261daa3604fada: Bootstrap starting.
I20250109 08:58:30.491196 24115 tablet_bootstrap.cc:492] T b572a6a3dfc04ea199bc703ddad8af67 P 235ad6af04c34c88a5261daa3604fada: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250109 08:58:30.491880 24115 tablet_bootstrap.cc:492] T b572a6a3dfc04ea199bc703ddad8af67 P 235ad6af04c34c88a5261daa3604fada: Bootstrap complete.
I20250109 08:58:30.492452 24115 ts_tablet_manager.cc:1397] T b572a6a3dfc04ea199bc703ddad8af67 P 235ad6af04c34c88a5261daa3604fada: Time spent bootstrapping tablet: real 0.014s user 0.014s sys 0.000s
I20250109 08:58:30.494396 24115 raft_consensus.cc:357] T b572a6a3dfc04ea199bc703ddad8af67 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:30.494982 24115 raft_consensus.cc:738] T b572a6a3dfc04ea199bc703ddad8af67 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Initialized, Role: FOLLOWER
I20250109 08:58:30.495592 24115 consensus_queue.cc:260] T b572a6a3dfc04ea199bc703ddad8af67 P 235ad6af04c34c88a5261daa3604fada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:30.496990 24115 ts_tablet_manager.cc:1428] T b572a6a3dfc04ea199bc703ddad8af67 P 235ad6af04c34c88a5261daa3604fada: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20250109 08:58:30.497537 24115 tablet_bootstrap.cc:492] T 4df64aae69ca41dc91286cae1603000f P 235ad6af04c34c88a5261daa3604fada: Bootstrap starting.
I20250109 08:58:30.509012 24115 tablet_bootstrap.cc:492] T 4df64aae69ca41dc91286cae1603000f P 235ad6af04c34c88a5261daa3604fada: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250109 08:58:30.509634 24115 tablet_bootstrap.cc:492] T 4df64aae69ca41dc91286cae1603000f P 235ad6af04c34c88a5261daa3604fada: Bootstrap complete.
I20250109 08:58:30.510079 24115 ts_tablet_manager.cc:1397] T 4df64aae69ca41dc91286cae1603000f P 235ad6af04c34c88a5261daa3604fada: Time spent bootstrapping tablet: real 0.013s user 0.006s sys 0.004s
I20250109 08:58:30.511637 24115 raft_consensus.cc:357] T 4df64aae69ca41dc91286cae1603000f P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:30.512087 24115 raft_consensus.cc:738] T 4df64aae69ca41dc91286cae1603000f P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Initialized, Role: FOLLOWER
I20250109 08:58:30.512606 24115 consensus_queue.cc:260] T 4df64aae69ca41dc91286cae1603000f P 235ad6af04c34c88a5261daa3604fada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:30.513949 24115 ts_tablet_manager.cc:1428] T 4df64aae69ca41dc91286cae1603000f P 235ad6af04c34c88a5261daa3604fada: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20250109 08:58:30.514501 24115 tablet_bootstrap.cc:492] T 5ccc0d42d4a148d7819366cea0b754a7 P 235ad6af04c34c88a5261daa3604fada: Bootstrap starting.
I20250109 08:58:30.526046 24115 tablet_bootstrap.cc:492] T 5ccc0d42d4a148d7819366cea0b754a7 P 235ad6af04c34c88a5261daa3604fada: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250109 08:58:30.526973 24115 tablet_bootstrap.cc:492] T 5ccc0d42d4a148d7819366cea0b754a7 P 235ad6af04c34c88a5261daa3604fada: Bootstrap complete.
I20250109 08:58:30.527609 24115 ts_tablet_manager.cc:1397] T 5ccc0d42d4a148d7819366cea0b754a7 P 235ad6af04c34c88a5261daa3604fada: Time spent bootstrapping tablet: real 0.013s user 0.009s sys 0.000s
I20250109 08:58:30.529925 24115 raft_consensus.cc:357] T 5ccc0d42d4a148d7819366cea0b754a7 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:30.530630 24115 raft_consensus.cc:738] T 5ccc0d42d4a148d7819366cea0b754a7 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Initialized, Role: FOLLOWER
I20250109 08:58:30.531256 24115 consensus_queue.cc:260] T 5ccc0d42d4a148d7819366cea0b754a7 P 235ad6af04c34c88a5261daa3604fada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:30.533360 24115 ts_tablet_manager.cc:1428] T 5ccc0d42d4a148d7819366cea0b754a7 P 235ad6af04c34c88a5261daa3604fada: Time spent starting tablet: real 0.005s user 0.008s sys 0.000s
I20250109 08:58:30.534134 24115 tablet_bootstrap.cc:492] T 444507f7812441ceaeebc44bc8f06293 P 235ad6af04c34c88a5261daa3604fada: Bootstrap starting.
I20250109 08:58:30.545207 24115 tablet_bootstrap.cc:492] T 444507f7812441ceaeebc44bc8f06293 P 235ad6af04c34c88a5261daa3604fada: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250109 08:58:30.545855 24115 tablet_bootstrap.cc:492] T 444507f7812441ceaeebc44bc8f06293 P 235ad6af04c34c88a5261daa3604fada: Bootstrap complete.
I20250109 08:58:30.546327 24115 ts_tablet_manager.cc:1397] T 444507f7812441ceaeebc44bc8f06293 P 235ad6af04c34c88a5261daa3604fada: Time spent bootstrapping tablet: real 0.012s user 0.010s sys 0.000s
I20250109 08:58:30.548079 24115 raft_consensus.cc:357] T 444507f7812441ceaeebc44bc8f06293 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:30.548743 24115 raft_consensus.cc:738] T 444507f7812441ceaeebc44bc8f06293 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Initialized, Role: FOLLOWER
I20250109 08:58:30.549237 24115 consensus_queue.cc:260] T 444507f7812441ceaeebc44bc8f06293 P 235ad6af04c34c88a5261daa3604fada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:30.550700 24115 ts_tablet_manager.cc:1428] T 444507f7812441ceaeebc44bc8f06293 P 235ad6af04c34c88a5261daa3604fada: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20250109 08:58:30.551227 24115 tablet_bootstrap.cc:492] T 8d3947300cfc4523acdab19fc992f26a P 235ad6af04c34c88a5261daa3604fada: Bootstrap starting.
I20250109 08:58:30.553864 20363 ts_tablet_manager.cc:610] Registered 45 tablets
I20250109 08:58:30.554137 20363 ts_tablet_manager.cc:589] Time spent register tablets: real 0.207s user 0.200s sys 0.000s
I20250109 08:58:30.563014 24115 tablet_bootstrap.cc:492] T 8d3947300cfc4523acdab19fc992f26a P 235ad6af04c34c88a5261daa3604fada: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250109 08:58:30.564249 24115 tablet_bootstrap.cc:492] T 8d3947300cfc4523acdab19fc992f26a P 235ad6af04c34c88a5261daa3604fada: Bootstrap complete.
I20250109 08:58:30.565053 24115 ts_tablet_manager.cc:1397] T 8d3947300cfc4523acdab19fc992f26a P 235ad6af04c34c88a5261daa3604fada: Time spent bootstrapping tablet: real 0.014s user 0.012s sys 0.000s
I20250109 08:58:30.568260 24115 raft_consensus.cc:357] T 8d3947300cfc4523acdab19fc992f26a P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:30.569226 24115 raft_consensus.cc:738] T 8d3947300cfc4523acdab19fc992f26a P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Initialized, Role: FOLLOWER
I20250109 08:58:30.570008 24115 consensus_queue.cc:260] T 8d3947300cfc4523acdab19fc992f26a P 235ad6af04c34c88a5261daa3604fada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:30.572445 24115 ts_tablet_manager.cc:1428] T 8d3947300cfc4523acdab19fc992f26a P 235ad6af04c34c88a5261daa3604fada: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20250109 08:58:30.573485 24115 tablet_bootstrap.cc:492] T 2ab70546dda143b5b5db15c3875aa348 P 235ad6af04c34c88a5261daa3604fada: Bootstrap starting.
I20250109 08:58:30.591320 24115 tablet_bootstrap.cc:492] T 2ab70546dda143b5b5db15c3875aa348 P 235ad6af04c34c88a5261daa3604fada: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250109 08:58:30.592244 24115 tablet_bootstrap.cc:492] T 2ab70546dda143b5b5db15c3875aa348 P 235ad6af04c34c88a5261daa3604fada: Bootstrap complete.
I20250109 08:58:30.592936 24115 ts_tablet_manager.cc:1397] T 2ab70546dda143b5b5db15c3875aa348 P 235ad6af04c34c88a5261daa3604fada: Time spent bootstrapping tablet: real 0.020s user 0.018s sys 0.000s
I20250109 08:58:30.595372 24115 raft_consensus.cc:357] T 2ab70546dda143b5b5db15c3875aa348 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:30.596159 24115 raft_consensus.cc:738] T 2ab70546dda143b5b5db15c3875aa348 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Initialized, Role: FOLLOWER
I20250109 08:58:30.596899 24115 consensus_queue.cc:260] T 2ab70546dda143b5b5db15c3875aa348 P 235ad6af04c34c88a5261daa3604fada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:30.599423 24115 ts_tablet_manager.cc:1428] T 2ab70546dda143b5b5db15c3875aa348 P 235ad6af04c34c88a5261daa3604fada: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20250109 08:58:30.600255 24115 tablet_bootstrap.cc:492] T e6b9aaeda9a947d4a87fedd774301cc0 P 235ad6af04c34c88a5261daa3604fada: Bootstrap starting.
I20250109 08:58:30.617039 24115 tablet_bootstrap.cc:492] T e6b9aaeda9a947d4a87fedd774301cc0 P 235ad6af04c34c88a5261daa3604fada: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250109 08:58:30.617844 24115 tablet_bootstrap.cc:492] T e6b9aaeda9a947d4a87fedd774301cc0 P 235ad6af04c34c88a5261daa3604fada: Bootstrap complete.
I20250109 08:58:30.618427 24115 ts_tablet_manager.cc:1397] T e6b9aaeda9a947d4a87fedd774301cc0 P 235ad6af04c34c88a5261daa3604fada: Time spent bootstrapping tablet: real 0.018s user 0.013s sys 0.000s
I20250109 08:58:30.620352 24115 raft_consensus.cc:357] T e6b9aaeda9a947d4a87fedd774301cc0 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:30.621047 24115 raft_consensus.cc:738] T e6b9aaeda9a947d4a87fedd774301cc0 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Initialized, Role: FOLLOWER
I20250109 08:58:30.621762 24115 consensus_queue.cc:260] T e6b9aaeda9a947d4a87fedd774301cc0 P 235ad6af04c34c88a5261daa3604fada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:30.623760 24115 ts_tablet_manager.cc:1428] T e6b9aaeda9a947d4a87fedd774301cc0 P 235ad6af04c34c88a5261daa3604fada: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20250109 08:58:30.624495 24115 tablet_bootstrap.cc:492] T 41e7f0cc38c4414ab88c92c1cdeb3984 P 235ad6af04c34c88a5261daa3604fada: Bootstrap starting.
I20250109 08:58:30.630756 20363 rpc_server.cc:307] RPC server started. Bound to: 127.19.226.193:42691
I20250109 08:58:30.631032 24184 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.226.193:42691 every 8 connection(s)
I20250109 08:58:30.642271 24115 tablet_bootstrap.cc:492] T 41e7f0cc38c4414ab88c92c1cdeb3984 P 235ad6af04c34c88a5261daa3604fada: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250109 08:58:30.643163 24115 tablet_bootstrap.cc:492] T 41e7f0cc38c4414ab88c92c1cdeb3984 P 235ad6af04c34c88a5261daa3604fada: Bootstrap complete.
I20250109 08:58:30.643747 24115 ts_tablet_manager.cc:1397] T 41e7f0cc38c4414ab88c92c1cdeb3984 P 235ad6af04c34c88a5261daa3604fada: Time spent bootstrapping tablet: real 0.019s user 0.011s sys 0.000s
I20250109 08:58:30.646220 24115 raft_consensus.cc:357] T 41e7f0cc38c4414ab88c92c1cdeb3984 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:30.646909 24115 raft_consensus.cc:738] T 41e7f0cc38c4414ab88c92c1cdeb3984 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Initialized, Role: FOLLOWER
I20250109 08:58:30.647504 24115 consensus_queue.cc:260] T 41e7f0cc38c4414ab88c92c1cdeb3984 P 235ad6af04c34c88a5261daa3604fada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:30.649781 24115 ts_tablet_manager.cc:1428] T 41e7f0cc38c4414ab88c92c1cdeb3984 P 235ad6af04c34c88a5261daa3604fada: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
I20250109 08:58:30.650519 24115 tablet_bootstrap.cc:492] T 5c37c58abaa54656b31c72160b4dec9e P 235ad6af04c34c88a5261daa3604fada: Bootstrap starting.
I20250109 08:58:30.688349 24115 tablet_bootstrap.cc:492] T 5c37c58abaa54656b31c72160b4dec9e P 235ad6af04c34c88a5261daa3604fada: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250109 08:58:30.688845 23945 raft_consensus.cc:491] T 444507f7812441ceaeebc44bc8f06293 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 235ad6af04c34c88a5261daa3604fada)
I20250109 08:58:30.689255 24115 tablet_bootstrap.cc:492] T 5c37c58abaa54656b31c72160b4dec9e P 235ad6af04c34c88a5261daa3604fada: Bootstrap complete.
I20250109 08:58:30.689411 23945 raft_consensus.cc:513] T 444507f7812441ceaeebc44bc8f06293 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:30.689967 24115 ts_tablet_manager.cc:1397] T 5c37c58abaa54656b31c72160b4dec9e P 235ad6af04c34c88a5261daa3604fada: Time spent bootstrapping tablet: real 0.040s user 0.018s sys 0.012s
I20250109 08:58:30.691414 23945 leader_election.cc:290] T 444507f7812441ceaeebc44bc8f06293 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691), ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365)
I20250109 08:58:30.692551 23376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "444507f7812441ceaeebc44bc8f06293" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" is_pre_election: true
I20250109 08:58:30.695410 24115 raft_consensus.cc:357] T 5c37c58abaa54656b31c72160b4dec9e P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:30.696107 24115 raft_consensus.cc:738] T 5c37c58abaa54656b31c72160b4dec9e P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Initialized, Role: FOLLOWER
I20250109 08:58:30.696833 24115 consensus_queue.cc:260] T 5c37c58abaa54656b31c72160b4dec9e P 235ad6af04c34c88a5261daa3604fada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:30.698552 24115 ts_tablet_manager.cc:1428] T 5c37c58abaa54656b31c72160b4dec9e P 235ad6af04c34c88a5261daa3604fada: Time spent starting tablet: real 0.008s user 0.001s sys 0.004s
I20250109 08:58:30.699565 24115 tablet_bootstrap.cc:492] T 08214ee9c4c14bda92bdc6fa8231c313 P 235ad6af04c34c88a5261daa3604fada: Bootstrap starting.
I20250109 08:58:30.706951 24158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "444507f7812441ceaeebc44bc8f06293" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada" is_pre_election: true
I20250109 08:58:30.707688 24158 raft_consensus.cc:2463] T 444507f7812441ceaeebc44bc8f06293 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05a74e1eb2474b3d8b3a2da1ef127e01 in term 1.
I20250109 08:58:30.709494 23494 leader_election.cc:304] T 444507f7812441ceaeebc44bc8f06293 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01, 235ad6af04c34c88a5261daa3604fada; no voters: ed1efbce9ee3401e99a5260edeae4e7c
I20250109 08:58:30.710215 23945 raft_consensus.cc:2798] T 444507f7812441ceaeebc44bc8f06293 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20250109 08:58:30.710587 23945 raft_consensus.cc:491] T 444507f7812441ceaeebc44bc8f06293 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 235ad6af04c34c88a5261daa3604fada)
I20250109 08:58:30.710940 23945 raft_consensus.cc:3054] T 444507f7812441ceaeebc44bc8f06293 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:30.724961 23945 raft_consensus.cc:513] T 444507f7812441ceaeebc44bc8f06293 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:30.728811 24115 tablet_bootstrap.cc:492] T 08214ee9c4c14bda92bdc6fa8231c313 P 235ad6af04c34c88a5261daa3604fada: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250109 08:58:30.732663 23945 leader_election.cc:290] T 444507f7812441ceaeebc44bc8f06293 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 2 election: Requested vote from peers 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691), ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365)
I20250109 08:58:30.733513 24115 tablet_bootstrap.cc:492] T 08214ee9c4c14bda92bdc6fa8231c313 P 235ad6af04c34c88a5261daa3604fada: Bootstrap complete.
I20250109 08:58:30.734288 24115 ts_tablet_manager.cc:1397] T 08214ee9c4c14bda92bdc6fa8231c313 P 235ad6af04c34c88a5261daa3604fada: Time spent bootstrapping tablet: real 0.035s user 0.009s sys 0.019s
I20250109 08:58:30.735944 23376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "444507f7812441ceaeebc44bc8f06293" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c"
I20250109 08:58:30.736675 24115 raft_consensus.cc:357] T 08214ee9c4c14bda92bdc6fa8231c313 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:30.737385 24115 raft_consensus.cc:738] T 08214ee9c4c14bda92bdc6fa8231c313 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Initialized, Role: FOLLOWER
I20250109 08:58:30.737337 24158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "444507f7812441ceaeebc44bc8f06293" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada"
I20250109 08:58:30.738103 24158 raft_consensus.cc:3054] T 444507f7812441ceaeebc44bc8f06293 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:30.738322 24115 consensus_queue.cc:260] T 08214ee9c4c14bda92bdc6fa8231c313 P 235ad6af04c34c88a5261daa3604fada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:30.742048 24115 ts_tablet_manager.cc:1428] T 08214ee9c4c14bda92bdc6fa8231c313 P 235ad6af04c34c88a5261daa3604fada: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20250109 08:58:30.742867 24115 tablet_bootstrap.cc:492] T 94f2be307f704c75bceb213e52307cfb P 235ad6af04c34c88a5261daa3604fada: Bootstrap starting.
I20250109 08:58:30.745452 24158 raft_consensus.cc:2463] T 444507f7812441ceaeebc44bc8f06293 P 235ad6af04c34c88a5261daa3604fada [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05a74e1eb2474b3d8b3a2da1ef127e01 in term 2.
I20250109 08:58:30.746665 23494 leader_election.cc:304] T 444507f7812441ceaeebc44bc8f06293 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01, 235ad6af04c34c88a5261daa3604fada; no voters: ed1efbce9ee3401e99a5260edeae4e7c
I20250109 08:58:30.747380 23945 raft_consensus.cc:2798] T 444507f7812441ceaeebc44bc8f06293 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:58:30.748639 23945 raft_consensus.cc:695] T 444507f7812441ceaeebc44bc8f06293 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 LEADER]: Becoming Leader. State: Replica: 05a74e1eb2474b3d8b3a2da1ef127e01, State: Running, Role: LEADER
I20250109 08:58:30.750443 23945 consensus_queue.cc:237] T 444507f7812441ceaeebc44bc8f06293 P 05a74e1eb2474b3d8b3a2da1ef127e01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:30.761332 24115 tablet_bootstrap.cc:492] T 94f2be307f704c75bceb213e52307cfb P 235ad6af04c34c88a5261daa3604fada: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250109 08:58:30.762233 24115 tablet_bootstrap.cc:492] T 94f2be307f704c75bceb213e52307cfb P 235ad6af04c34c88a5261daa3604fada: Bootstrap complete.
I20250109 08:58:30.762859 24115 ts_tablet_manager.cc:1397] T 94f2be307f704c75bceb213e52307cfb P 235ad6af04c34c88a5261daa3604fada: Time spent bootstrapping tablet: real 0.020s user 0.012s sys 0.003s
I20250109 08:58:30.765102 24115 raft_consensus.cc:357] T 94f2be307f704c75bceb213e52307cfb P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:30.765756 24187 heartbeater.cc:346] Connected to a master server at 127.19.226.254:46425
I20250109 08:58:30.765792 24115 raft_consensus.cc:738] T 94f2be307f704c75bceb213e52307cfb P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Initialized, Role: FOLLOWER
I20250109 08:58:30.766152 24187 heartbeater.cc:463] Registering TS with master...
I20250109 08:58:30.767077 24187 heartbeater.cc:510] Master 127.19.226.254:46425 requested a full tablet report, sending...
I20250109 08:58:30.766754 24115 consensus_queue.cc:260] T 94f2be307f704c75bceb213e52307cfb P 235ad6af04c34c88a5261daa3604fada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:30.767686 23202 catalog_manager.cc:5526] T 444507f7812441ceaeebc44bc8f06293 P 05a74e1eb2474b3d8b3a2da1ef127e01 reported cstate change: term changed from 1 to 2, leader changed from 235ad6af04c34c88a5261daa3604fada (127.19.226.193) to 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196). New cstate: current_term: 2 leader_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:30.769186 24115 ts_tablet_manager.cc:1428] T 94f2be307f704c75bceb213e52307cfb P 235ad6af04c34c88a5261daa3604fada: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20250109 08:58:30.769989 24115 tablet_bootstrap.cc:492] T ea2b0762940e44dbbf69b9a163ace846 P 235ad6af04c34c88a5261daa3604fada: Bootstrap starting.
I20250109 08:58:30.785293 24115 tablet_bootstrap.cc:492] T ea2b0762940e44dbbf69b9a163ace846 P 235ad6af04c34c88a5261daa3604fada: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250109 08:58:30.786113 24115 tablet_bootstrap.cc:492] T ea2b0762940e44dbbf69b9a163ace846 P 235ad6af04c34c88a5261daa3604fada: Bootstrap complete.
I20250109 08:58:30.786762 24115 ts_tablet_manager.cc:1397] T ea2b0762940e44dbbf69b9a163ace846 P 235ad6af04c34c88a5261daa3604fada: Time spent bootstrapping tablet: real 0.017s user 0.010s sys 0.003s
I20250109 08:58:30.787156 23201 ts_manager.cc:194] Re-registered known tserver with Master: 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691)
I20250109 08:58:30.788576 24115 raft_consensus.cc:357] T ea2b0762940e44dbbf69b9a163ace846 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:30.789119 24115 raft_consensus.cc:738] T ea2b0762940e44dbbf69b9a163ace846 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Initialized, Role: FOLLOWER
I20250109 08:58:30.789661 24115 consensus_queue.cc:260] T ea2b0762940e44dbbf69b9a163ace846 P 235ad6af04c34c88a5261daa3604fada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:30.791653 24115 ts_tablet_manager.cc:1428] T ea2b0762940e44dbbf69b9a163ace846 P 235ad6af04c34c88a5261daa3604fada: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20250109 08:58:30.792428 24115 tablet_bootstrap.cc:492] T f1f9274aeda748f493f5752814fb262d P 235ad6af04c34c88a5261daa3604fada: Bootstrap starting.
I20250109 08:58:30.804019 24115 tablet_bootstrap.cc:492] T f1f9274aeda748f493f5752814fb262d P 235ad6af04c34c88a5261daa3604fada: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250109 08:58:30.804874 24115 tablet_bootstrap.cc:492] T f1f9274aeda748f493f5752814fb262d P 235ad6af04c34c88a5261daa3604fada: Bootstrap complete.
I20250109 08:58:30.805348 24115 ts_tablet_manager.cc:1397] T f1f9274aeda748f493f5752814fb262d P 235ad6af04c34c88a5261daa3604fada: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.007s
I20250109 08:58:30.807492 24115 raft_consensus.cc:357] T f1f9274aeda748f493f5752814fb262d P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:30.808148 24115 raft_consensus.cc:738] T f1f9274aeda748f493f5752814fb262d P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Initialized, Role: FOLLOWER
I20250109 08:58:30.808794 24115 consensus_queue.cc:260] T f1f9274aeda748f493f5752814fb262d P 235ad6af04c34c88a5261daa3604fada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:30.810709 24115 ts_tablet_manager.cc:1428] T f1f9274aeda748f493f5752814fb262d P 235ad6af04c34c88a5261daa3604fada: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20250109 08:58:30.811434 24115 tablet_bootstrap.cc:492] T d714271c7ea64c0b8b13fcf63591ec07 P 235ad6af04c34c88a5261daa3604fada: Bootstrap starting.
I20250109 08:58:30.822942 23201 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37814
I20250109 08:58:30.828044 24187 heartbeater.cc:502] Master 127.19.226.254:46425 was elected leader, sending a full tablet report...
I20250109 08:58:30.825345 24115 tablet_bootstrap.cc:492] T d714271c7ea64c0b8b13fcf63591ec07 P 235ad6af04c34c88a5261daa3604fada: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250109 08:58:30.829082 24115 tablet_bootstrap.cc:492] T d714271c7ea64c0b8b13fcf63591ec07 P 235ad6af04c34c88a5261daa3604fada: Bootstrap complete.
I20250109 08:58:30.829625 24115 ts_tablet_manager.cc:1397] T d714271c7ea64c0b8b13fcf63591ec07 P 235ad6af04c34c88a5261daa3604fada: Time spent bootstrapping tablet: real 0.018s user 0.010s sys 0.004s
I20250109 08:58:30.831516 24115 raft_consensus.cc:357] T d714271c7ea64c0b8b13fcf63591ec07 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:30.832120 23945 raft_consensus.cc:491] T ea2b0762940e44dbbf69b9a163ace846 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 235ad6af04c34c88a5261daa3604fada)
I20250109 08:58:30.832183 24115 raft_consensus.cc:738] T d714271c7ea64c0b8b13fcf63591ec07 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Initialized, Role: FOLLOWER
I20250109 08:58:30.832695 23945 raft_consensus.cc:513] T ea2b0762940e44dbbf69b9a163ace846 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:30.832963 24115 consensus_queue.cc:260] T d714271c7ea64c0b8b13fcf63591ec07 P 235ad6af04c34c88a5261daa3604fada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:30.834568 24115 ts_tablet_manager.cc:1428] T d714271c7ea64c0b8b13fcf63591ec07 P 235ad6af04c34c88a5261daa3604fada: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20250109 08:58:30.834918 23945 leader_election.cc:290] T ea2b0762940e44dbbf69b9a163ace846 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691), ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365)
I20250109 08:58:30.835321 24115 tablet_bootstrap.cc:492] T ed79047393d2425f8ecff31ea08b9cad P 235ad6af04c34c88a5261daa3604fada: Bootstrap starting.
I20250109 08:58:30.836179 24158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea2b0762940e44dbbf69b9a163ace846" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada" is_pre_election: true
I20250109 08:58:30.836342 23376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea2b0762940e44dbbf69b9a163ace846" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" is_pre_election: true
I20250109 08:58:30.836916 24158 raft_consensus.cc:2463] T ea2b0762940e44dbbf69b9a163ace846 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05a74e1eb2474b3d8b3a2da1ef127e01 in term 1.
I20250109 08:58:30.839036 23494 leader_election.cc:304] T ea2b0762940e44dbbf69b9a163ace846 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01, 235ad6af04c34c88a5261daa3604fada; no voters:
I20250109 08:58:30.839767 23945 raft_consensus.cc:2798] T ea2b0762940e44dbbf69b9a163ace846 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20250109 08:58:30.840174 23945 raft_consensus.cc:491] T ea2b0762940e44dbbf69b9a163ace846 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 235ad6af04c34c88a5261daa3604fada)
I20250109 08:58:30.841018 23945 raft_consensus.cc:3054] T ea2b0762940e44dbbf69b9a163ace846 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:30.848542 23945 raft_consensus.cc:513] T ea2b0762940e44dbbf69b9a163ace846 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:30.851552 23945 leader_election.cc:290] T ea2b0762940e44dbbf69b9a163ace846 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 2 election: Requested vote from peers 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691), ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365)
I20250109 08:58:30.852098 23376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea2b0762940e44dbbf69b9a163ace846" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c"
I20250109 08:58:30.855681 24158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea2b0762940e44dbbf69b9a163ace846" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada"
I20250109 08:58:30.856763 24115 tablet_bootstrap.cc:492] T ed79047393d2425f8ecff31ea08b9cad P 235ad6af04c34c88a5261daa3604fada: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250109 08:58:30.857669 24115 tablet_bootstrap.cc:492] T ed79047393d2425f8ecff31ea08b9cad P 235ad6af04c34c88a5261daa3604fada: Bootstrap complete.
I20250109 08:58:30.858279 24115 ts_tablet_manager.cc:1397] T ed79047393d2425f8ecff31ea08b9cad P 235ad6af04c34c88a5261daa3604fada: Time spent bootstrapping tablet: real 0.023s user 0.010s sys 0.004s
I20250109 08:58:30.856293 24158 raft_consensus.cc:3054] T ea2b0762940e44dbbf69b9a163ace846 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:30.860699 24115 raft_consensus.cc:357] T ed79047393d2425f8ecff31ea08b9cad P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:30.861399 24115 raft_consensus.cc:738] T ed79047393d2425f8ecff31ea08b9cad P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Initialized, Role: FOLLOWER
I20250109 08:58:30.862048 24115 consensus_queue.cc:260] T ed79047393d2425f8ecff31ea08b9cad P 235ad6af04c34c88a5261daa3604fada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:30.864213 24115 ts_tablet_manager.cc:1428] T ed79047393d2425f8ecff31ea08b9cad P 235ad6af04c34c88a5261daa3604fada: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20250109 08:58:30.864600 23945 raft_consensus.cc:491] T 94f2be307f704c75bceb213e52307cfb P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 235ad6af04c34c88a5261daa3604fada)
I20250109 08:58:30.865137 24115 tablet_bootstrap.cc:492] T 24ce131f7bb143bea1581d5b0c2a7109 P 235ad6af04c34c88a5261daa3604fada: Bootstrap starting.
I20250109 08:58:30.865196 23945 raft_consensus.cc:513] T 94f2be307f704c75bceb213e52307cfb P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:30.865887 24158 raft_consensus.cc:2463] T ea2b0762940e44dbbf69b9a163ace846 P 235ad6af04c34c88a5261daa3604fada [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05a74e1eb2474b3d8b3a2da1ef127e01 in term 2.
I20250109 08:58:30.866962 23494 leader_election.cc:304] T ea2b0762940e44dbbf69b9a163ace846 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01, 235ad6af04c34c88a5261daa3604fada; no voters: ed1efbce9ee3401e99a5260edeae4e7c
I20250109 08:58:30.868052 24084 raft_consensus.cc:2798] T ea2b0762940e44dbbf69b9a163ace846 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:58:30.868525 24084 raft_consensus.cc:695] T ea2b0762940e44dbbf69b9a163ace846 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 LEADER]: Becoming Leader. State: Replica: 05a74e1eb2474b3d8b3a2da1ef127e01, State: Running, Role: LEADER
I20250109 08:58:30.868894 23945 leader_election.cc:290] T 94f2be307f704c75bceb213e52307cfb P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551), 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691)
I20250109 08:58:30.869324 24084 consensus_queue.cc:237] T ea2b0762940e44dbbf69b9a163ace846 P 05a74e1eb2474b3d8b3a2da1ef127e01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:30.873939 23451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94f2be307f704c75bceb213e52307cfb" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc" is_pre_election: true
I20250109 08:58:30.874627 24159 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94f2be307f704c75bceb213e52307cfb" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada" is_pre_election: true
I20250109 08:58:30.875319 24159 raft_consensus.cc:2463] T 94f2be307f704c75bceb213e52307cfb P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05a74e1eb2474b3d8b3a2da1ef127e01 in term 1.
I20250109 08:58:30.876314 23494 leader_election.cc:304] T 94f2be307f704c75bceb213e52307cfb P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01, 235ad6af04c34c88a5261daa3604fada; no voters: cf55383bef6649eb9e78cb2d2512dedc
I20250109 08:58:30.877265 24085 raft_consensus.cc:2798] T 94f2be307f704c75bceb213e52307cfb P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20250109 08:58:30.877666 24085 raft_consensus.cc:491] T 94f2be307f704c75bceb213e52307cfb P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 235ad6af04c34c88a5261daa3604fada)
I20250109 08:58:30.878016 24085 raft_consensus.cc:3054] T 94f2be307f704c75bceb213e52307cfb P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:30.887048 24115 tablet_bootstrap.cc:492] T 24ce131f7bb143bea1581d5b0c2a7109 P 235ad6af04c34c88a5261daa3604fada: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250109 08:58:30.887823 24115 tablet_bootstrap.cc:492] T 24ce131f7bb143bea1581d5b0c2a7109 P 235ad6af04c34c88a5261daa3604fada: Bootstrap complete.
I20250109 08:58:30.888320 24115 ts_tablet_manager.cc:1397] T 24ce131f7bb143bea1581d5b0c2a7109 P 235ad6af04c34c88a5261daa3604fada: Time spent bootstrapping tablet: real 0.023s user 0.014s sys 0.004s
I20250109 08:58:30.890460 24115 raft_consensus.cc:357] T 24ce131f7bb143bea1581d5b0c2a7109 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:30.891359 24115 raft_consensus.cc:738] T 24ce131f7bb143bea1581d5b0c2a7109 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Initialized, Role: FOLLOWER
I20250109 08:58:30.892030 24115 consensus_queue.cc:260] T 24ce131f7bb143bea1581d5b0c2a7109 P 235ad6af04c34c88a5261daa3604fada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:30.890442 24085 raft_consensus.cc:513] T 94f2be307f704c75bceb213e52307cfb P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:30.894651 23451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94f2be307f704c75bceb213e52307cfb" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc"
I20250109 08:58:30.894497 23202 catalog_manager.cc:5526] T ea2b0762940e44dbbf69b9a163ace846 P 05a74e1eb2474b3d8b3a2da1ef127e01 reported cstate change: term changed from 1 to 2, leader changed from 235ad6af04c34c88a5261daa3604fada (127.19.226.193) to 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196). New cstate: current_term: 2 leader_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:30.895962 24159 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94f2be307f704c75bceb213e52307cfb" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada"
I20250109 08:58:30.896620 24115 ts_tablet_manager.cc:1428] T 24ce131f7bb143bea1581d5b0c2a7109 P 235ad6af04c34c88a5261daa3604fada: Time spent starting tablet: real 0.008s user 0.005s sys 0.000s
I20250109 08:58:30.896570 24159 raft_consensus.cc:3054] T 94f2be307f704c75bceb213e52307cfb P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:30.897455 24115 tablet_bootstrap.cc:492] T 060daed1d3334c6188413dd381b82bf9 P 235ad6af04c34c88a5261daa3604fada: Bootstrap starting.
I20250109 08:58:30.901615 24085 leader_election.cc:290] T 94f2be307f704c75bceb213e52307cfb P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 2 election: Requested vote from peers cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551), 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691)
I20250109 08:58:30.908741 24159 raft_consensus.cc:2463] T 94f2be307f704c75bceb213e52307cfb P 235ad6af04c34c88a5261daa3604fada [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05a74e1eb2474b3d8b3a2da1ef127e01 in term 2.
I20250109 08:58:30.909898 23494 leader_election.cc:304] T 94f2be307f704c75bceb213e52307cfb P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01, 235ad6af04c34c88a5261daa3604fada; no voters: cf55383bef6649eb9e78cb2d2512dedc
I20250109 08:58:30.910656 24085 raft_consensus.cc:2798] T 94f2be307f704c75bceb213e52307cfb P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:58:30.911010 24115 tablet_bootstrap.cc:492] T 060daed1d3334c6188413dd381b82bf9 P 235ad6af04c34c88a5261daa3604fada: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250109 08:58:30.911144 24085 raft_consensus.cc:695] T 94f2be307f704c75bceb213e52307cfb P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 LEADER]: Becoming Leader. State: Replica: 05a74e1eb2474b3d8b3a2da1ef127e01, State: Running, Role: LEADER
I20250109 08:58:30.911897 24115 tablet_bootstrap.cc:492] T 060daed1d3334c6188413dd381b82bf9 P 235ad6af04c34c88a5261daa3604fada: Bootstrap complete.
I20250109 08:58:30.912005 24085 consensus_queue.cc:237] T 94f2be307f704c75bceb213e52307cfb P 05a74e1eb2474b3d8b3a2da1ef127e01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:30.912596 24115 ts_tablet_manager.cc:1397] T 060daed1d3334c6188413dd381b82bf9 P 235ad6af04c34c88a5261daa3604fada: Time spent bootstrapping tablet: real 0.015s user 0.011s sys 0.004s
I20250109 08:58:30.915153 24115 raft_consensus.cc:357] T 060daed1d3334c6188413dd381b82bf9 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:30.915848 24115 raft_consensus.cc:738] T 060daed1d3334c6188413dd381b82bf9 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Initialized, Role: FOLLOWER
I20250109 08:58:30.916623 24115 consensus_queue.cc:260] T 060daed1d3334c6188413dd381b82bf9 P 235ad6af04c34c88a5261daa3604fada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:30.918823 24115 ts_tablet_manager.cc:1428] T 060daed1d3334c6188413dd381b82bf9 P 235ad6af04c34c88a5261daa3604fada: Time spent starting tablet: real 0.006s user 0.004s sys 0.001s
I20250109 08:58:30.919716 24115 tablet_bootstrap.cc:492] T cc9886333c0343798226d59758a1ea5e P 235ad6af04c34c88a5261daa3604fada: Bootstrap starting.
I20250109 08:58:30.926614 23202 catalog_manager.cc:5526] T 94f2be307f704c75bceb213e52307cfb P 05a74e1eb2474b3d8b3a2da1ef127e01 reported cstate change: term changed from 1 to 2, leader changed from 235ad6af04c34c88a5261daa3604fada (127.19.226.193) to 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196). New cstate: current_term: 2 leader_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:30.930687 24088 raft_consensus.cc:491] T db111e41c1be497e8e673a2c42b9ba66 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 235ad6af04c34c88a5261daa3604fada)
I20250109 08:58:30.931231 24088 raft_consensus.cc:513] T db111e41c1be497e8e673a2c42b9ba66 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:30.936815 24088 leader_election.cc:290] T db111e41c1be497e8e673a2c42b9ba66 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691), cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551)
I20250109 08:58:30.937659 24075 raft_consensus.cc:491] T 1340cf8ee2cb423f875276137da17e8b P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 235ad6af04c34c88a5261daa3604fada)
I20250109 08:58:30.938207 24075 raft_consensus.cc:513] T 1340cf8ee2cb423f875276137da17e8b P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:30.940832 23376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1340cf8ee2cb423f875276137da17e8b" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" is_pre_election: true
I20250109 08:58:30.941417 24075 leader_election.cc:290] T 1340cf8ee2cb423f875276137da17e8b P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365), 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691)
I20250109 08:58:30.942380 24115 tablet_bootstrap.cc:492] T cc9886333c0343798226d59758a1ea5e P 235ad6af04c34c88a5261daa3604fada: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250109 08:58:30.942776 24156 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db111e41c1be497e8e673a2c42b9ba66" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada" is_pre_election: true
I20250109 08:58:30.943171 24115 tablet_bootstrap.cc:492] T cc9886333c0343798226d59758a1ea5e P 235ad6af04c34c88a5261daa3604fada: Bootstrap complete.
I20250109 08:58:30.943120 23451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db111e41c1be497e8e673a2c42b9ba66" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc" is_pre_election: true
I20250109 08:58:30.943997 24115 ts_tablet_manager.cc:1397] T cc9886333c0343798226d59758a1ea5e P 235ad6af04c34c88a5261daa3604fada: Time spent bootstrapping tablet: real 0.024s user 0.013s sys 0.008s
I20250109 08:58:30.944028 24156 raft_consensus.cc:2463] T db111e41c1be497e8e673a2c42b9ba66 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed1efbce9ee3401e99a5260edeae4e7c in term 1.
I20250109 08:58:30.946482 23346 leader_election.cc:304] T db111e41c1be497e8e673a2c42b9ba66 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 235ad6af04c34c88a5261daa3604fada, ed1efbce9ee3401e99a5260edeae4e7c; no voters: cf55383bef6649eb9e78cb2d2512dedc
I20250109 08:58:30.946624 24115 raft_consensus.cc:357] T cc9886333c0343798226d59758a1ea5e P 235ad6af04c34c88a5261daa3604fada [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:30.947345 24115 raft_consensus.cc:738] T cc9886333c0343798226d59758a1ea5e P 235ad6af04c34c88a5261daa3604fada [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Initialized, Role: FOLLOWER
I20250109 08:58:30.948036 24115 consensus_queue.cc:260] T cc9886333c0343798226d59758a1ea5e P 235ad6af04c34c88a5261daa3604fada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:30.949676 24077 raft_consensus.cc:2798] T db111e41c1be497e8e673a2c42b9ba66 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20250109 08:58:30.950068 24077 raft_consensus.cc:491] T db111e41c1be497e8e673a2c42b9ba66 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 235ad6af04c34c88a5261daa3604fada)
I20250109 08:58:30.950341 24115 ts_tablet_manager.cc:1428] T cc9886333c0343798226d59758a1ea5e P 235ad6af04c34c88a5261daa3604fada: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20250109 08:58:30.950412 24077 raft_consensus.cc:3054] T db111e41c1be497e8e673a2c42b9ba66 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:30.951278 24115 tablet_bootstrap.cc:492] T 970a82e754db47b6b1a19551d4ba43d8 P 235ad6af04c34c88a5261daa3604fada: Bootstrap starting.
I20250109 08:58:30.956919 24095 raft_consensus.cc:491] T db111e41c1be497e8e673a2c42b9ba66 P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 235ad6af04c34c88a5261daa3604fada)
I20250109 08:58:30.957041 24084 raft_consensus.cc:491] T 060daed1d3334c6188413dd381b82bf9 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 235ad6af04c34c88a5261daa3604fada)
I20250109 08:58:30.957403 24095 raft_consensus.cc:513] T db111e41c1be497e8e673a2c42b9ba66 P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:30.958067 24084 raft_consensus.cc:513] T 060daed1d3334c6188413dd381b82bf9 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:30.959371 24095 leader_election.cc:290] T db111e41c1be497e8e673a2c42b9ba66 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691), ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365)
I20250109 08:58:30.960024 24084 leader_election.cc:290] T 060daed1d3334c6188413dd381b82bf9 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691), ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365)
I20250109 08:58:30.962456 24157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "060daed1d3334c6188413dd381b82bf9" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada" is_pre_election: true
I20250109 08:58:30.963215 24157 raft_consensus.cc:2463] T 060daed1d3334c6188413dd381b82bf9 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05a74e1eb2474b3d8b3a2da1ef127e01 in term 1.
I20250109 08:58:30.964061 23376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db111e41c1be497e8e673a2c42b9ba66" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" is_pre_election: true
I20250109 08:58:30.964977 23379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "060daed1d3334c6188413dd381b82bf9" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" is_pre_election: true
I20250109 08:58:30.966184 23494 leader_election.cc:304] T 060daed1d3334c6188413dd381b82bf9 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01, 235ad6af04c34c88a5261daa3604fada; no voters:
I20250109 08:58:30.970683 24085 raft_consensus.cc:2798] T 060daed1d3334c6188413dd381b82bf9 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20250109 08:58:30.971082 24085 raft_consensus.cc:491] T 060daed1d3334c6188413dd381b82bf9 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 235ad6af04c34c88a5261daa3604fada)
I20250109 08:58:30.971427 24085 raft_consensus.cc:3054] T 060daed1d3334c6188413dd381b82bf9 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:30.972774 24159 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1340cf8ee2cb423f875276137da17e8b" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada" is_pre_election: true
I20250109 08:58:30.969413 24077 raft_consensus.cc:513] T db111e41c1be497e8e673a2c42b9ba66 P ed1efbce9ee3401e99a5260edeae4e7c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
W20250109 08:58:30.974085 23420 leader_election.cc:343] T 1340cf8ee2cb423f875276137da17e8b P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:30.974514 23420 leader_election.cc:304] T 1340cf8ee2cb423f875276137da17e8b P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf55383bef6649eb9e78cb2d2512dedc; no voters: 235ad6af04c34c88a5261daa3604fada, ed1efbce9ee3401e99a5260edeae4e7c
I20250109 08:58:30.976550 23376 raft_consensus.cc:2388] T db111e41c1be497e8e673a2c42b9ba66 P ed1efbce9ee3401e99a5260edeae4e7c [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf55383bef6649eb9e78cb2d2512dedc in current term 2: Already voted for candidate ed1efbce9ee3401e99a5260edeae4e7c in this term.
I20250109 08:58:30.977450 24156 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db111e41c1be497e8e673a2c42b9ba66" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada" is_pre_election: true
I20250109 08:58:30.978092 24156 raft_consensus.cc:2463] T db111e41c1be497e8e673a2c42b9ba66 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf55383bef6649eb9e78cb2d2512dedc in term 1.
I20250109 08:58:30.979557 23420 leader_election.cc:304] T db111e41c1be497e8e673a2c42b9ba66 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 235ad6af04c34c88a5261daa3604fada, cf55383bef6649eb9e78cb2d2512dedc; no voters:
I20250109 08:58:30.979954 23445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db111e41c1be497e8e673a2c42b9ba66" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc"
I20250109 08:58:30.980683 23445 raft_consensus.cc:3054] T db111e41c1be497e8e673a2c42b9ba66 P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:30.977984 24077 leader_election.cc:290] T db111e41c1be497e8e673a2c42b9ba66 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 2 election: Requested vote from peers 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691), cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551)
I20250109 08:58:30.979936 24095 raft_consensus.cc:2743] T 1340cf8ee2cb423f875276137da17e8b P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250109 08:58:30.983405 24085 raft_consensus.cc:513] T 060daed1d3334c6188413dd381b82bf9 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:30.986068 24096 raft_consensus.cc:491] T 3208a99957554f6db341654856980c8a P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 235ad6af04c34c88a5261daa3604fada)
I20250109 08:58:30.986399 24156 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "060daed1d3334c6188413dd381b82bf9" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada"
I20250109 08:58:30.987022 24156 raft_consensus.cc:3054] T 060daed1d3334c6188413dd381b82bf9 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:30.988444 24096 raft_consensus.cc:513] T 3208a99957554f6db341654856980c8a P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:30.990506 24209 raft_consensus.cc:491] T f1f9274aeda748f493f5752814fb262d P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 235ad6af04c34c88a5261daa3604fada)
I20250109 08:58:30.991060 24209 raft_consensus.cc:513] T f1f9274aeda748f493f5752814fb262d P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:30.993618 24157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1f9274aeda748f493f5752814fb262d" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada" is_pre_election: true
I20250109 08:58:30.990459 24096 leader_election.cc:290] T 3208a99957554f6db341654856980c8a P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691), ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365)
I20250109 08:58:30.994661 24158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3208a99957554f6db341654856980c8a" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada" is_pre_election: true
I20250109 08:58:30.994242 24157 raft_consensus.cc:2463] T f1f9274aeda748f493f5752814fb262d P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed1efbce9ee3401e99a5260edeae4e7c in term 1.
I20250109 08:58:30.995949 23346 leader_election.cc:304] T f1f9274aeda748f493f5752814fb262d P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 235ad6af04c34c88a5261daa3604fada, ed1efbce9ee3401e99a5260edeae4e7c; no voters:
W20250109 08:58:30.996213 23420 leader_election.cc:343] T 3208a99957554f6db341654856980c8a P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:30.996716 24077 raft_consensus.cc:491] T 4df64aae69ca41dc91286cae1603000f P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 235ad6af04c34c88a5261daa3604fada)
I20250109 08:58:30.997195 24077 raft_consensus.cc:513] T 4df64aae69ca41dc91286cae1603000f P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:30.998184 24085 leader_election.cc:290] T 060daed1d3334c6188413dd381b82bf9 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 2 election: Requested vote from peers 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691), ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365)
I20250109 08:58:30.999073 24077 leader_election.cc:290] T 4df64aae69ca41dc91286cae1603000f P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829), 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691)
I20250109 08:58:30.999862 23376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3208a99957554f6db341654856980c8a" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" is_pre_election: true
I20250109 08:58:31.001197 23420 leader_election.cc:304] T 3208a99957554f6db341654856980c8a P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf55383bef6649eb9e78cb2d2512dedc; no voters: 235ad6af04c34c88a5261daa3604fada, ed1efbce9ee3401e99a5260edeae4e7c
I20250109 08:58:31.001989 24156 raft_consensus.cc:2463] T 060daed1d3334c6188413dd381b82bf9 P 235ad6af04c34c88a5261daa3604fada [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05a74e1eb2474b3d8b3a2da1ef127e01 in term 2.
I20250109 08:58:31.002192 24096 raft_consensus.cc:2743] T 3208a99957554f6db341654856980c8a P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250109 08:58:31.003036 23494 leader_election.cc:304] T 060daed1d3334c6188413dd381b82bf9 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01, 235ad6af04c34c88a5261daa3604fada; no voters:
I20250109 08:58:31.003115 23527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4df64aae69ca41dc91286cae1603000f" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" is_pre_election: true
I20250109 08:58:31.003970 24158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4df64aae69ca41dc91286cae1603000f" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada" is_pre_election: true
I20250109 08:58:31.004675 24158 raft_consensus.cc:2463] T 4df64aae69ca41dc91286cae1603000f P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed1efbce9ee3401e99a5260edeae4e7c in term 1.
I20250109 08:58:31.004729 24159 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db111e41c1be497e8e673a2c42b9ba66" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada"
I20250109 08:58:31.005340 24159 raft_consensus.cc:3054] T db111e41c1be497e8e673a2c42b9ba66 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:31.005842 24088 raft_consensus.cc:2798] T f1f9274aeda748f493f5752814fb262d P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20250109 08:58:31.005489 24085 raft_consensus.cc:2798] T 060daed1d3334c6188413dd381b82bf9 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:58:31.006219 24088 raft_consensus.cc:491] T f1f9274aeda748f493f5752814fb262d P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 235ad6af04c34c88a5261daa3604fada)
I20250109 08:58:31.006690 24085 raft_consensus.cc:695] T 060daed1d3334c6188413dd381b82bf9 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 LEADER]: Becoming Leader. State: Replica: 05a74e1eb2474b3d8b3a2da1ef127e01, State: Running, Role: LEADER
I20250109 08:58:31.007134 23376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "060daed1d3334c6188413dd381b82bf9" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c"
I20250109 08:58:31.007457 23451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1f9274aeda748f493f5752814fb262d" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc" is_pre_election: true
I20250109 08:58:31.007625 24085 consensus_queue.cc:237] T 060daed1d3334c6188413dd381b82bf9 P 05a74e1eb2474b3d8b3a2da1ef127e01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:31.006654 24088 raft_consensus.cc:3054] T f1f9274aeda748f493f5752814fb262d P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:31.011137 24209 leader_election.cc:290] T f1f9274aeda748f493f5752814fb262d P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691), cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551)
I20250109 08:58:31.008105 24077 raft_consensus.cc:491] T 1340cf8ee2cb423f875276137da17e8b P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 235ad6af04c34c88a5261daa3604fada)
I20250109 08:58:31.011675 24209 raft_consensus.cc:491] T 444507f7812441ceaeebc44bc8f06293 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 235ad6af04c34c88a5261daa3604fada)
I20250109 08:58:31.012039 24077 raft_consensus.cc:513] T 1340cf8ee2cb423f875276137da17e8b P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:31.012174 24209 raft_consensus.cc:513] T 444507f7812441ceaeebc44bc8f06293 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:31.018810 24209 leader_election.cc:290] T 444507f7812441ceaeebc44bc8f06293 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829), 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691)
I20250109 08:58:31.003219 23445 raft_consensus.cc:2463] T db111e41c1be497e8e673a2c42b9ba66 P cf55383bef6649eb9e78cb2d2512dedc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed1efbce9ee3401e99a5260edeae4e7c in term 2.
I20250109 08:58:31.019770 24095 raft_consensus.cc:2758] T db111e41c1be497e8e673a2c42b9ba66 P cf55383bef6649eb9e78cb2d2512dedc [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20250109 08:58:31.019456 24088 raft_consensus.cc:513] T f1f9274aeda748f493f5752814fb262d P ed1efbce9ee3401e99a5260edeae4e7c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:31.017107 24077 leader_election.cc:290] T 1340cf8ee2cb423f875276137da17e8b P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691), cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551)
I20250109 08:58:31.020905 24077 raft_consensus.cc:491] T 060daed1d3334c6188413dd381b82bf9 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 235ad6af04c34c88a5261daa3604fada)
I20250109 08:58:31.021543 23945 raft_consensus.cc:491] T 2ab70546dda143b5b5db15c3875aa348 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 235ad6af04c34c88a5261daa3604fada)
I20250109 08:58:31.021374 24077 raft_consensus.cc:513] T 060daed1d3334c6188413dd381b82bf9 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:31.022042 23945 raft_consensus.cc:513] T 2ab70546dda143b5b5db15c3875aa348 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:31.023905 23945 leader_election.cc:290] T 2ab70546dda143b5b5db15c3875aa348 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691), ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365)
I20250109 08:58:31.025602 24156 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ab70546dda143b5b5db15c3875aa348" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada" is_pre_election: true
I20250109 08:58:31.026435 24156 raft_consensus.cc:2463] T 2ab70546dda143b5b5db15c3875aa348 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05a74e1eb2474b3d8b3a2da1ef127e01 in term 1.
I20250109 08:58:31.026607 24157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "444507f7812441ceaeebc44bc8f06293" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada" is_pre_election: true
I20250109 08:58:31.027571 23494 leader_election.cc:304] T 2ab70546dda143b5b5db15c3875aa348 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01, 235ad6af04c34c88a5261daa3604fada; no voters:
I20250109 08:58:31.025681 24158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1340cf8ee2cb423f875276137da17e8b" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada" is_pre_election: true
I20250109 08:58:31.028650 23445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1340cf8ee2cb423f875276137da17e8b" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc" is_pre_election: true
I20250109 08:58:31.028823 23945 raft_consensus.cc:2798] T 2ab70546dda143b5b5db15c3875aa348 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20250109 08:58:31.027562 24157 raft_consensus.cc:2388] T 444507f7812441ceaeebc44bc8f06293 P 235ad6af04c34c88a5261daa3604fada [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ed1efbce9ee3401e99a5260edeae4e7c in current term 2: Already voted for candidate 05a74e1eb2474b3d8b3a2da1ef127e01 in this term.
I20250109 08:58:31.029347 23945 raft_consensus.cc:491] T 2ab70546dda143b5b5db15c3875aa348 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 235ad6af04c34c88a5261daa3604fada)
I20250109 08:58:31.029548 24158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1f9274aeda748f493f5752814fb262d" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada"
I20250109 08:58:31.029851 23945 raft_consensus.cc:3054] T 2ab70546dda143b5b5db15c3875aa348 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:31.030292 24158 raft_consensus.cc:3054] T f1f9274aeda748f493f5752814fb262d P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:31.021497 24088 leader_election.cc:290] T f1f9274aeda748f493f5752814fb262d P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 2 election: Requested vote from peers 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691), cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551)
I20250109 08:58:31.036026 24088 raft_consensus.cc:491] T fc1038255dd64cb5ba2425037d22f1f7 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 235ad6af04c34c88a5261daa3604fada)
I20250109 08:58:31.036566 24088 raft_consensus.cc:513] T fc1038255dd64cb5ba2425037d22f1f7 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:31.037422 23445 raft_consensus.cc:2463] T 1340cf8ee2cb423f875276137da17e8b P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed1efbce9ee3401e99a5260edeae4e7c in term 1.
I20250109 08:58:31.039012 23451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1f9274aeda748f493f5752814fb262d" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc"
I20250109 08:58:31.041158 23346 leader_election.cc:304] T db111e41c1be497e8e673a2c42b9ba66 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf55383bef6649eb9e78cb2d2512dedc, ed1efbce9ee3401e99a5260edeae4e7c; no voters:
I20250109 08:58:31.037674 24158 raft_consensus.cc:2463] T f1f9274aeda748f493f5752814fb262d P 235ad6af04c34c88a5261daa3604fada [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed1efbce9ee3401e99a5260edeae4e7c in term 2.
I20250109 08:58:31.041364 23201 catalog_manager.cc:5526] T 060daed1d3334c6188413dd381b82bf9 P 05a74e1eb2474b3d8b3a2da1ef127e01 reported cstate change: term changed from 1 to 2, leader changed from 235ad6af04c34c88a5261daa3604fada (127.19.226.193) to 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196). New cstate: current_term: 2 leader_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:31.038448 23945 raft_consensus.cc:513] T 2ab70546dda143b5b5db15c3875aa348 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:31.043167 23346 leader_election.cc:304] T 1340cf8ee2cb423f875276137da17e8b P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf55383bef6649eb9e78cb2d2512dedc, ed1efbce9ee3401e99a5260edeae4e7c; no voters:
I20250109 08:58:31.045133 24209 raft_consensus.cc:2798] T db111e41c1be497e8e673a2c42b9ba66 P ed1efbce9ee3401e99a5260edeae4e7c [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:58:31.045634 24209 raft_consensus.cc:695] T db111e41c1be497e8e673a2c42b9ba66 P ed1efbce9ee3401e99a5260edeae4e7c [term 2 LEADER]: Becoming Leader. State: Replica: ed1efbce9ee3401e99a5260edeae4e7c, State: Running, Role: LEADER
I20250109 08:58:31.046975 24211 raft_consensus.cc:2798] T 1340cf8ee2cb423f875276137da17e8b P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20250109 08:58:31.048595 24211 raft_consensus.cc:491] T 1340cf8ee2cb423f875276137da17e8b P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 235ad6af04c34c88a5261daa3604fada)
I20250109 08:58:31.048614 23945 leader_election.cc:290] T 2ab70546dda143b5b5db15c3875aa348 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 2 election: Requested vote from peers 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691), ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365)
W20250109 08:58:31.049103 23344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.226.196:41829, user_credentials={real_user=slave}} blocked reactor thread for 43709.3us
I20250109 08:58:31.048957 24211 raft_consensus.cc:3054] T 1340cf8ee2cb423f875276137da17e8b P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:31.050441 24209 consensus_queue.cc:237] T db111e41c1be497e8e673a2c42b9ba66 P ed1efbce9ee3401e99a5260edeae4e7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:31.046983 23346 leader_election.cc:304] T 4df64aae69ca41dc91286cae1603000f P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 235ad6af04c34c88a5261daa3604fada, ed1efbce9ee3401e99a5260edeae4e7c; no voters:
I20250109 08:58:31.053166 24095 raft_consensus.cc:491] T 77597d68423749e7b87f9f0db7a1ff3b P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 235ad6af04c34c88a5261daa3604fada)
I20250109 08:58:31.053721 24095 raft_consensus.cc:513] T 77597d68423749e7b87f9f0db7a1ff3b P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:31.054410 24159 raft_consensus.cc:2463] T db111e41c1be497e8e673a2c42b9ba66 P 235ad6af04c34c88a5261daa3604fada [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed1efbce9ee3401e99a5260edeae4e7c in term 2.
I20250109 08:58:31.055650 24096 raft_consensus.cc:491] T 94f2be307f704c75bceb213e52307cfb P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 235ad6af04c34c88a5261daa3604fada)
I20250109 08:58:31.056172 24096 raft_consensus.cc:513] T 94f2be307f704c75bceb213e52307cfb P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:31.056277 24211 raft_consensus.cc:513] T 1340cf8ee2cb423f875276137da17e8b P ed1efbce9ee3401e99a5260edeae4e7c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:31.057641 24115 tablet_bootstrap.cc:492] T 970a82e754db47b6b1a19551d4ba43d8 P 235ad6af04c34c88a5261daa3604fada: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250109 08:58:31.057693 23527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77597d68423749e7b87f9f0db7a1ff3b" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" is_pre_election: true
I20250109 08:58:31.058226 24211 leader_election.cc:290] T 1340cf8ee2cb423f875276137da17e8b P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 2 election: Requested vote from peers 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691), cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551)
I20250109 08:58:31.058853 24211 raft_consensus.cc:2798] T 4df64aae69ca41dc91286cae1603000f P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20250109 08:58:31.059214 24115 tablet_bootstrap.cc:492] T 970a82e754db47b6b1a19551d4ba43d8 P 235ad6af04c34c88a5261daa3604fada: Bootstrap complete.
I20250109 08:58:31.059960 23527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94f2be307f704c75bceb213e52307cfb" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" is_pre_election: true
I20250109 08:58:31.060488 24115 ts_tablet_manager.cc:1397] T 970a82e754db47b6b1a19551d4ba43d8 P 235ad6af04c34c88a5261daa3604fada: Time spent bootstrapping tablet: real 0.109s user 0.012s sys 0.016s
I20250109 08:58:31.059942 24157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77597d68423749e7b87f9f0db7a1ff3b" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada" is_pre_election: true
W20250109 08:58:31.061813 23420 leader_election.cc:343] T 77597d68423749e7b87f9f0db7a1ff3b P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:31.062211 23420 leader_election.cc:304] T 77597d68423749e7b87f9f0db7a1ff3b P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf55383bef6649eb9e78cb2d2512dedc; no voters: 05a74e1eb2474b3d8b3a2da1ef127e01, 235ad6af04c34c88a5261daa3604fada
I20250109 08:58:31.060732 24156 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94f2be307f704c75bceb213e52307cfb" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada" is_pre_election: true
W20250109 08:58:31.053359 23346 leader_election.cc:343] T 1340cf8ee2cb423f875276137da17e8b P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:31.063657 24156 raft_consensus.cc:2388] T 94f2be307f704c75bceb213e52307cfb P 235ad6af04c34c88a5261daa3604fada [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf55383bef6649eb9e78cb2d2512dedc in current term 2: Already voted for candidate 05a74e1eb2474b3d8b3a2da1ef127e01 in this term.
I20250109 08:58:31.055745 24095 leader_election.cc:290] T 77597d68423749e7b87f9f0db7a1ff3b P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829), 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691)
I20250109 08:58:31.063375 23527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "444507f7812441ceaeebc44bc8f06293" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" is_pre_election: true
I20250109 08:58:31.064157 24095 raft_consensus.cc:491] T f1f9274aeda748f493f5752814fb262d P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 235ad6af04c34c88a5261daa3604fada)
I20250109 08:58:31.064671 24075 raft_consensus.cc:2743] T 77597d68423749e7b87f9f0db7a1ff3b P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250109 08:58:31.064833 23346 leader_election.cc:304] T f1f9274aeda748f493f5752814fb262d P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 235ad6af04c34c88a5261daa3604fada, ed1efbce9ee3401e99a5260edeae4e7c; no voters: cf55383bef6649eb9e78cb2d2512dedc
I20250109 08:58:31.058097 24096 leader_election.cc:290] T 94f2be307f704c75bceb213e52307cfb P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829), 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691)
I20250109 08:58:31.065284 23527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc1038255dd64cb5ba2425037d22f1f7" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" is_pre_election: true
I20250109 08:58:31.041216 24088 leader_election.cc:290] T fc1038255dd64cb5ba2425037d22f1f7 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829), 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691)
I20250109 08:58:31.064674 24095 raft_consensus.cc:513] T f1f9274aeda748f493f5752814fb262d P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:31.068480 24095 leader_election.cc:290] T f1f9274aeda748f493f5752814fb262d P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691), ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365)
I20250109 08:58:31.063529 24115 raft_consensus.cc:357] T 970a82e754db47b6b1a19551d4ba43d8 P 235ad6af04c34c88a5261daa3604fada [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:31.069041 23527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "060daed1d3334c6188413dd381b82bf9" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" is_pre_election: true
I20250109 08:58:31.064996 24158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ab70546dda143b5b5db15c3875aa348" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada"
I20250109 08:58:31.070731 24088 raft_consensus.cc:2798] T f1f9274aeda748f493f5752814fb262d P ed1efbce9ee3401e99a5260edeae4e7c [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:58:31.070726 24158 raft_consensus.cc:3054] T 2ab70546dda143b5b5db15c3875aa348 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:31.072852 23454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1340cf8ee2cb423f875276137da17e8b" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc"
I20250109 08:58:31.073429 23454 raft_consensus.cc:3054] T 1340cf8ee2cb423f875276137da17e8b P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:31.072883 24155 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1340cf8ee2cb423f875276137da17e8b" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada"
I20250109 08:58:31.050141 24077 leader_election.cc:290] T 060daed1d3334c6188413dd381b82bf9 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829), 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691)
I20250109 08:58:31.074925 23376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ab70546dda143b5b5db15c3875aa348" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" is_pre_election: true
I20250109 08:58:31.075188 24157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "060daed1d3334c6188413dd381b82bf9" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada" is_pre_election: true
I20250109 08:58:31.076011 24157 raft_consensus.cc:2388] T 060daed1d3334c6188413dd381b82bf9 P 235ad6af04c34c88a5261daa3604fada [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ed1efbce9ee3401e99a5260edeae4e7c in current term 2: Already voted for candidate 05a74e1eb2474b3d8b3a2da1ef127e01 in this term.
I20250109 08:58:31.076604 23376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ab70546dda143b5b5db15c3875aa348" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c"
I20250109 08:58:31.071280 24156 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc1038255dd64cb5ba2425037d22f1f7" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada" is_pre_election: true
I20250109 08:58:31.081238 23454 raft_consensus.cc:2463] T 1340cf8ee2cb423f875276137da17e8b P cf55383bef6649eb9e78cb2d2512dedc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed1efbce9ee3401e99a5260edeae4e7c in term 2.
I20250109 08:58:31.082185 24085 raft_consensus.cc:491] T 4df64aae69ca41dc91286cae1603000f P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 235ad6af04c34c88a5261daa3604fada)
I20250109 08:58:31.082643 24115 raft_consensus.cc:738] T 970a82e754db47b6b1a19551d4ba43d8 P 235ad6af04c34c88a5261daa3604fada [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Initialized, Role: FOLLOWER
I20250109 08:58:31.083423 24212 raft_consensus.cc:491] T ea2b0762940e44dbbf69b9a163ace846 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 235ad6af04c34c88a5261daa3604fada)
I20250109 08:58:31.072201 24088 raft_consensus.cc:695] T f1f9274aeda748f493f5752814fb262d P ed1efbce9ee3401e99a5260edeae4e7c [term 2 LEADER]: Becoming Leader. State: Replica: ed1efbce9ee3401e99a5260edeae4e7c, State: Running, Role: LEADER
I20250109 08:58:31.059168 24211 raft_consensus.cc:491] T 4df64aae69ca41dc91286cae1603000f P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 235ad6af04c34c88a5261daa3604fada)
I20250109 08:58:31.083913 24212 raft_consensus.cc:513] T ea2b0762940e44dbbf69b9a163ace846 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:31.083175 24085 raft_consensus.cc:513] T 4df64aae69ca41dc91286cae1603000f P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:31.086694 24212 leader_election.cc:290] T ea2b0762940e44dbbf69b9a163ace846 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829), 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691)
I20250109 08:58:31.087160 24212 raft_consensus.cc:491] T 2ab70546dda143b5b5db15c3875aa348 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 235ad6af04c34c88a5261daa3604fada)
I20250109 08:58:31.087606 24212 raft_consensus.cc:513] T 2ab70546dda143b5b5db15c3875aa348 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:31.084079 24211 raft_consensus.cc:3054] T 4df64aae69ca41dc91286cae1603000f P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:31.087899 24085 leader_election.cc:290] T 4df64aae69ca41dc91286cae1603000f P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691), ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365)
I20250109 08:58:31.083715 24115 consensus_queue.cc:260] T 970a82e754db47b6b1a19551d4ba43d8 P 235ad6af04c34c88a5261daa3604fada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:31.090273 23420 leader_election.cc:304] T 94f2be307f704c75bceb213e52307cfb P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf55383bef6649eb9e78cb2d2512dedc; no voters: 05a74e1eb2474b3d8b3a2da1ef127e01, 235ad6af04c34c88a5261daa3604fada
I20250109 08:58:31.083181 23344 leader_election.cc:304] T 444507f7812441ceaeebc44bc8f06293 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ed1efbce9ee3401e99a5260edeae4e7c; no voters: 05a74e1eb2474b3d8b3a2da1ef127e01, 235ad6af04c34c88a5261daa3604fada
I20250109 08:58:31.090905 24156 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4df64aae69ca41dc91286cae1603000f" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada" is_pre_election: true
I20250109 08:58:31.091545 24156 raft_consensus.cc:2463] T 4df64aae69ca41dc91286cae1603000f P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05a74e1eb2474b3d8b3a2da1ef127e01 in term 1.
I20250109 08:58:31.089470 24212 leader_election.cc:290] T 2ab70546dda143b5b5db15c3875aa348 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829), 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691)
I20250109 08:58:31.092566 23494 leader_election.cc:304] T 4df64aae69ca41dc91286cae1603000f P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01, 235ad6af04c34c88a5261daa3604fada; no voters:
I20250109 08:58:31.092777 24215 raft_consensus.cc:3054] T 94f2be307f704c75bceb213e52307cfb P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:31.084463 24088 consensus_queue.cc:237] T f1f9274aeda748f493f5752814fb262d P ed1efbce9ee3401e99a5260edeae4e7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:31.089953 24157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1f9274aeda748f493f5752814fb262d" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada" is_pre_election: true
I20250109 08:58:31.093919 24157 raft_consensus.cc:2388] T f1f9274aeda748f493f5752814fb262d P 235ad6af04c34c88a5261daa3604fada [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cf55383bef6649eb9e78cb2d2512dedc in current term 2: Already voted for candidate ed1efbce9ee3401e99a5260edeae4e7c in this term.
I20250109 08:58:31.093959 24085 raft_consensus.cc:2798] T 4df64aae69ca41dc91286cae1603000f P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20250109 08:58:31.094667 24085 raft_consensus.cc:491] T 4df64aae69ca41dc91286cae1603000f P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 235ad6af04c34c88a5261daa3604fada)
I20250109 08:58:31.095041 24085 raft_consensus.cc:3054] T 4df64aae69ca41dc91286cae1603000f P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:31.091910 24115 ts_tablet_manager.cc:1428] T 970a82e754db47b6b1a19551d4ba43d8 P 235ad6af04c34c88a5261daa3604fada: Time spent starting tablet: real 0.031s user 0.005s sys 0.000s
I20250109 08:58:31.097996 24115 tablet_bootstrap.cc:492] T 2e3d34c0d09843d2be54e47f3ccc16d8 P 235ad6af04c34c88a5261daa3604fada: Bootstrap starting.
I20250109 08:58:31.097647 24211 raft_consensus.cc:513] T 4df64aae69ca41dc91286cae1603000f P ed1efbce9ee3401e99a5260edeae4e7c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:31.099702 24211 leader_election.cc:290] T 4df64aae69ca41dc91286cae1603000f P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 2 election: Requested vote from peers 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829), 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691)
I20250109 08:58:31.101116 24211 raft_consensus.cc:3054] T 444507f7812441ceaeebc44bc8f06293 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:31.109563 23346 leader_election.cc:304] T 1340cf8ee2cb423f875276137da17e8b P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf55383bef6649eb9e78cb2d2512dedc, ed1efbce9ee3401e99a5260edeae4e7c; no voters:
I20250109 08:58:31.110338 24158 raft_consensus.cc:2463] T 2ab70546dda143b5b5db15c3875aa348 P 235ad6af04c34c88a5261daa3604fada [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05a74e1eb2474b3d8b3a2da1ef127e01 in term 2.
I20250109 08:58:31.111197 24212 raft_consensus.cc:2798] T 1340cf8ee2cb423f875276137da17e8b P ed1efbce9ee3401e99a5260edeae4e7c [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:58:31.111584 23494 leader_election.cc:304] T 2ab70546dda143b5b5db15c3875aa348 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01, 235ad6af04c34c88a5261daa3604fada; no voters: ed1efbce9ee3401e99a5260edeae4e7c
W20250109 08:58:31.112282 23346 leader_election.cc:343] T 1340cf8ee2cb423f875276137da17e8b P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:31.111711 24212 raft_consensus.cc:695] T 1340cf8ee2cb423f875276137da17e8b P ed1efbce9ee3401e99a5260edeae4e7c [term 2 LEADER]: Becoming Leader. State: Replica: ed1efbce9ee3401e99a5260edeae4e7c, State: Running, Role: LEADER
I20250109 08:58:31.112473 24211 raft_consensus.cc:2743] T 444507f7812441ceaeebc44bc8f06293 P ed1efbce9ee3401e99a5260edeae4e7c [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250109 08:58:31.113368 24212 consensus_queue.cc:237] T 1340cf8ee2cb423f875276137da17e8b P ed1efbce9ee3401e99a5260edeae4e7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:31.112648 23945 raft_consensus.cc:2798] T 2ab70546dda143b5b5db15c3875aa348 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:58:31.117483 23945 raft_consensus.cc:695] T 2ab70546dda143b5b5db15c3875aa348 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 LEADER]: Becoming Leader. State: Replica: 05a74e1eb2474b3d8b3a2da1ef127e01, State: Running, Role: LEADER
W20250109 08:58:31.113920 23346 leader_election.cc:343] T fc1038255dd64cb5ba2425037d22f1f7 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:31.122942 23346 leader_election.cc:304] T fc1038255dd64cb5ba2425037d22f1f7 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ed1efbce9ee3401e99a5260edeae4e7c; no voters: 05a74e1eb2474b3d8b3a2da1ef127e01, 235ad6af04c34c88a5261daa3604fada
I20250109 08:58:31.118695 23945 consensus_queue.cc:237] T 2ab70546dda143b5b5db15c3875aa348 P 05a74e1eb2474b3d8b3a2da1ef127e01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:31.129313 24157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4df64aae69ca41dc91286cae1603000f" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada"
I20250109 08:58:31.129487 24159 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ab70546dda143b5b5db15c3875aa348" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada" is_pre_election: true
I20250109 08:58:31.130203 24157 raft_consensus.cc:3054] T 4df64aae69ca41dc91286cae1603000f P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:31.130489 24159 raft_consensus.cc:2388] T 2ab70546dda143b5b5db15c3875aa348 P 235ad6af04c34c88a5261daa3604fada [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ed1efbce9ee3401e99a5260edeae4e7c in current term 2: Already voted for candidate 05a74e1eb2474b3d8b3a2da1ef127e01 in this term.
I20250109 08:58:31.131774 24158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea2b0762940e44dbbf69b9a163ace846" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada" is_pre_election: true
I20250109 08:58:31.135612 23344 leader_election.cc:304] T 060daed1d3334c6188413dd381b82bf9 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ed1efbce9ee3401e99a5260edeae4e7c; no voters: 05a74e1eb2474b3d8b3a2da1ef127e01, 235ad6af04c34c88a5261daa3604fada
I20250109 08:58:31.144738 23379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4df64aae69ca41dc91286cae1603000f" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" is_pre_election: true
I20250109 08:58:31.144778 23527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea2b0762940e44dbbf69b9a163ace846" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" is_pre_election: true
I20250109 08:58:31.140003 24211 raft_consensus.cc:3054] T 060daed1d3334c6188413dd381b82bf9 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:31.149055 23379 raft_consensus.cc:2388] T 4df64aae69ca41dc91286cae1603000f P ed1efbce9ee3401e99a5260edeae4e7c [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 05a74e1eb2474b3d8b3a2da1ef127e01 in current term 2: Already voted for candidate ed1efbce9ee3401e99a5260edeae4e7c in this term.
I20250109 08:58:31.150168 23202 catalog_manager.cc:5526] T db111e41c1be497e8e673a2c42b9ba66 P ed1efbce9ee3401e99a5260edeae4e7c reported cstate change: term changed from 1 to 2, leader changed from 235ad6af04c34c88a5261daa3604fada (127.19.226.193) to ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194). New cstate: current_term: 2 leader_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:31.146168 24158 raft_consensus.cc:2388] T ea2b0762940e44dbbf69b9a163ace846 P 235ad6af04c34c88a5261daa3604fada [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ed1efbce9ee3401e99a5260edeae4e7c in current term 2: Already voted for candidate 05a74e1eb2474b3d8b3a2da1ef127e01 in this term.
I20250109 08:58:31.153566 23527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4df64aae69ca41dc91286cae1603000f" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01"
I20250109 08:58:31.155979 23521 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ab70546dda143b5b5db15c3875aa348" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" is_pre_election: true
I20250109 08:58:31.146528 24212 raft_consensus.cc:491] T 3208a99957554f6db341654856980c8a P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 235ad6af04c34c88a5261daa3604fada)
I20250109 08:58:31.163535 23346 leader_election.cc:304] T ea2b0762940e44dbbf69b9a163ace846 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ed1efbce9ee3401e99a5260edeae4e7c; no voters: 05a74e1eb2474b3d8b3a2da1ef127e01, 235ad6af04c34c88a5261daa3604fada
I20250109 08:58:31.157229 24084 raft_consensus.cc:491] T 77597d68423749e7b87f9f0db7a1ff3b P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 235ad6af04c34c88a5261daa3604fada)
I20250109 08:58:31.152864 23376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1f9274aeda748f493f5752814fb262d" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" is_pre_election: true
I20250109 08:58:31.161854 24219 raft_consensus.cc:2743] T fc1038255dd64cb5ba2425037d22f1f7 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250109 08:58:31.170751 23344 leader_election.cc:304] T 2ab70546dda143b5b5db15c3875aa348 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ed1efbce9ee3401e99a5260edeae4e7c; no voters: 05a74e1eb2474b3d8b3a2da1ef127e01, 235ad6af04c34c88a5261daa3604fada
I20250109 08:58:31.161676 24085 raft_consensus.cc:513] T 4df64aae69ca41dc91286cae1603000f P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:31.166970 24084 raft_consensus.cc:513] T 77597d68423749e7b87f9f0db7a1ff3b P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:31.173028 23420 leader_election.cc:304] T f1f9274aeda748f493f5752814fb262d P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf55383bef6649eb9e78cb2d2512dedc; no voters: 235ad6af04c34c88a5261daa3604fada, ed1efbce9ee3401e99a5260edeae4e7c
I20250109 08:58:31.163614 24212 raft_consensus.cc:513] T 3208a99957554f6db341654856980c8a P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:31.180099 24084 leader_election.cc:290] T 77597d68423749e7b87f9f0db7a1ff3b P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551), 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691)
I20250109 08:58:31.184386 23527 raft_consensus.cc:2388] T 4df64aae69ca41dc91286cae1603000f P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate ed1efbce9ee3401e99a5260edeae4e7c in current term 2: Already voted for candidate 05a74e1eb2474b3d8b3a2da1ef127e01 in this term.
I20250109 08:58:31.189006 24075 raft_consensus.cc:3054] T f1f9274aeda748f493f5752814fb262d P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:31.190668 24159 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3208a99957554f6db341654856980c8a" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada" is_pre_election: true
W20250109 08:58:31.193312 23346 leader_election.cc:343] T 3208a99957554f6db341654856980c8a P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:31.193452 23454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77597d68423749e7b87f9f0db7a1ff3b" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc" is_pre_election: true
I20250109 08:58:31.194242 23454 raft_consensus.cc:2463] T 77597d68423749e7b87f9f0db7a1ff3b P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05a74e1eb2474b3d8b3a2da1ef127e01 in term 1.
I20250109 08:58:31.196615 23494 leader_election.cc:304] T 77597d68423749e7b87f9f0db7a1ff3b P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01, cf55383bef6649eb9e78cb2d2512dedc; no voters:
I20250109 08:58:31.175292 24219 raft_consensus.cc:3054] T 2ab70546dda143b5b5db15c3875aa348 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:31.199712 24159 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77597d68423749e7b87f9f0db7a1ff3b" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada" is_pre_election: true
W20250109 08:58:31.200963 23494 leader_election.cc:343] T 77597d68423749e7b87f9f0db7a1ff3b P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:31.201498 24212 leader_election.cc:290] T 3208a99957554f6db341654856980c8a P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691), cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551)
I20250109 08:58:31.202024 24223 raft_consensus.cc:2798] T 77597d68423749e7b87f9f0db7a1ff3b P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20250109 08:58:31.202471 24223 raft_consensus.cc:491] T 77597d68423749e7b87f9f0db7a1ff3b P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 235ad6af04c34c88a5261daa3604fada)
I20250109 08:58:31.202879 24223 raft_consensus.cc:3054] T 77597d68423749e7b87f9f0db7a1ff3b P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:31.203894 23454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3208a99957554f6db341654856980c8a" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc" is_pre_election: true
I20250109 08:58:31.204555 23454 raft_consensus.cc:2463] T 3208a99957554f6db341654856980c8a P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed1efbce9ee3401e99a5260edeae4e7c in term 1.
I20250109 08:58:31.205483 24216 raft_consensus.cc:3054] T ea2b0762940e44dbbf69b9a163ace846 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:31.199937 23202 catalog_manager.cc:5526] T 2ab70546dda143b5b5db15c3875aa348 P 05a74e1eb2474b3d8b3a2da1ef127e01 reported cstate change: term changed from 1 to 2, leader changed from 235ad6af04c34c88a5261daa3604fada (127.19.226.193) to 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196). New cstate: current_term: 2 leader_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:31.208828 23346 leader_election.cc:304] T 3208a99957554f6db341654856980c8a P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf55383bef6649eb9e78cb2d2512dedc, ed1efbce9ee3401e99a5260edeae4e7c; no voters: 235ad6af04c34c88a5261daa3604fada
I20250109 08:58:31.197388 24157 raft_consensus.cc:2463] T 4df64aae69ca41dc91286cae1603000f P 235ad6af04c34c88a5261daa3604fada [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed1efbce9ee3401e99a5260edeae4e7c in term 2.
I20250109 08:58:31.211557 24213 raft_consensus.cc:2798] T 3208a99957554f6db341654856980c8a P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20250109 08:58:31.211840 23346 leader_election.cc:304] T 4df64aae69ca41dc91286cae1603000f P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 235ad6af04c34c88a5261daa3604fada, ed1efbce9ee3401e99a5260edeae4e7c; no voters:
I20250109 08:58:31.211935 24213 raft_consensus.cc:491] T 3208a99957554f6db341654856980c8a P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 235ad6af04c34c88a5261daa3604fada)
I20250109 08:58:31.212790 24213 raft_consensus.cc:3054] T 3208a99957554f6db341654856980c8a P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:31.213183 24215 raft_consensus.cc:2743] T 94f2be307f704c75bceb213e52307cfb P cf55383bef6649eb9e78cb2d2512dedc [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250109 08:58:31.215719 24209 raft_consensus.cc:2798] T 4df64aae69ca41dc91286cae1603000f P ed1efbce9ee3401e99a5260edeae4e7c [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:58:31.180702 24084 raft_consensus.cc:491] T fc1038255dd64cb5ba2425037d22f1f7 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 235ad6af04c34c88a5261daa3604fada)
I20250109 08:58:31.216223 24209 raft_consensus.cc:695] T 4df64aae69ca41dc91286cae1603000f P ed1efbce9ee3401e99a5260edeae4e7c [term 2 LEADER]: Becoming Leader. State: Replica: ed1efbce9ee3401e99a5260edeae4e7c, State: Running, Role: LEADER
I20250109 08:58:31.216681 24084 raft_consensus.cc:513] T fc1038255dd64cb5ba2425037d22f1f7 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:31.217185 24209 consensus_queue.cc:237] T 4df64aae69ca41dc91286cae1603000f P ed1efbce9ee3401e99a5260edeae4e7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:31.225396 23201 catalog_manager.cc:5526] T f1f9274aeda748f493f5752814fb262d P ed1efbce9ee3401e99a5260edeae4e7c reported cstate change: term changed from 1 to 2, leader changed from 235ad6af04c34c88a5261daa3604fada (127.19.226.193) to ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194). New cstate: current_term: 2 leader_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:31.227231 23201 catalog_manager.cc:5526] T 1340cf8ee2cb423f875276137da17e8b P ed1efbce9ee3401e99a5260edeae4e7c reported cstate change: term changed from 1 to 2, leader changed from 235ad6af04c34c88a5261daa3604fada (127.19.226.193) to ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194). New cstate: current_term: 2 leader_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:31.251905 24158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4df64aae69ca41dc91286cae1603000f" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada"
I20250109 08:58:31.252812 24158 raft_consensus.cc:2388] T 4df64aae69ca41dc91286cae1603000f P 235ad6af04c34c88a5261daa3604fada [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 05a74e1eb2474b3d8b3a2da1ef127e01 in current term 2: Already voted for candidate ed1efbce9ee3401e99a5260edeae4e7c in this term.
I20250109 08:58:31.256114 24085 leader_election.cc:290] T 4df64aae69ca41dc91286cae1603000f P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 2 election: Requested vote from peers 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691), ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365)
I20250109 08:58:31.262163 24213 raft_consensus.cc:513] T 3208a99957554f6db341654856980c8a P ed1efbce9ee3401e99a5260edeae4e7c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:31.265054 24211 raft_consensus.cc:2743] T 060daed1d3334c6188413dd381b82bf9 P ed1efbce9ee3401e99a5260edeae4e7c [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250109 08:58:31.266448 24223 raft_consensus.cc:513] T 77597d68423749e7b87f9f0db7a1ff3b P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:31.268021 24084 leader_election.cc:290] T fc1038255dd64cb5ba2425037d22f1f7 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691), ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365)
I20250109 08:58:31.268595 24223 leader_election.cc:290] T 77597d68423749e7b87f9f0db7a1ff3b P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 2 election: Requested vote from peers cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551), 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691)
I20250109 08:58:31.270625 24158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc1038255dd64cb5ba2425037d22f1f7" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada" is_pre_election: true
I20250109 08:58:31.271029 24219 raft_consensus.cc:2743] T 2ab70546dda143b5b5db15c3875aa348 P ed1efbce9ee3401e99a5260edeae4e7c [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20250109 08:58:31.272063 23494 leader_election.cc:343] T fc1038255dd64cb5ba2425037d22f1f7 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:31.272933 23376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4df64aae69ca41dc91286cae1603000f" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c"
I20250109 08:58:31.273203 23454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77597d68423749e7b87f9f0db7a1ff3b" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc"
I20250109 08:58:31.273808 23454 raft_consensus.cc:3054] T 77597d68423749e7b87f9f0db7a1ff3b P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:31.273994 24213 leader_election.cc:290] T 3208a99957554f6db341654856980c8a P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 2 election: Requested vote from peers 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691), cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551)
I20250109 08:58:31.274567 23376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc1038255dd64cb5ba2425037d22f1f7" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" is_pre_election: true
I20250109 08:58:31.273228 24155 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77597d68423749e7b87f9f0db7a1ff3b" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada"
I20250109 08:58:31.275238 23376 raft_consensus.cc:2463] T fc1038255dd64cb5ba2425037d22f1f7 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05a74e1eb2474b3d8b3a2da1ef127e01 in term 1.
W20250109 08:58:31.276201 23494 leader_election.cc:343] T 77597d68423749e7b87f9f0db7a1ff3b P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:31.276968 23494 leader_election.cc:304] T 4df64aae69ca41dc91286cae1603000f P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01; no voters: 235ad6af04c34c88a5261daa3604fada, ed1efbce9ee3401e99a5260edeae4e7c
I20250109 08:58:31.278018 23494 leader_election.cc:304] T fc1038255dd64cb5ba2425037d22f1f7 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01, ed1efbce9ee3401e99a5260edeae4e7c; no voters: 235ad6af04c34c88a5261daa3604fada
I20250109 08:58:31.279134 24158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3208a99957554f6db341654856980c8a" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada"
I20250109 08:58:31.281199 24227 raft_consensus.cc:2743] T 4df64aae69ca41dc91286cae1603000f P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20250109 08:58:31.281853 23454 raft_consensus.cc:2463] T 77597d68423749e7b87f9f0db7a1ff3b P cf55383bef6649eb9e78cb2d2512dedc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05a74e1eb2474b3d8b3a2da1ef127e01 in term 2.
I20250109 08:58:31.283162 23494 leader_election.cc:304] T 77597d68423749e7b87f9f0db7a1ff3b P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01, cf55383bef6649eb9e78cb2d2512dedc; no voters: 235ad6af04c34c88a5261daa3604fada
I20250109 08:58:31.283196 24075 raft_consensus.cc:2743] T f1f9274aeda748f493f5752814fb262d P cf55383bef6649eb9e78cb2d2512dedc [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250109 08:58:31.284062 24085 raft_consensus.cc:2798] T 77597d68423749e7b87f9f0db7a1ff3b P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:58:31.284547 24085 raft_consensus.cc:695] T 77597d68423749e7b87f9f0db7a1ff3b P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 LEADER]: Becoming Leader. State: Replica: 05a74e1eb2474b3d8b3a2da1ef127e01, State: Running, Role: LEADER
W20250109 08:58:31.284750 23346 leader_election.cc:343] T 3208a99957554f6db341654856980c8a P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:31.285390 24085 consensus_queue.cc:237] T 77597d68423749e7b87f9f0db7a1ff3b P 05a74e1eb2474b3d8b3a2da1ef127e01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:31.281904 24227 raft_consensus.cc:2798] T fc1038255dd64cb5ba2425037d22f1f7 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20250109 08:58:31.283156 24216 raft_consensus.cc:2743] T ea2b0762940e44dbbf69b9a163ace846 P ed1efbce9ee3401e99a5260edeae4e7c [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250109 08:58:31.315774 24227 raft_consensus.cc:491] T fc1038255dd64cb5ba2425037d22f1f7 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 235ad6af04c34c88a5261daa3604fada)
I20250109 08:58:31.320497 24227 raft_consensus.cc:3054] T fc1038255dd64cb5ba2425037d22f1f7 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:31.350006 23454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3208a99957554f6db341654856980c8a" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc"
I20250109 08:58:31.350747 23454 raft_consensus.cc:3054] T 3208a99957554f6db341654856980c8a P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:31.358417 23454 raft_consensus.cc:2463] T 3208a99957554f6db341654856980c8a P cf55383bef6649eb9e78cb2d2512dedc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed1efbce9ee3401e99a5260edeae4e7c in term 2.
I20250109 08:58:31.359903 23346 leader_election.cc:304] T 3208a99957554f6db341654856980c8a P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf55383bef6649eb9e78cb2d2512dedc, ed1efbce9ee3401e99a5260edeae4e7c; no voters: 235ad6af04c34c88a5261daa3604fada
I20250109 08:58:31.362792 24115 tablet_bootstrap.cc:492] T 2e3d34c0d09843d2be54e47f3ccc16d8 P 235ad6af04c34c88a5261daa3604fada: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20250109 08:58:31.362749 23344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.226.196:41829, user_credentials={real_user=slave}} blocked reactor thread for 53607.7us
W20250109 08:58:31.361243 23420 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.226.194:41365, user_credentials={real_user=slave}} blocked reactor thread for 110311us
I20250109 08:58:31.365726 24209 raft_consensus.cc:2798] T 3208a99957554f6db341654856980c8a P ed1efbce9ee3401e99a5260edeae4e7c [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:58:31.368083 24209 raft_consensus.cc:695] T 3208a99957554f6db341654856980c8a P ed1efbce9ee3401e99a5260edeae4e7c [term 2 LEADER]: Becoming Leader. State: Replica: ed1efbce9ee3401e99a5260edeae4e7c, State: Running, Role: LEADER
I20250109 08:58:31.368904 24209 consensus_queue.cc:237] T 3208a99957554f6db341654856980c8a P ed1efbce9ee3401e99a5260edeae4e7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:31.370121 24227 raft_consensus.cc:513] T fc1038255dd64cb5ba2425037d22f1f7 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:31.372074 24227 leader_election.cc:290] T fc1038255dd64cb5ba2425037d22f1f7 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 2 election: Requested vote from peers 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691), ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365)
I20250109 08:58:31.380842 23201 catalog_manager.cc:5526] T 4df64aae69ca41dc91286cae1603000f P ed1efbce9ee3401e99a5260edeae4e7c reported cstate change: term changed from 1 to 2, leader changed from 235ad6af04c34c88a5261daa3604fada (127.19.226.193) to ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194). New cstate: current_term: 2 leader_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } health_report { overall_health: HEALTHY } } }
I20250109 08:58:31.380542 24115 tablet_bootstrap.cc:492] T 2e3d34c0d09843d2be54e47f3ccc16d8 P 235ad6af04c34c88a5261daa3604fada: Bootstrap complete.
I20250109 08:58:31.383172 24115 ts_tablet_manager.cc:1397] T 2e3d34c0d09843d2be54e47f3ccc16d8 P 235ad6af04c34c88a5261daa3604fada: Time spent bootstrapping tablet: real 0.285s user 0.032s sys 0.013s
I20250109 08:58:31.388726 24159 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc1038255dd64cb5ba2425037d22f1f7" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada"
I20250109 08:58:31.395280 24158 raft_consensus.cc:1270] T 444507f7812441ceaeebc44bc8f06293 P 235ad6af04c34c88a5261daa3604fada [term 2 FOLLOWER]: Refusing update from remote peer 05a74e1eb2474b3d8b3a2da1ef127e01: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:58:31.396883 24157 raft_consensus.cc:1270] T ea2b0762940e44dbbf69b9a163ace846 P 235ad6af04c34c88a5261daa3604fada [term 2 FOLLOWER]: Refusing update from remote peer 05a74e1eb2474b3d8b3a2da1ef127e01: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20250109 08:58:31.400143 23494 leader_election.cc:343] T fc1038255dd64cb5ba2425037d22f1f7 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:31.401621 24237 consensus_queue.cc:1035] T 444507f7812441ceaeebc44bc8f06293 P 05a74e1eb2474b3d8b3a2da1ef127e01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20250109 08:58:31.409718 24115 raft_consensus.cc:357] T 2e3d34c0d09843d2be54e47f3ccc16d8 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:31.410305 24115 raft_consensus.cc:738] T 2e3d34c0d09843d2be54e47f3ccc16d8 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Initialized, Role: FOLLOWER
I20250109 08:58:31.410837 24115 consensus_queue.cc:260] T 2e3d34c0d09843d2be54e47f3ccc16d8 P 235ad6af04c34c88a5261daa3604fada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:31.413429 24115 ts_tablet_manager.cc:1428] T 2e3d34c0d09843d2be54e47f3ccc16d8 P 235ad6af04c34c88a5261daa3604fada: Time spent starting tablet: real 0.030s user 0.005s sys 0.000s
I20250109 08:58:31.417886 24115 tablet_bootstrap.cc:492] T a09d8dffc2e446f1acd3c7f6cc1705bb P 235ad6af04c34c88a5261daa3604fada: Bootstrap starting.
I20250109 08:58:31.421909 24224 consensus_queue.cc:1035] T ea2b0762940e44dbbf69b9a163ace846 P 05a74e1eb2474b3d8b3a2da1ef127e01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20250109 08:58:31.426663 24155 raft_consensus.cc:1270] T 94f2be307f704c75bceb213e52307cfb P 235ad6af04c34c88a5261daa3604fada [term 2 FOLLOWER]: Refusing update from remote peer 05a74e1eb2474b3d8b3a2da1ef127e01: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:58:31.437513 24085 consensus_queue.cc:1035] T 94f2be307f704c75bceb213e52307cfb P 05a74e1eb2474b3d8b3a2da1ef127e01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:58:31.485266 24115 tablet_bootstrap.cc:492] T a09d8dffc2e446f1acd3c7f6cc1705bb P 235ad6af04c34c88a5261daa3604fada: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250109 08:58:31.486387 24115 tablet_bootstrap.cc:492] T a09d8dffc2e446f1acd3c7f6cc1705bb P 235ad6af04c34c88a5261daa3604fada: Bootstrap complete.
I20250109 08:58:31.487178 24115 ts_tablet_manager.cc:1397] T a09d8dffc2e446f1acd3c7f6cc1705bb P 235ad6af04c34c88a5261daa3604fada: Time spent bootstrapping tablet: real 0.069s user 0.012s sys 0.005s
I20250109 08:58:31.493459 24115 raft_consensus.cc:357] T a09d8dffc2e446f1acd3c7f6cc1705bb P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:31.494254 24115 raft_consensus.cc:738] T a09d8dffc2e446f1acd3c7f6cc1705bb P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Initialized, Role: FOLLOWER
I20250109 08:58:31.494994 24115 consensus_queue.cc:260] T a09d8dffc2e446f1acd3c7f6cc1705bb P 235ad6af04c34c88a5261daa3604fada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:31.497589 24115 ts_tablet_manager.cc:1428] T a09d8dffc2e446f1acd3c7f6cc1705bb P 235ad6af04c34c88a5261daa3604fada: Time spent starting tablet: real 0.010s user 0.006s sys 0.001s
I20250109 08:58:31.498541 24115 tablet_bootstrap.cc:492] T 77597d68423749e7b87f9f0db7a1ff3b P 235ad6af04c34c88a5261daa3604fada: Bootstrap starting.
I20250109 08:58:31.715828 24115 tablet_bootstrap.cc:492] T 77597d68423749e7b87f9f0db7a1ff3b P 235ad6af04c34c88a5261daa3604fada: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250109 08:58:31.716811 24115 tablet_bootstrap.cc:492] T 77597d68423749e7b87f9f0db7a1ff3b P 235ad6af04c34c88a5261daa3604fada: Bootstrap complete.
I20250109 08:58:31.717437 24115 ts_tablet_manager.cc:1397] T 77597d68423749e7b87f9f0db7a1ff3b P 235ad6af04c34c88a5261daa3604fada: Time spent bootstrapping tablet: real 0.219s user 0.011s sys 0.004s
W20250109 08:58:31.719537 23420 consensus_peers.cc:487] T e5251927614642639bd46ecd5373d2a4 P cf55383bef6649eb9e78cb2d2512dedc -> Peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Couldn't send request to peer 235ad6af04c34c88a5261daa3604fada. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20250109 08:58:31.720767 24115 raft_consensus.cc:357] T 77597d68423749e7b87f9f0db7a1ff3b P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:31.721572 24115 raft_consensus.cc:738] T 77597d68423749e7b87f9f0db7a1ff3b P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Initialized, Role: FOLLOWER
I20250109 08:58:31.722223 24115 consensus_queue.cc:260] T 77597d68423749e7b87f9f0db7a1ff3b P 235ad6af04c34c88a5261daa3604fada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:31.724360 24115 ts_tablet_manager.cc:1428] T 77597d68423749e7b87f9f0db7a1ff3b P 235ad6af04c34c88a5261daa3604fada: Time spent starting tablet: real 0.007s user 0.004s sys 0.000s
I20250109 08:58:31.725152 24115 tablet_bootstrap.cc:492] T 6eef9bb3c645428d9f1cb843bae9a264 P 235ad6af04c34c88a5261daa3604fada: Bootstrap starting.
I20250109 08:58:31.727743 23375 raft_consensus.cc:1270] T ea2b0762940e44dbbf69b9a163ace846 P ed1efbce9ee3401e99a5260edeae4e7c [term 2 FOLLOWER]: Refusing update from remote peer 05a74e1eb2474b3d8b3a2da1ef127e01: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:58:31.729049 23371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc1038255dd64cb5ba2425037d22f1f7" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c"
I20250109 08:58:31.729709 23371 raft_consensus.cc:3054] T fc1038255dd64cb5ba2425037d22f1f7 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:31.741921 23371 raft_consensus.cc:2463] T fc1038255dd64cb5ba2425037d22f1f7 P ed1efbce9ee3401e99a5260edeae4e7c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05a74e1eb2474b3d8b3a2da1ef127e01 in term 2.
I20250109 08:58:31.745800 23375 raft_consensus.cc:1270] T 444507f7812441ceaeebc44bc8f06293 P ed1efbce9ee3401e99a5260edeae4e7c [term 2 FOLLOWER]: Refusing update from remote peer 05a74e1eb2474b3d8b3a2da1ef127e01: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:58:31.751962 24115 tablet_bootstrap.cc:492] T 6eef9bb3c645428d9f1cb843bae9a264 P 235ad6af04c34c88a5261daa3604fada: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250109 08:58:31.753268 24115 tablet_bootstrap.cc:492] T 6eef9bb3c645428d9f1cb843bae9a264 P 235ad6af04c34c88a5261daa3604fada: Bootstrap complete.
I20250109 08:58:31.753924 24115 ts_tablet_manager.cc:1397] T 6eef9bb3c645428d9f1cb843bae9a264 P 235ad6af04c34c88a5261daa3604fada: Time spent bootstrapping tablet: real 0.029s user 0.024s sys 0.000s
I20250109 08:58:31.755852 24115 raft_consensus.cc:357] T 6eef9bb3c645428d9f1cb843bae9a264 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:31.756345 24115 raft_consensus.cc:738] T 6eef9bb3c645428d9f1cb843bae9a264 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Initialized, Role: FOLLOWER
I20250109 08:58:31.757064 23494 leader_election.cc:304] T fc1038255dd64cb5ba2425037d22f1f7 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01, ed1efbce9ee3401e99a5260edeae4e7c; no voters: 235ad6af04c34c88a5261daa3604fada
I20250109 08:58:31.757292 24115 consensus_queue.cc:260] T 6eef9bb3c645428d9f1cb843bae9a264 P 235ad6af04c34c88a5261daa3604fada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:31.758358 24237 consensus_queue.cc:1035] T ea2b0762940e44dbbf69b9a163ace846 P 05a74e1eb2474b3d8b3a2da1ef127e01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:58:31.758980 23945 raft_consensus.cc:2798] T fc1038255dd64cb5ba2425037d22f1f7 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:58:31.759508 23945 raft_consensus.cc:695] T fc1038255dd64cb5ba2425037d22f1f7 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 LEADER]: Becoming Leader. State: Replica: 05a74e1eb2474b3d8b3a2da1ef127e01, State: Running, Role: LEADER
I20250109 08:58:31.759632 24237 consensus_queue.cc:1035] T 444507f7812441ceaeebc44bc8f06293 P 05a74e1eb2474b3d8b3a2da1ef127e01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:58:31.760259 24115 ts_tablet_manager.cc:1428] T 6eef9bb3c645428d9f1cb843bae9a264 P 235ad6af04c34c88a5261daa3604fada: Time spent starting tablet: real 0.006s user 0.002s sys 0.003s
I20250109 08:58:31.760958 24115 tablet_bootstrap.cc:492] T 1340cf8ee2cb423f875276137da17e8b P 235ad6af04c34c88a5261daa3604fada: Bootstrap starting.
I20250109 08:58:31.766207 24157 raft_consensus.cc:1270] T 2ab70546dda143b5b5db15c3875aa348 P 235ad6af04c34c88a5261daa3604fada [term 2 FOLLOWER]: Refusing update from remote peer 05a74e1eb2474b3d8b3a2da1ef127e01: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:58:31.767949 23945 consensus_queue.cc:237] T fc1038255dd64cb5ba2425037d22f1f7 P 05a74e1eb2474b3d8b3a2da1ef127e01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:31.775645 24158 raft_consensus.cc:1270] T db111e41c1be497e8e673a2c42b9ba66 P 235ad6af04c34c88a5261daa3604fada [term 2 FOLLOWER]: Refusing update from remote peer ed1efbce9ee3401e99a5260edeae4e7c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:58:31.777256 23945 consensus_queue.cc:1035] T 2ab70546dda143b5b5db15c3875aa348 P 05a74e1eb2474b3d8b3a2da1ef127e01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:58:31.778504 24153 raft_consensus.cc:1270] T f1f9274aeda748f493f5752814fb262d P 235ad6af04c34c88a5261daa3604fada [term 2 FOLLOWER]: Refusing update from remote peer ed1efbce9ee3401e99a5260edeae4e7c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:58:31.782703 24115 tablet_bootstrap.cc:492] T 1340cf8ee2cb423f875276137da17e8b P 235ad6af04c34c88a5261daa3604fada: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250109 08:58:31.797858 24213 consensus_queue.cc:1035] T db111e41c1be497e8e673a2c42b9ba66 P ed1efbce9ee3401e99a5260edeae4e7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20250109 08:58:31.800710 24213 consensus_queue.cc:1035] T f1f9274aeda748f493f5752814fb262d P ed1efbce9ee3401e99a5260edeae4e7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20250109 08:58:31.802253 23346 consensus_peers.cc:487] T 1340cf8ee2cb423f875276137da17e8b P ed1efbce9ee3401e99a5260edeae4e7c -> Peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Couldn't send request to peer 235ad6af04c34c88a5261daa3604fada. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:31.855974 23379 raft_consensus.cc:1270] T 2ab70546dda143b5b5db15c3875aa348 P ed1efbce9ee3401e99a5260edeae4e7c [term 2 FOLLOWER]: Refusing update from remote peer 05a74e1eb2474b3d8b3a2da1ef127e01: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:58:31.857908 24254 consensus_queue.cc:1035] T 2ab70546dda143b5b5db15c3875aa348 P 05a74e1eb2474b3d8b3a2da1ef127e01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20250109 08:58:31.878683 23454 raft_consensus.cc:1270] T 94f2be307f704c75bceb213e52307cfb P cf55383bef6649eb9e78cb2d2512dedc [term 2 FOLLOWER]: Refusing update from remote peer 05a74e1eb2474b3d8b3a2da1ef127e01: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:58:31.880059 23453 raft_consensus.cc:1270] T 77597d68423749e7b87f9f0db7a1ff3b P cf55383bef6649eb9e78cb2d2512dedc [term 2 FOLLOWER]: Refusing update from remote peer 05a74e1eb2474b3d8b3a2da1ef127e01: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:58:31.890213 24229 consensus_queue.cc:1035] T 94f2be307f704c75bceb213e52307cfb P 05a74e1eb2474b3d8b3a2da1ef127e01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20250109 08:58:31.894075 24085 consensus_queue.cc:1035] T 77597d68423749e7b87f9f0db7a1ff3b P 05a74e1eb2474b3d8b3a2da1ef127e01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:58:31.921979 24115 tablet_bootstrap.cc:492] T 1340cf8ee2cb423f875276137da17e8b P 235ad6af04c34c88a5261daa3604fada: Bootstrap complete.
I20250109 08:58:31.923317 24115 ts_tablet_manager.cc:1397] T 1340cf8ee2cb423f875276137da17e8b P 235ad6af04c34c88a5261daa3604fada: Time spent bootstrapping tablet: real 0.162s user 0.017s sys 0.008s
I20250109 08:58:31.926301 23374 raft_consensus.cc:1270] T 060daed1d3334c6188413dd381b82bf9 P ed1efbce9ee3401e99a5260edeae4e7c [term 2 FOLLOWER]: Refusing update from remote peer 05a74e1eb2474b3d8b3a2da1ef127e01: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:58:31.928150 24084 consensus_queue.cc:1035] T 060daed1d3334c6188413dd381b82bf9 P 05a74e1eb2474b3d8b3a2da1ef127e01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:58:31.930529 23520 raft_consensus.cc:1270] T 4df64aae69ca41dc91286cae1603000f P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 FOLLOWER]: Refusing update from remote peer ed1efbce9ee3401e99a5260edeae4e7c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:58:31.935600 24115 raft_consensus.cc:357] T 1340cf8ee2cb423f875276137da17e8b P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:31.938055 24115 raft_consensus.cc:738] T 1340cf8ee2cb423f875276137da17e8b P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Initialized, Role: FOLLOWER
I20250109 08:58:31.938696 24115 consensus_queue.cc:260] T 1340cf8ee2cb423f875276137da17e8b P 235ad6af04c34c88a5261daa3604fada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:31.940480 24154 raft_consensus.cc:1270] T 4df64aae69ca41dc91286cae1603000f P 235ad6af04c34c88a5261daa3604fada [term 2 FOLLOWER]: Refusing update from remote peer ed1efbce9ee3401e99a5260edeae4e7c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20250109 08:58:31.944820 23346 consensus_peers.cc:487] T 1365130ef97c4581b623c22c39753831 P ed1efbce9ee3401e99a5260edeae4e7c -> Peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Couldn't send request to peer 235ad6af04c34c88a5261daa3604fada. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20250109 08:58:31.946074 24216 consensus_queue.cc:1035] T 4df64aae69ca41dc91286cae1603000f P ed1efbce9ee3401e99a5260edeae4e7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:58:31.949904 24212 consensus_queue.cc:1035] T 4df64aae69ca41dc91286cae1603000f P ed1efbce9ee3401e99a5260edeae4e7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:58:31.955281 24115 ts_tablet_manager.cc:1428] T 1340cf8ee2cb423f875276137da17e8b P 235ad6af04c34c88a5261daa3604fada: Time spent starting tablet: real 0.031s user 0.001s sys 0.004s
I20250109 08:58:31.962277 24115 tablet_bootstrap.cc:492] T d87d3b4755344b82b0aec7ffd1ce785c P 235ad6af04c34c88a5261daa3604fada: Bootstrap starting.
I20250109 08:58:31.971683 24156 raft_consensus.cc:3054] T 77597d68423749e7b87f9f0db7a1ff3b P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:31.987237 23454 raft_consensus.cc:1270] T 1340cf8ee2cb423f875276137da17e8b P cf55383bef6649eb9e78cb2d2512dedc [term 2 FOLLOWER]: Refusing update from remote peer ed1efbce9ee3401e99a5260edeae4e7c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:58:31.985872 23452 raft_consensus.cc:1270] T db111e41c1be497e8e673a2c42b9ba66 P cf55383bef6649eb9e78cb2d2512dedc [term 2 FOLLOWER]: Refusing update from remote peer ed1efbce9ee3401e99a5260edeae4e7c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:58:31.994596 23200 catalog_manager.cc:5526] T 77597d68423749e7b87f9f0db7a1ff3b P 05a74e1eb2474b3d8b3a2da1ef127e01 reported cstate change: term changed from 1 to 2, leader changed from 235ad6af04c34c88a5261daa3604fada (127.19.226.193) to 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196). New cstate: current_term: 2 leader_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:31.985888 23453 raft_consensus.cc:1270] T f1f9274aeda748f493f5752814fb262d P cf55383bef6649eb9e78cb2d2512dedc [term 2 FOLLOWER]: Refusing update from remote peer ed1efbce9ee3401e99a5260edeae4e7c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:58:32.029218 24154 raft_consensus.cc:1270] T 060daed1d3334c6188413dd381b82bf9 P 235ad6af04c34c88a5261daa3604fada [term 2 FOLLOWER]: Refusing update from remote peer 05a74e1eb2474b3d8b3a2da1ef127e01: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:58:32.046873 24156 raft_consensus.cc:1270] T 77597d68423749e7b87f9f0db7a1ff3b P 235ad6af04c34c88a5261daa3604fada [term 2 FOLLOWER]: Refusing update from remote peer 05a74e1eb2474b3d8b3a2da1ef127e01: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20250109 08:58:32.081455 23346 consensus_peers.cc:487] T 3208a99957554f6db341654856980c8a P ed1efbce9ee3401e99a5260edeae4e7c -> Peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Couldn't send request to peer 235ad6af04c34c88a5261daa3604fada. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:32.096752 24216 consensus_queue.cc:1035] T 1340cf8ee2cb423f875276137da17e8b P ed1efbce9ee3401e99a5260edeae4e7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:58:32.099306 24216 consensus_queue.cc:1035] T db111e41c1be497e8e673a2c42b9ba66 P ed1efbce9ee3401e99a5260edeae4e7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:58:32.102995 24212 consensus_queue.cc:1035] T f1f9274aeda748f493f5752814fb262d P ed1efbce9ee3401e99a5260edeae4e7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:58:32.104462 24084 consensus_queue.cc:1035] T 060daed1d3334c6188413dd381b82bf9 P 05a74e1eb2474b3d8b3a2da1ef127e01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:58:32.104683 24085 consensus_queue.cc:1035] T 77597d68423749e7b87f9f0db7a1ff3b P 05a74e1eb2474b3d8b3a2da1ef127e01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:58:32.107995 23201 catalog_manager.cc:5526] T 3208a99957554f6db341654856980c8a P ed1efbce9ee3401e99a5260edeae4e7c reported cstate change: term changed from 1 to 2, leader changed from 235ad6af04c34c88a5261daa3604fada (127.19.226.193) to ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194). New cstate: current_term: 2 leader_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:32.124397 23449 raft_consensus.cc:1270] T 3208a99957554f6db341654856980c8a P cf55383bef6649eb9e78cb2d2512dedc [term 2 FOLLOWER]: Refusing update from remote peer ed1efbce9ee3401e99a5260edeae4e7c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:58:32.128182 24213 consensus_queue.cc:1035] T 3208a99957554f6db341654856980c8a P ed1efbce9ee3401e99a5260edeae4e7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20250109 08:58:32.153442 23420 consensus_peers.cc:487] T de98c70e25494d2a8e581443be836bd3 P cf55383bef6649eb9e78cb2d2512dedc -> Peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Couldn't send request to peer 235ad6af04c34c88a5261daa3604fada. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20250109 08:58:32.153744 24115 tablet_bootstrap.cc:492] T d87d3b4755344b82b0aec7ffd1ce785c P 235ad6af04c34c88a5261daa3604fada: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250109 08:58:32.157965 24115 tablet_bootstrap.cc:492] T d87d3b4755344b82b0aec7ffd1ce785c P 235ad6af04c34c88a5261daa3604fada: Bootstrap complete.
I20250109 08:58:32.158619 24115 ts_tablet_manager.cc:1397] T d87d3b4755344b82b0aec7ffd1ce785c P 235ad6af04c34c88a5261daa3604fada: Time spent bootstrapping tablet: real 0.197s user 0.015s sys 0.000s
I20250109 08:58:32.161136 24115 raft_consensus.cc:357] T d87d3b4755344b82b0aec7ffd1ce785c P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:32.161975 24115 raft_consensus.cc:738] T d87d3b4755344b82b0aec7ffd1ce785c P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Initialized, Role: FOLLOWER
I20250109 08:58:32.162571 24115 consensus_queue.cc:260] T d87d3b4755344b82b0aec7ffd1ce785c P 235ad6af04c34c88a5261daa3604fada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:32.164868 24115 ts_tablet_manager.cc:1428] T d87d3b4755344b82b0aec7ffd1ce785c P 235ad6af04c34c88a5261daa3604fada: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20250109 08:58:32.165629 24115 tablet_bootstrap.cc:492] T e5251927614642639bd46ecd5373d2a4 P 235ad6af04c34c88a5261daa3604fada: Bootstrap starting.
I20250109 08:58:32.179736 24115 tablet_bootstrap.cc:492] T e5251927614642639bd46ecd5373d2a4 P 235ad6af04c34c88a5261daa3604fada: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250109 08:58:32.180902 24115 tablet_bootstrap.cc:492] T e5251927614642639bd46ecd5373d2a4 P 235ad6af04c34c88a5261daa3604fada: Bootstrap complete.
I20250109 08:58:32.181542 24115 ts_tablet_manager.cc:1397] T e5251927614642639bd46ecd5373d2a4 P 235ad6af04c34c88a5261daa3604fada: Time spent bootstrapping tablet: real 0.016s user 0.008s sys 0.007s
I20250109 08:58:32.183748 24115 raft_consensus.cc:357] T e5251927614642639bd46ecd5373d2a4 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:32.184484 24115 raft_consensus.cc:738] T e5251927614642639bd46ecd5373d2a4 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Initialized, Role: FOLLOWER
I20250109 08:58:32.185207 24115 consensus_queue.cc:260] T e5251927614642639bd46ecd5373d2a4 P 235ad6af04c34c88a5261daa3604fada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:32.187322 24115 ts_tablet_manager.cc:1428] T e5251927614642639bd46ecd5373d2a4 P 235ad6af04c34c88a5261daa3604fada: Time spent starting tablet: real 0.005s user 0.003s sys 0.001s
I20250109 08:58:32.188066 24115 tablet_bootstrap.cc:492] T bd6fcd838b0249cabd5f1f95e5538ab6 P 235ad6af04c34c88a5261daa3604fada: Bootstrap starting.
I20250109 08:58:32.237067 24115 tablet_bootstrap.cc:492] T bd6fcd838b0249cabd5f1f95e5538ab6 P 235ad6af04c34c88a5261daa3604fada: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250109 08:58:32.238094 24115 tablet_bootstrap.cc:492] T bd6fcd838b0249cabd5f1f95e5538ab6 P 235ad6af04c34c88a5261daa3604fada: Bootstrap complete.
I20250109 08:58:32.238801 24115 ts_tablet_manager.cc:1397] T bd6fcd838b0249cabd5f1f95e5538ab6 P 235ad6af04c34c88a5261daa3604fada: Time spent bootstrapping tablet: real 0.051s user 0.025s sys 0.016s
I20250109 08:58:32.241117 24115 raft_consensus.cc:357] T bd6fcd838b0249cabd5f1f95e5538ab6 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:32.241673 24115 raft_consensus.cc:738] T bd6fcd838b0249cabd5f1f95e5538ab6 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Initialized, Role: FOLLOWER
I20250109 08:58:32.242197 24115 consensus_queue.cc:260] T bd6fcd838b0249cabd5f1f95e5538ab6 P 235ad6af04c34c88a5261daa3604fada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:32.244284 24115 ts_tablet_manager.cc:1428] T bd6fcd838b0249cabd5f1f95e5538ab6 P 235ad6af04c34c88a5261daa3604fada: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20250109 08:58:32.245026 24115 tablet_bootstrap.cc:492] T 5f0d500a91b54851bda52e5ccf741fd6 P 235ad6af04c34c88a5261daa3604fada: Bootstrap starting.
I20250109 08:58:32.245872 23202 catalog_manager.cc:5526] T fc1038255dd64cb5ba2425037d22f1f7 P 05a74e1eb2474b3d8b3a2da1ef127e01 reported cstate change: term changed from 1 to 2, leader changed from 235ad6af04c34c88a5261daa3604fada (127.19.226.193) to 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196). New cstate: current_term: 2 leader_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:32.260872 24115 tablet_bootstrap.cc:492] T 5f0d500a91b54851bda52e5ccf741fd6 P 235ad6af04c34c88a5261daa3604fada: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250109 08:58:32.261777 24115 tablet_bootstrap.cc:492] T 5f0d500a91b54851bda52e5ccf741fd6 P 235ad6af04c34c88a5261daa3604fada: Bootstrap complete.
I20250109 08:58:32.262392 24115 ts_tablet_manager.cc:1397] T 5f0d500a91b54851bda52e5ccf741fd6 P 235ad6af04c34c88a5261daa3604fada: Time spent bootstrapping tablet: real 0.018s user 0.015s sys 0.000s
I20250109 08:58:32.264652 24115 raft_consensus.cc:357] T 5f0d500a91b54851bda52e5ccf741fd6 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:32.265290 24115 raft_consensus.cc:738] T 5f0d500a91b54851bda52e5ccf741fd6 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Initialized, Role: FOLLOWER
I20250109 08:58:32.265944 24115 consensus_queue.cc:260] T 5f0d500a91b54851bda52e5ccf741fd6 P 235ad6af04c34c88a5261daa3604fada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:32.267005 24156 raft_consensus.cc:3054] T 1340cf8ee2cb423f875276137da17e8b P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:32.268445 24115 ts_tablet_manager.cc:1428] T 5f0d500a91b54851bda52e5ccf741fd6 P 235ad6af04c34c88a5261daa3604fada: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20250109 08:58:32.269256 24115 tablet_bootstrap.cc:492] T fc1038255dd64cb5ba2425037d22f1f7 P 235ad6af04c34c88a5261daa3604fada: Bootstrap starting.
W20250109 08:58:32.276790 23494 consensus_peers.cc:487] T 29828d9befb845d5bd49a80e018502e3 P 05a74e1eb2474b3d8b3a2da1ef127e01 -> Peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Couldn't send request to peer 235ad6af04c34c88a5261daa3604fada. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20250109 08:58:32.283635 24115 tablet_bootstrap.cc:492] T fc1038255dd64cb5ba2425037d22f1f7 P 235ad6af04c34c88a5261daa3604fada: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250109 08:58:32.284613 24115 tablet_bootstrap.cc:492] T fc1038255dd64cb5ba2425037d22f1f7 P 235ad6af04c34c88a5261daa3604fada: Bootstrap complete.
I20250109 08:58:32.286062 24115 ts_tablet_manager.cc:1397] T fc1038255dd64cb5ba2425037d22f1f7 P 235ad6af04c34c88a5261daa3604fada: Time spent bootstrapping tablet: real 0.017s user 0.010s sys 0.006s
I20250109 08:58:32.288494 24115 raft_consensus.cc:357] T fc1038255dd64cb5ba2425037d22f1f7 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:32.289161 24115 raft_consensus.cc:738] T fc1038255dd64cb5ba2425037d22f1f7 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Initialized, Role: FOLLOWER
I20250109 08:58:32.289839 24115 consensus_queue.cc:260] T fc1038255dd64cb5ba2425037d22f1f7 P 235ad6af04c34c88a5261daa3604fada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:32.292237 24115 ts_tablet_manager.cc:1428] T fc1038255dd64cb5ba2425037d22f1f7 P 235ad6af04c34c88a5261daa3604fada: Time spent starting tablet: real 0.006s user 0.002s sys 0.002s
I20250109 08:58:32.293046 24115 tablet_bootstrap.cc:492] T 1365130ef97c4581b623c22c39753831 P 235ad6af04c34c88a5261daa3604fada: Bootstrap starting.
I20250109 08:58:32.294260 23374 raft_consensus.cc:1270] T fc1038255dd64cb5ba2425037d22f1f7 P ed1efbce9ee3401e99a5260edeae4e7c [term 2 FOLLOWER]: Refusing update from remote peer 05a74e1eb2474b3d8b3a2da1ef127e01: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:58:32.297158 24227 consensus_queue.cc:1035] T fc1038255dd64cb5ba2425037d22f1f7 P 05a74e1eb2474b3d8b3a2da1ef127e01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20250109 08:58:32.302143 23346 consensus_peers.cc:487] T ee703ea575234c659da04f21ef3cdd70 P ed1efbce9ee3401e99a5260edeae4e7c -> Peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Couldn't send request to peer 235ad6af04c34c88a5261daa3604fada. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20250109 08:58:32.317773 24157 raft_consensus.cc:3054] T fc1038255dd64cb5ba2425037d22f1f7 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:32.321388 24115 tablet_bootstrap.cc:492] T 1365130ef97c4581b623c22c39753831 P 235ad6af04c34c88a5261daa3604fada: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250109 08:58:32.322293 24115 tablet_bootstrap.cc:492] T 1365130ef97c4581b623c22c39753831 P 235ad6af04c34c88a5261daa3604fada: Bootstrap complete.
I20250109 08:58:32.322876 24115 ts_tablet_manager.cc:1397] T 1365130ef97c4581b623c22c39753831 P 235ad6af04c34c88a5261daa3604fada: Time spent bootstrapping tablet: real 0.030s user 0.008s sys 0.004s
I20250109 08:58:32.324574 24157 raft_consensus.cc:1270] T fc1038255dd64cb5ba2425037d22f1f7 P 235ad6af04c34c88a5261daa3604fada [term 2 FOLLOWER]: Refusing update from remote peer 05a74e1eb2474b3d8b3a2da1ef127e01: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:58:32.325244 24115 raft_consensus.cc:357] T 1365130ef97c4581b623c22c39753831 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:32.325953 24115 raft_consensus.cc:738] T 1365130ef97c4581b623c22c39753831 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Initialized, Role: FOLLOWER
I20250109 08:58:32.325909 24227 consensus_queue.cc:1035] T fc1038255dd64cb5ba2425037d22f1f7 P 05a74e1eb2474b3d8b3a2da1ef127e01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20250109 08:58:32.326808 24115 consensus_queue.cc:260] T 1365130ef97c4581b623c22c39753831 P 235ad6af04c34c88a5261daa3604fada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:32.329108 24115 ts_tablet_manager.cc:1428] T 1365130ef97c4581b623c22c39753831 P 235ad6af04c34c88a5261daa3604fada: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
I20250109 08:58:32.329823 24115 tablet_bootstrap.cc:492] T 9b245cb3666c421c884cf20161a22831 P 235ad6af04c34c88a5261daa3604fada: Bootstrap starting.
W20250109 08:58:32.330308 23494 consensus_peers.cc:487] T 82ea7dc042dd4d05b260d508f4422c3c P 05a74e1eb2474b3d8b3a2da1ef127e01 -> Peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Couldn't send request to peer 235ad6af04c34c88a5261daa3604fada. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20250109 08:58:32.345031 24115 tablet_bootstrap.cc:492] T 9b245cb3666c421c884cf20161a22831 P 235ad6af04c34c88a5261daa3604fada: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250109 08:58:32.345865 24115 tablet_bootstrap.cc:492] T 9b245cb3666c421c884cf20161a22831 P 235ad6af04c34c88a5261daa3604fada: Bootstrap complete.
I20250109 08:58:32.346468 24115 ts_tablet_manager.cc:1397] T 9b245cb3666c421c884cf20161a22831 P 235ad6af04c34c88a5261daa3604fada: Time spent bootstrapping tablet: real 0.017s user 0.012s sys 0.000s
I20250109 08:58:32.348769 24115 raft_consensus.cc:357] T 9b245cb3666c421c884cf20161a22831 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:32.349452 24115 raft_consensus.cc:738] T 9b245cb3666c421c884cf20161a22831 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Initialized, Role: FOLLOWER
I20250109 08:58:32.350162 24115 consensus_queue.cc:260] T 9b245cb3666c421c884cf20161a22831 P 235ad6af04c34c88a5261daa3604fada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:32.352499 24115 ts_tablet_manager.cc:1428] T 9b245cb3666c421c884cf20161a22831 P 235ad6af04c34c88a5261daa3604fada: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
I20250109 08:58:32.353227 24115 tablet_bootstrap.cc:492] T de98c70e25494d2a8e581443be836bd3 P 235ad6af04c34c88a5261daa3604fada: Bootstrap starting.
I20250109 08:58:32.385982 24115 tablet_bootstrap.cc:492] T de98c70e25494d2a8e581443be836bd3 P 235ad6af04c34c88a5261daa3604fada: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250109 08:58:32.386703 24115 tablet_bootstrap.cc:492] T de98c70e25494d2a8e581443be836bd3 P 235ad6af04c34c88a5261daa3604fada: Bootstrap complete.
I20250109 08:58:32.387297 24115 ts_tablet_manager.cc:1397] T de98c70e25494d2a8e581443be836bd3 P 235ad6af04c34c88a5261daa3604fada: Time spent bootstrapping tablet: real 0.034s user 0.013s sys 0.017s
I20250109 08:58:32.389664 24115 raft_consensus.cc:357] T de98c70e25494d2a8e581443be836bd3 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:32.390367 24115 raft_consensus.cc:738] T de98c70e25494d2a8e581443be836bd3 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Initialized, Role: FOLLOWER
I20250109 08:58:32.391052 24115 consensus_queue.cc:260] T de98c70e25494d2a8e581443be836bd3 P 235ad6af04c34c88a5261daa3604fada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:32.392836 24115 ts_tablet_manager.cc:1428] T de98c70e25494d2a8e581443be836bd3 P 235ad6af04c34c88a5261daa3604fada: Time spent starting tablet: real 0.005s user 0.008s sys 0.000s
I20250109 08:58:32.393538 24115 tablet_bootstrap.cc:492] T ee703ea575234c659da04f21ef3cdd70 P 235ad6af04c34c88a5261daa3604fada: Bootstrap starting.
I20250109 08:58:32.413998 24115 tablet_bootstrap.cc:492] T ee703ea575234c659da04f21ef3cdd70 P 235ad6af04c34c88a5261daa3604fada: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250109 08:58:32.414795 24115 tablet_bootstrap.cc:492] T ee703ea575234c659da04f21ef3cdd70 P 235ad6af04c34c88a5261daa3604fada: Bootstrap complete.
I20250109 08:58:32.415378 24115 ts_tablet_manager.cc:1397] T ee703ea575234c659da04f21ef3cdd70 P 235ad6af04c34c88a5261daa3604fada: Time spent bootstrapping tablet: real 0.022s user 0.011s sys 0.003s
I20250109 08:58:32.417169 24115 raft_consensus.cc:357] T ee703ea575234c659da04f21ef3cdd70 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:32.418068 24115 raft_consensus.cc:738] T ee703ea575234c659da04f21ef3cdd70 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Initialized, Role: FOLLOWER
I20250109 08:58:32.418633 24115 consensus_queue.cc:260] T ee703ea575234c659da04f21ef3cdd70 P 235ad6af04c34c88a5261daa3604fada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:32.420890 24115 ts_tablet_manager.cc:1428] T ee703ea575234c659da04f21ef3cdd70 P 235ad6af04c34c88a5261daa3604fada: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20250109 08:58:32.421897 24115 tablet_bootstrap.cc:492] T 82ea7dc042dd4d05b260d508f4422c3c P 235ad6af04c34c88a5261daa3604fada: Bootstrap starting.
I20250109 08:58:32.438057 24115 tablet_bootstrap.cc:492] T 82ea7dc042dd4d05b260d508f4422c3c P 235ad6af04c34c88a5261daa3604fada: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250109 08:58:32.438727 24115 tablet_bootstrap.cc:492] T 82ea7dc042dd4d05b260d508f4422c3c P 235ad6af04c34c88a5261daa3604fada: Bootstrap complete.
I20250109 08:58:32.439203 24115 ts_tablet_manager.cc:1397] T 82ea7dc042dd4d05b260d508f4422c3c P 235ad6af04c34c88a5261daa3604fada: Time spent bootstrapping tablet: real 0.017s user 0.011s sys 0.000s
W20250109 08:58:32.441501 23346 consensus_peers.cc:487] T 1c5b5beea4a840eb9677097cacee3408 P ed1efbce9ee3401e99a5260edeae4e7c -> Peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Couldn't send request to peer 235ad6af04c34c88a5261daa3604fada. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20250109 08:58:32.441303 24115 raft_consensus.cc:357] T 82ea7dc042dd4d05b260d508f4422c3c P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:32.441933 24115 raft_consensus.cc:738] T 82ea7dc042dd4d05b260d508f4422c3c P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Initialized, Role: FOLLOWER
I20250109 08:58:32.442440 24115 consensus_queue.cc:260] T 82ea7dc042dd4d05b260d508f4422c3c P 235ad6af04c34c88a5261daa3604fada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:32.444278 24115 ts_tablet_manager.cc:1428] T 82ea7dc042dd4d05b260d508f4422c3c P 235ad6af04c34c88a5261daa3604fada: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20250109 08:58:32.445164 24115 tablet_bootstrap.cc:492] T 86d4074509744900abe94402467a7d85 P 235ad6af04c34c88a5261daa3604fada: Bootstrap starting.
I20250109 08:58:32.459820 24115 tablet_bootstrap.cc:492] T 86d4074509744900abe94402467a7d85 P 235ad6af04c34c88a5261daa3604fada: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250109 08:58:32.460722 24115 tablet_bootstrap.cc:492] T 86d4074509744900abe94402467a7d85 P 235ad6af04c34c88a5261daa3604fada: Bootstrap complete.
I20250109 08:58:32.461261 24115 ts_tablet_manager.cc:1397] T 86d4074509744900abe94402467a7d85 P 235ad6af04c34c88a5261daa3604fada: Time spent bootstrapping tablet: real 0.016s user 0.015s sys 0.000s
I20250109 08:58:32.462858 24115 raft_consensus.cc:357] T 86d4074509744900abe94402467a7d85 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:32.463332 24115 raft_consensus.cc:738] T 86d4074509744900abe94402467a7d85 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Initialized, Role: FOLLOWER
I20250109 08:58:32.463824 24115 consensus_queue.cc:260] T 86d4074509744900abe94402467a7d85 P 235ad6af04c34c88a5261daa3604fada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:32.465991 24115 ts_tablet_manager.cc:1428] T 86d4074509744900abe94402467a7d85 P 235ad6af04c34c88a5261daa3604fada: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20250109 08:58:32.467342 24115 tablet_bootstrap.cc:492] T bfb140bb3a8b44bd8b4e1fcf0946162b P 235ad6af04c34c88a5261daa3604fada: Bootstrap starting.
I20250109 08:58:32.479883 24115 tablet_bootstrap.cc:492] T bfb140bb3a8b44bd8b4e1fcf0946162b P 235ad6af04c34c88a5261daa3604fada: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250109 08:58:32.480688 24115 tablet_bootstrap.cc:492] T bfb140bb3a8b44bd8b4e1fcf0946162b P 235ad6af04c34c88a5261daa3604fada: Bootstrap complete.
I20250109 08:58:32.481354 24115 ts_tablet_manager.cc:1397] T bfb140bb3a8b44bd8b4e1fcf0946162b P 235ad6af04c34c88a5261daa3604fada: Time spent bootstrapping tablet: real 0.014s user 0.009s sys 0.003s
I20250109 08:58:32.483057 24115 raft_consensus.cc:357] T bfb140bb3a8b44bd8b4e1fcf0946162b P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:32.483863 24115 raft_consensus.cc:738] T bfb140bb3a8b44bd8b4e1fcf0946162b P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Initialized, Role: FOLLOWER
I20250109 08:58:32.484346 24115 consensus_queue.cc:260] T bfb140bb3a8b44bd8b4e1fcf0946162b P 235ad6af04c34c88a5261daa3604fada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:32.485901 24115 ts_tablet_manager.cc:1428] T bfb140bb3a8b44bd8b4e1fcf0946162b P 235ad6af04c34c88a5261daa3604fada: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20250109 08:58:32.486909 24115 tablet_bootstrap.cc:492] T 3208a99957554f6db341654856980c8a P 235ad6af04c34c88a5261daa3604fada: Bootstrap starting.
I20250109 08:58:32.499117 24115 tablet_bootstrap.cc:492] T 3208a99957554f6db341654856980c8a P 235ad6af04c34c88a5261daa3604fada: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250109 08:58:32.499783 24115 tablet_bootstrap.cc:492] T 3208a99957554f6db341654856980c8a P 235ad6af04c34c88a5261daa3604fada: Bootstrap complete.
I20250109 08:58:32.500355 24115 ts_tablet_manager.cc:1397] T 3208a99957554f6db341654856980c8a P 235ad6af04c34c88a5261daa3604fada: Time spent bootstrapping tablet: real 0.014s user 0.009s sys 0.000s
I20250109 08:58:32.502593 24115 raft_consensus.cc:357] T 3208a99957554f6db341654856980c8a P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:32.503196 24115 raft_consensus.cc:738] T 3208a99957554f6db341654856980c8a P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Initialized, Role: FOLLOWER
I20250109 08:58:32.503767 24115 consensus_queue.cc:260] T 3208a99957554f6db341654856980c8a P 235ad6af04c34c88a5261daa3604fada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:32.505664 24115 ts_tablet_manager.cc:1428] T 3208a99957554f6db341654856980c8a P 235ad6af04c34c88a5261daa3604fada: Time spent starting tablet: real 0.005s user 0.000s sys 0.004s
I20250109 08:58:32.506392 24115 tablet_bootstrap.cc:492] T 29828d9befb845d5bd49a80e018502e3 P 235ad6af04c34c88a5261daa3604fada: Bootstrap starting.
I20250109 08:58:32.518406 24115 tablet_bootstrap.cc:492] T 29828d9befb845d5bd49a80e018502e3 P 235ad6af04c34c88a5261daa3604fada: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250109 08:58:32.519258 24115 tablet_bootstrap.cc:492] T 29828d9befb845d5bd49a80e018502e3 P 235ad6af04c34c88a5261daa3604fada: Bootstrap complete.
I20250109 08:58:32.519850 24115 ts_tablet_manager.cc:1397] T 29828d9befb845d5bd49a80e018502e3 P 235ad6af04c34c88a5261daa3604fada: Time spent bootstrapping tablet: real 0.014s user 0.009s sys 0.000s
I20250109 08:58:32.521975 24115 raft_consensus.cc:357] T 29828d9befb845d5bd49a80e018502e3 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:32.522519 24115 raft_consensus.cc:738] T 29828d9befb845d5bd49a80e018502e3 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Initialized, Role: FOLLOWER
I20250109 08:58:32.523100 24115 consensus_queue.cc:260] T 29828d9befb845d5bd49a80e018502e3 P 235ad6af04c34c88a5261daa3604fada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:32.524864 24115 ts_tablet_manager.cc:1428] T 29828d9befb845d5bd49a80e018502e3 P 235ad6af04c34c88a5261daa3604fada: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20250109 08:58:32.525679 24115 tablet_bootstrap.cc:492] T a7d2ab61d238471c9f287ac1d5db4ed0 P 235ad6af04c34c88a5261daa3604fada: Bootstrap starting.
I20250109 08:58:32.542992 24115 tablet_bootstrap.cc:492] T a7d2ab61d238471c9f287ac1d5db4ed0 P 235ad6af04c34c88a5261daa3604fada: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250109 08:58:32.543864 24115 tablet_bootstrap.cc:492] T a7d2ab61d238471c9f287ac1d5db4ed0 P 235ad6af04c34c88a5261daa3604fada: Bootstrap complete.
I20250109 08:58:32.544508 24115 ts_tablet_manager.cc:1397] T a7d2ab61d238471c9f287ac1d5db4ed0 P 235ad6af04c34c88a5261daa3604fada: Time spent bootstrapping tablet: real 0.019s user 0.011s sys 0.004s
I20250109 08:58:32.546388 24115 raft_consensus.cc:357] T a7d2ab61d238471c9f287ac1d5db4ed0 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:32.546844 24115 raft_consensus.cc:738] T a7d2ab61d238471c9f287ac1d5db4ed0 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Initialized, Role: FOLLOWER
I20250109 08:58:32.547350 24115 consensus_queue.cc:260] T a7d2ab61d238471c9f287ac1d5db4ed0 P 235ad6af04c34c88a5261daa3604fada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:32.548823 24115 ts_tablet_manager.cc:1428] T a7d2ab61d238471c9f287ac1d5db4ed0 P 235ad6af04c34c88a5261daa3604fada: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20250109 08:58:32.549587 24115 tablet_bootstrap.cc:492] T 1c5b5beea4a840eb9677097cacee3408 P 235ad6af04c34c88a5261daa3604fada: Bootstrap starting.
I20250109 08:58:32.570230 24115 tablet_bootstrap.cc:492] T 1c5b5beea4a840eb9677097cacee3408 P 235ad6af04c34c88a5261daa3604fada: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250109 08:58:32.571192 24115 tablet_bootstrap.cc:492] T 1c5b5beea4a840eb9677097cacee3408 P 235ad6af04c34c88a5261daa3604fada: Bootstrap complete.
I20250109 08:58:32.571861 24115 ts_tablet_manager.cc:1397] T 1c5b5beea4a840eb9677097cacee3408 P 235ad6af04c34c88a5261daa3604fada: Time spent bootstrapping tablet: real 0.022s user 0.014s sys 0.000s
I20250109 08:58:32.574147 24115 raft_consensus.cc:357] T 1c5b5beea4a840eb9677097cacee3408 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:32.574885 24115 raft_consensus.cc:738] T 1c5b5beea4a840eb9677097cacee3408 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Initialized, Role: FOLLOWER
I20250109 08:58:32.575510 24115 consensus_queue.cc:260] T 1c5b5beea4a840eb9677097cacee3408 P 235ad6af04c34c88a5261daa3604fada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:32.577576 24115 ts_tablet_manager.cc:1428] T 1c5b5beea4a840eb9677097cacee3408 P 235ad6af04c34c88a5261daa3604fada: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
W20250109 08:58:32.810731 23549 debug-util.cc:398] Leaking SignalData structure 0x7b08001bcc00 after lost signal to thread 20366
W20250109 08:58:32.811723 23549 debug-util.cc:398] Leaking SignalData structure 0x7b08001e68c0 after lost signal to thread 23237
W20250109 08:58:32.812723 23549 debug-util.cc:398] Leaking SignalData structure 0x7b0800319c60 after lost signal to thread 23404
W20250109 08:58:32.813661 23549 debug-util.cc:398] Leaking SignalData structure 0x7b0800209a00 after lost signal to thread 23478
W20250109 08:58:32.814479 23549 debug-util.cc:398] Leaking SignalData structure 0x7b08000c6880 after lost signal to thread 23552
W20250109 08:58:32.815474 23549 debug-util.cc:398] Leaking SignalData structure 0x7b08002b9600 after lost signal to thread 24184
I20250109 08:58:32.917207 24157 raft_consensus.cc:3054] T 3208a99957554f6db341654856980c8a P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:40.916344 20363 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250109 08:58:40.967998 23526 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "0dab6c23ce444dda9427901a98f07b15"
dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01"
mode: GRACEFUL
new_leader_uuid: "cf55383bef6649eb9e78cb2d2512dedc"
from {username='slave'} at 127.0.0.1:45238
I20250109 08:58:40.969018 23526 raft_consensus.cc:604] T 0dab6c23ce444dda9427901a98f07b15 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 LEADER]: Received request to transfer leadership to TS cf55383bef6649eb9e78cb2d2512dedc
I20250109 08:58:40.971891 23526 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "2ab70546dda143b5b5db15c3875aa348"
dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01"
mode: GRACEFUL
new_leader_uuid: "235ad6af04c34c88a5261daa3604fada"
from {username='slave'} at 127.0.0.1:45238
I20250109 08:58:40.972404 23526 raft_consensus.cc:604] T 2ab70546dda143b5b5db15c3875aa348 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 LEADER]: Received request to transfer leadership to TS 235ad6af04c34c88a5261daa3604fada
I20250109 08:58:40.984215 23450 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "2e3d34c0d09843d2be54e47f3ccc16d8"
dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc"
mode: GRACEFUL
new_leader_uuid: "235ad6af04c34c88a5261daa3604fada"
from {username='slave'} at 127.0.0.1:33404
I20250109 08:58:40.984760 23450 raft_consensus.cc:604] T 2e3d34c0d09843d2be54e47f3ccc16d8 P cf55383bef6649eb9e78cb2d2512dedc [term 1 LEADER]: Received request to transfer leadership to TS 235ad6af04c34c88a5261daa3604fada
I20250109 08:58:40.998359 23374 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "940653b6d55c46439827697b9aadf16b"
dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c"
mode: GRACEFUL
new_leader_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01"
from {username='slave'} at 127.0.0.1:51604
I20250109 08:58:40.998870 23374 raft_consensus.cc:604] T 940653b6d55c46439827697b9aadf16b P ed1efbce9ee3401e99a5260edeae4e7c [term 1 LEADER]: Received request to transfer leadership to TS 05a74e1eb2474b3d8b3a2da1ef127e01
I20250109 08:58:41.001503 23526 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "970a82e754db47b6b1a19551d4ba43d8"
dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01"
mode: GRACEFUL
new_leader_uuid: "235ad6af04c34c88a5261daa3604fada"
from {username='slave'} at 127.0.0.1:45238
I20250109 08:58:41.004420 23526 raft_consensus.cc:604] T 970a82e754db47b6b1a19551d4ba43d8 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 LEADER]: Received request to transfer leadership to TS 235ad6af04c34c88a5261daa3604fada
I20250109 08:58:41.006717 23526 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "b572a6a3dfc04ea199bc703ddad8af67"
dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01"
mode: GRACEFUL
new_leader_uuid: "235ad6af04c34c88a5261daa3604fada"
from {username='slave'} at 127.0.0.1:45238
I20250109 08:58:41.007157 23526 raft_consensus.cc:604] T b572a6a3dfc04ea199bc703ddad8af67 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 LEADER]: Received request to transfer leadership to TS 235ad6af04c34c88a5261daa3604fada
I20250109 08:58:41.009181 23526 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "ea2b0762940e44dbbf69b9a163ace846"
dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01"
mode: GRACEFUL
new_leader_uuid: "235ad6af04c34c88a5261daa3604fada"
from {username='slave'} at 127.0.0.1:45238
I20250109 08:58:41.009693 23526 raft_consensus.cc:604] T ea2b0762940e44dbbf69b9a163ace846 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 LEADER]: Received request to transfer leadership to TS 235ad6af04c34c88a5261daa3604fada
I20250109 08:58:41.010619 20363 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 7
I20250109 08:58:41.011795 20363 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250109 08:58:41.012307 20363 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250109 08:58:41.019150 24319 raft_consensus.cc:988] T 970a82e754db47b6b1a19551d4ba43d8 P 05a74e1eb2474b3d8b3a2da1ef127e01: : Instructing follower 235ad6af04c34c88a5261daa3604fada to start an election
I20250109 08:58:41.019821 24319 raft_consensus.cc:1076] T 970a82e754db47b6b1a19551d4ba43d8 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 LEADER]: Signalling peer 235ad6af04c34c88a5261daa3604fada to start an election
I20250109 08:58:41.021479 24157 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "970a82e754db47b6b1a19551d4ba43d8"
dest_uuid: "235ad6af04c34c88a5261daa3604fada"
from {username='slave'} at 127.0.0.1:43346
I20250109 08:58:41.021926 24157 raft_consensus.cc:491] T 970a82e754db47b6b1a19551d4ba43d8 P 235ad6af04c34c88a5261daa3604fada [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:58:41.022325 24157 raft_consensus.cc:3054] T 970a82e754db47b6b1a19551d4ba43d8 P 235ad6af04c34c88a5261daa3604fada [term 2 FOLLOWER]: Advancing to term 3
I20250109 08:58:41.029825 24325 raft_consensus.cc:988] T 0dab6c23ce444dda9427901a98f07b15 P 05a74e1eb2474b3d8b3a2da1ef127e01: : Instructing follower cf55383bef6649eb9e78cb2d2512dedc to start an election
I20250109 08:58:41.030369 24325 raft_consensus.cc:1076] T 0dab6c23ce444dda9427901a98f07b15 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 LEADER]: Signalling peer cf55383bef6649eb9e78cb2d2512dedc to start an election
I20250109 08:58:41.031883 23450 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "0dab6c23ce444dda9427901a98f07b15"
dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc"
from {username='slave'} at 127.0.0.1:58530
I20250109 08:58:41.031749 24157 raft_consensus.cc:513] T 970a82e754db47b6b1a19551d4ba43d8 P 235ad6af04c34c88a5261daa3604fada [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:41.032593 23450 raft_consensus.cc:491] T 0dab6c23ce444dda9427901a98f07b15 P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:58:41.032979 23450 raft_consensus.cc:3054] T 0dab6c23ce444dda9427901a98f07b15 P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:41.034080 24157 leader_election.cc:290] T 970a82e754db47b6b1a19551d4ba43d8 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 3 election: Requested vote from peers 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829), ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365)
I20250109 08:58:41.040228 23450 raft_consensus.cc:513] T 0dab6c23ce444dda9427901a98f07b15 P cf55383bef6649eb9e78cb2d2512dedc [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:41.044080 23450 leader_election.cc:290] T 0dab6c23ce444dda9427901a98f07b15 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 2 election: Requested vote from peers ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365), 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829)
I20250109 08:58:41.044509 23374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0dab6c23ce444dda9427901a98f07b15" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c"
I20250109 08:58:41.045400 23374 raft_consensus.cc:3054] T 0dab6c23ce444dda9427901a98f07b15 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:41.050927 23522 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0dab6c23ce444dda9427901a98f07b15" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01"
I20250109 08:58:41.051573 23522 raft_consensus.cc:3049] T 0dab6c23ce444dda9427901a98f07b15 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 LEADER]: Stepping down as leader of term 1
I20250109 08:58:41.051860 23522 raft_consensus.cc:738] T 0dab6c23ce444dda9427901a98f07b15 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 05a74e1eb2474b3d8b3a2da1ef127e01, State: Running, Role: LEADER
I20250109 08:58:41.052716 23522 consensus_queue.cc:260] T 0dab6c23ce444dda9427901a98f07b15 P 05a74e1eb2474b3d8b3a2da1ef127e01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:41.053858 23522 raft_consensus.cc:3054] T 0dab6c23ce444dda9427901a98f07b15 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:41.066946 24326 raft_consensus.cc:988] T b572a6a3dfc04ea199bc703ddad8af67 P 05a74e1eb2474b3d8b3a2da1ef127e01: : Instructing follower 235ad6af04c34c88a5261daa3604fada to start an election
I20250109 08:58:41.067515 24326 raft_consensus.cc:1076] T b572a6a3dfc04ea199bc703ddad8af67 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 LEADER]: Signalling peer 235ad6af04c34c88a5261daa3604fada to start an election
I20250109 08:58:41.068079 23526 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "970a82e754db47b6b1a19551d4ba43d8" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01"
I20250109 08:58:41.068740 23526 raft_consensus.cc:3049] T 970a82e754db47b6b1a19551d4ba43d8 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 LEADER]: Stepping down as leader of term 2
I20250109 08:58:41.068923 23526 raft_consensus.cc:738] T 970a82e754db47b6b1a19551d4ba43d8 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 05a74e1eb2474b3d8b3a2da1ef127e01, State: Running, Role: LEADER
I20250109 08:58:41.069521 23526 consensus_queue.cc:260] T 970a82e754db47b6b1a19551d4ba43d8 P 05a74e1eb2474b3d8b3a2da1ef127e01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:41.070887 23526 raft_consensus.cc:3054] T 970a82e754db47b6b1a19551d4ba43d8 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 FOLLOWER]: Advancing to term 3
I20250109 08:58:41.074898 23374 raft_consensus.cc:2463] T 0dab6c23ce444dda9427901a98f07b15 P ed1efbce9ee3401e99a5260edeae4e7c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf55383bef6649eb9e78cb2d2512dedc in term 2.
I20250109 08:58:41.076239 23420 leader_election.cc:304] T 0dab6c23ce444dda9427901a98f07b15 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf55383bef6649eb9e78cb2d2512dedc, ed1efbce9ee3401e99a5260edeae4e7c; no voters:
I20250109 08:58:41.077174 24152 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "b572a6a3dfc04ea199bc703ddad8af67"
dest_uuid: "235ad6af04c34c88a5261daa3604fada"
from {username='slave'} at 127.0.0.1:43346
I20250109 08:58:41.077991 24152 raft_consensus.cc:491] T b572a6a3dfc04ea199bc703ddad8af67 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:58:41.078423 24152 raft_consensus.cc:3054] T b572a6a3dfc04ea199bc703ddad8af67 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:41.077777 24316 raft_consensus.cc:2798] T 0dab6c23ce444dda9427901a98f07b15 P cf55383bef6649eb9e78cb2d2512dedc [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:58:41.079111 24316 raft_consensus.cc:695] T 0dab6c23ce444dda9427901a98f07b15 P cf55383bef6649eb9e78cb2d2512dedc [term 2 LEADER]: Becoming Leader. State: Replica: cf55383bef6649eb9e78cb2d2512dedc, State: Running, Role: LEADER
I20250109 08:58:41.079356 23522 raft_consensus.cc:2463] T 0dab6c23ce444dda9427901a98f07b15 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf55383bef6649eb9e78cb2d2512dedc in term 2.
I20250109 08:58:41.080843 24316 consensus_queue.cc:237] T 0dab6c23ce444dda9427901a98f07b15 P cf55383bef6649eb9e78cb2d2512dedc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:41.084951 23374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "970a82e754db47b6b1a19551d4ba43d8" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c"
I20250109 08:58:41.085937 23374 raft_consensus.cc:3054] T 970a82e754db47b6b1a19551d4ba43d8 P ed1efbce9ee3401e99a5260edeae4e7c [term 2 FOLLOWER]: Advancing to term 3
I20250109 08:58:41.077131 23526 raft_consensus.cc:2463] T 970a82e754db47b6b1a19551d4ba43d8 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 235ad6af04c34c88a5261daa3604fada in term 3.
I20250109 08:58:41.088030 24110 leader_election.cc:304] T 970a82e754db47b6b1a19551d4ba43d8 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01, 235ad6af04c34c88a5261daa3604fada; no voters:
I20250109 08:58:41.091267 24152 raft_consensus.cc:513] T b572a6a3dfc04ea199bc703ddad8af67 P 235ad6af04c34c88a5261daa3604fada [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:41.097492 24152 leader_election.cc:290] T b572a6a3dfc04ea199bc703ddad8af67 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 2 election: Requested vote from peers cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551), 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829)
I20250109 08:58:41.101984 24330 raft_consensus.cc:2798] T 970a82e754db47b6b1a19551d4ba43d8 P 235ad6af04c34c88a5261daa3604fada [term 3 FOLLOWER]: Leader election won for term 3
I20250109 08:58:41.102581 23522 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b572a6a3dfc04ea199bc703ddad8af67" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01"
I20250109 08:58:41.103230 23522 raft_consensus.cc:3049] T b572a6a3dfc04ea199bc703ddad8af67 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 LEADER]: Stepping down as leader of term 1
I20250109 08:58:41.103543 23522 raft_consensus.cc:738] T b572a6a3dfc04ea199bc703ddad8af67 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 05a74e1eb2474b3d8b3a2da1ef127e01, State: Running, Role: LEADER
I20250109 08:58:41.104238 23522 consensus_queue.cc:260] T b572a6a3dfc04ea199bc703ddad8af67 P 05a74e1eb2474b3d8b3a2da1ef127e01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:41.105326 23522 raft_consensus.cc:3054] T b572a6a3dfc04ea199bc703ddad8af67 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:41.112022 23450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b572a6a3dfc04ea199bc703ddad8af67" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc"
I20250109 08:58:41.112756 23450 raft_consensus.cc:3054] T b572a6a3dfc04ea199bc703ddad8af67 P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:41.114760 23202 catalog_manager.cc:5526] T 0dab6c23ce444dda9427901a98f07b15 P cf55383bef6649eb9e78cb2d2512dedc reported cstate change: term changed from 1 to 2, leader changed from 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196) to cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195). New cstate: current_term: 2 leader_uuid: "cf55383bef6649eb9e78cb2d2512dedc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:41.116712 23374 raft_consensus.cc:2463] T 970a82e754db47b6b1a19551d4ba43d8 P ed1efbce9ee3401e99a5260edeae4e7c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 235ad6af04c34c88a5261daa3604fada in term 3.
I20250109 08:58:41.118206 24330 raft_consensus.cc:695] T 970a82e754db47b6b1a19551d4ba43d8 P 235ad6af04c34c88a5261daa3604fada [term 3 LEADER]: Becoming Leader. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Running, Role: LEADER
I20250109 08:58:41.119220 24330 consensus_queue.cc:237] T 970a82e754db47b6b1a19551d4ba43d8 P 235ad6af04c34c88a5261daa3604fada [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:41.120574 23450 raft_consensus.cc:2463] T b572a6a3dfc04ea199bc703ddad8af67 P cf55383bef6649eb9e78cb2d2512dedc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 235ad6af04c34c88a5261daa3604fada in term 2.
I20250109 08:58:41.125327 24112 leader_election.cc:304] T b572a6a3dfc04ea199bc703ddad8af67 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 235ad6af04c34c88a5261daa3604fada, cf55383bef6649eb9e78cb2d2512dedc; no voters:
I20250109 08:58:41.123492 23522 raft_consensus.cc:2463] T b572a6a3dfc04ea199bc703ddad8af67 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 235ad6af04c34c88a5261daa3604fada in term 2.
I20250109 08:58:41.126304 24331 raft_consensus.cc:2798] T b572a6a3dfc04ea199bc703ddad8af67 P 235ad6af04c34c88a5261daa3604fada [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:58:41.139720 24331 raft_consensus.cc:695] T b572a6a3dfc04ea199bc703ddad8af67 P 235ad6af04c34c88a5261daa3604fada [term 2 LEADER]: Becoming Leader. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Running, Role: LEADER
I20250109 08:58:41.140848 24331 consensus_queue.cc:237] T b572a6a3dfc04ea199bc703ddad8af67 P 235ad6af04c34c88a5261daa3604fada [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:41.145105 24319 raft_consensus.cc:988] T 2ab70546dda143b5b5db15c3875aa348 P 05a74e1eb2474b3d8b3a2da1ef127e01: : Instructing follower 235ad6af04c34c88a5261daa3604fada to start an election
I20250109 08:58:41.145620 24326 raft_consensus.cc:1076] T 2ab70546dda143b5b5db15c3875aa348 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 LEADER]: Signalling peer 235ad6af04c34c88a5261daa3604fada to start an election
I20250109 08:58:41.146270 23201 catalog_manager.cc:5526] T 970a82e754db47b6b1a19551d4ba43d8 P 235ad6af04c34c88a5261daa3604fada reported cstate change: term changed from 2 to 3, leader changed from 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196) to 235ad6af04c34c88a5261daa3604fada (127.19.226.193). New cstate: current_term: 3 leader_uuid: "235ad6af04c34c88a5261daa3604fada" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:41.147125 24152 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2ab70546dda143b5b5db15c3875aa348"
dest_uuid: "235ad6af04c34c88a5261daa3604fada"
from {username='slave'} at 127.0.0.1:43346
I20250109 08:58:41.148717 24152 raft_consensus.cc:491] T 2ab70546dda143b5b5db15c3875aa348 P 235ad6af04c34c88a5261daa3604fada [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:58:41.149159 24152 raft_consensus.cc:3054] T 2ab70546dda143b5b5db15c3875aa348 P 235ad6af04c34c88a5261daa3604fada [term 2 FOLLOWER]: Advancing to term 3
I20250109 08:58:41.157214 24152 raft_consensus.cc:513] T 2ab70546dda143b5b5db15c3875aa348 P 235ad6af04c34c88a5261daa3604fada [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:41.160851 23374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ab70546dda143b5b5db15c3875aa348" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c"
I20250109 08:58:41.161460 23374 raft_consensus.cc:3054] T 2ab70546dda143b5b5db15c3875aa348 P ed1efbce9ee3401e99a5260edeae4e7c [term 2 FOLLOWER]: Advancing to term 3
I20250109 08:58:41.164273 23522 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ab70546dda143b5b5db15c3875aa348" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01"
I20250109 08:58:41.164916 23522 raft_consensus.cc:3049] T 2ab70546dda143b5b5db15c3875aa348 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 LEADER]: Stepping down as leader of term 2
I20250109 08:58:41.165181 23522 raft_consensus.cc:738] T 2ab70546dda143b5b5db15c3875aa348 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 05a74e1eb2474b3d8b3a2da1ef127e01, State: Running, Role: LEADER
I20250109 08:58:41.165932 23522 consensus_queue.cc:260] T 2ab70546dda143b5b5db15c3875aa348 P 05a74e1eb2474b3d8b3a2da1ef127e01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:41.167093 23522 raft_consensus.cc:3054] T 2ab70546dda143b5b5db15c3875aa348 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 FOLLOWER]: Advancing to term 3
I20250109 08:58:41.185876 23522 raft_consensus.cc:2463] T 2ab70546dda143b5b5db15c3875aa348 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 235ad6af04c34c88a5261daa3604fada in term 3.
I20250109 08:58:41.187114 24110 leader_election.cc:304] T 2ab70546dda143b5b5db15c3875aa348 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01, 235ad6af04c34c88a5261daa3604fada; no voters:
I20250109 08:58:41.187963 24331 raft_consensus.cc:2798] T 2ab70546dda143b5b5db15c3875aa348 P 235ad6af04c34c88a5261daa3604fada [term 3 FOLLOWER]: Leader election won for term 3
I20250109 08:58:41.188452 24331 raft_consensus.cc:695] T 2ab70546dda143b5b5db15c3875aa348 P 235ad6af04c34c88a5261daa3604fada [term 3 LEADER]: Becoming Leader. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Running, Role: LEADER
I20250109 08:58:41.189419 24331 consensus_queue.cc:237] T 2ab70546dda143b5b5db15c3875aa348 P 235ad6af04c34c88a5261daa3604fada [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:41.194182 23374 raft_consensus.cc:2463] T 2ab70546dda143b5b5db15c3875aa348 P ed1efbce9ee3401e99a5260edeae4e7c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 235ad6af04c34c88a5261daa3604fada in term 3.
I20250109 08:58:41.196162 24152 leader_election.cc:290] T 2ab70546dda143b5b5db15c3875aa348 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 3 election: Requested vote from peers 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829), ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365)
I20250109 08:58:41.203661 23202 catalog_manager.cc:5526] T b572a6a3dfc04ea199bc703ddad8af67 P 235ad6af04c34c88a5261daa3604fada reported cstate change: term changed from 1 to 2, leader changed from 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196) to 235ad6af04c34c88a5261daa3604fada (127.19.226.193). New cstate: current_term: 2 leader_uuid: "235ad6af04c34c88a5261daa3604fada" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } health_report { overall_health: HEALTHY } } }
I20250109 08:58:41.217794 23202 catalog_manager.cc:5526] T 2ab70546dda143b5b5db15c3875aa348 P 235ad6af04c34c88a5261daa3604fada reported cstate change: term changed from 2 to 3, leader changed from 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196) to 235ad6af04c34c88a5261daa3604fada (127.19.226.193). New cstate: current_term: 3 leader_uuid: "235ad6af04c34c88a5261daa3604fada" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:41.243964 24319 raft_consensus.cc:988] T ea2b0762940e44dbbf69b9a163ace846 P 05a74e1eb2474b3d8b3a2da1ef127e01: : Instructing follower 235ad6af04c34c88a5261daa3604fada to start an election
I20250109 08:58:41.244349 24326 raft_consensus.cc:1076] T ea2b0762940e44dbbf69b9a163ace846 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 LEADER]: Signalling peer 235ad6af04c34c88a5261daa3604fada to start an election
I20250109 08:58:41.245538 24157 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "ea2b0762940e44dbbf69b9a163ace846"
dest_uuid: "235ad6af04c34c88a5261daa3604fada"
from {username='slave'} at 127.0.0.1:43346
I20250109 08:58:41.245939 24157 raft_consensus.cc:491] T ea2b0762940e44dbbf69b9a163ace846 P 235ad6af04c34c88a5261daa3604fada [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:58:41.246171 24157 raft_consensus.cc:3054] T ea2b0762940e44dbbf69b9a163ace846 P 235ad6af04c34c88a5261daa3604fada [term 2 FOLLOWER]: Advancing to term 3
I20250109 08:58:41.250778 24157 raft_consensus.cc:513] T ea2b0762940e44dbbf69b9a163ace846 P 235ad6af04c34c88a5261daa3604fada [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:41.252413 24157 leader_election.cc:290] T ea2b0762940e44dbbf69b9a163ace846 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 3 election: Requested vote from peers 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829), ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365)
I20250109 08:58:41.252996 23522 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea2b0762940e44dbbf69b9a163ace846" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01"
I20250109 08:58:41.253150 23374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea2b0762940e44dbbf69b9a163ace846" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c"
I20250109 08:58:41.253620 23522 raft_consensus.cc:3049] T ea2b0762940e44dbbf69b9a163ace846 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 LEADER]: Stepping down as leader of term 2
I20250109 08:58:41.253758 23374 raft_consensus.cc:3054] T ea2b0762940e44dbbf69b9a163ace846 P ed1efbce9ee3401e99a5260edeae4e7c [term 2 FOLLOWER]: Advancing to term 3
I20250109 08:58:41.253842 23522 raft_consensus.cc:738] T ea2b0762940e44dbbf69b9a163ace846 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 05a74e1eb2474b3d8b3a2da1ef127e01, State: Running, Role: LEADER
I20250109 08:58:41.254695 23522 consensus_queue.cc:260] T ea2b0762940e44dbbf69b9a163ace846 P 05a74e1eb2474b3d8b3a2da1ef127e01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:41.255623 23522 raft_consensus.cc:3054] T ea2b0762940e44dbbf69b9a163ace846 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 FOLLOWER]: Advancing to term 3
I20250109 08:58:41.259806 23374 raft_consensus.cc:2463] T ea2b0762940e44dbbf69b9a163ace846 P ed1efbce9ee3401e99a5260edeae4e7c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 235ad6af04c34c88a5261daa3604fada in term 3.
I20250109 08:58:41.260478 23522 raft_consensus.cc:2463] T ea2b0762940e44dbbf69b9a163ace846 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 235ad6af04c34c88a5261daa3604fada in term 3.
I20250109 08:58:41.261161 24112 leader_election.cc:304] T ea2b0762940e44dbbf69b9a163ace846 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 235ad6af04c34c88a5261daa3604fada, ed1efbce9ee3401e99a5260edeae4e7c; no voters:
I20250109 08:58:41.262006 24331 raft_consensus.cc:2798] T ea2b0762940e44dbbf69b9a163ace846 P 235ad6af04c34c88a5261daa3604fada [term 3 FOLLOWER]: Leader election won for term 3
I20250109 08:58:41.262719 24331 raft_consensus.cc:695] T ea2b0762940e44dbbf69b9a163ace846 P 235ad6af04c34c88a5261daa3604fada [term 3 LEADER]: Becoming Leader. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Running, Role: LEADER
I20250109 08:58:41.263936 24331 consensus_queue.cc:237] T ea2b0762940e44dbbf69b9a163ace846 P 235ad6af04c34c88a5261daa3604fada [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:41.274722 23202 catalog_manager.cc:5526] T ea2b0762940e44dbbf69b9a163ace846 P 235ad6af04c34c88a5261daa3604fada reported cstate change: term changed from 2 to 3, leader changed from 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196) to 235ad6af04c34c88a5261daa3604fada (127.19.226.193). New cstate: current_term: 3 leader_uuid: "235ad6af04c34c88a5261daa3604fada" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:41.301622 24320 raft_consensus.cc:988] T 2e3d34c0d09843d2be54e47f3ccc16d8 P cf55383bef6649eb9e78cb2d2512dedc: : Instructing follower 235ad6af04c34c88a5261daa3604fada to start an election
I20250109 08:58:41.302096 24316 raft_consensus.cc:1076] T 2e3d34c0d09843d2be54e47f3ccc16d8 P cf55383bef6649eb9e78cb2d2512dedc [term 1 LEADER]: Signalling peer 235ad6af04c34c88a5261daa3604fada to start an election
I20250109 08:58:41.303565 24152 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2e3d34c0d09843d2be54e47f3ccc16d8"
dest_uuid: "235ad6af04c34c88a5261daa3604fada"
from {username='slave'} at 127.0.0.1:43338
I20250109 08:58:41.304014 24152 raft_consensus.cc:491] T 2e3d34c0d09843d2be54e47f3ccc16d8 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:58:41.304271 24152 raft_consensus.cc:3054] T 2e3d34c0d09843d2be54e47f3ccc16d8 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:41.309491 24152 raft_consensus.cc:513] T 2e3d34c0d09843d2be54e47f3ccc16d8 P 235ad6af04c34c88a5261daa3604fada [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:41.311015 24152 leader_election.cc:290] T 2e3d34c0d09843d2be54e47f3ccc16d8 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 2 election: Requested vote from peers ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365), cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551)
I20250109 08:58:41.311790 23374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e3d34c0d09843d2be54e47f3ccc16d8" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c"
I20250109 08:58:41.312209 23446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e3d34c0d09843d2be54e47f3ccc16d8" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc"
I20250109 08:58:41.312526 23374 raft_consensus.cc:3054] T 2e3d34c0d09843d2be54e47f3ccc16d8 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:41.312870 23446 raft_consensus.cc:3049] T 2e3d34c0d09843d2be54e47f3ccc16d8 P cf55383bef6649eb9e78cb2d2512dedc [term 1 LEADER]: Stepping down as leader of term 1
I20250109 08:58:41.313158 23446 raft_consensus.cc:738] T 2e3d34c0d09843d2be54e47f3ccc16d8 P cf55383bef6649eb9e78cb2d2512dedc [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cf55383bef6649eb9e78cb2d2512dedc, State: Running, Role: LEADER
I20250109 08:58:41.313921 23446 consensus_queue.cc:260] T 2e3d34c0d09843d2be54e47f3ccc16d8 P cf55383bef6649eb9e78cb2d2512dedc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:41.315217 23446 raft_consensus.cc:3054] T 2e3d34c0d09843d2be54e47f3ccc16d8 P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:41.319070 23374 raft_consensus.cc:2463] T 2e3d34c0d09843d2be54e47f3ccc16d8 P ed1efbce9ee3401e99a5260edeae4e7c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 235ad6af04c34c88a5261daa3604fada in term 2.
I20250109 08:58:41.320112 24112 leader_election.cc:304] T 2e3d34c0d09843d2be54e47f3ccc16d8 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 235ad6af04c34c88a5261daa3604fada, ed1efbce9ee3401e99a5260edeae4e7c; no voters:
I20250109 08:58:41.320909 24331 raft_consensus.cc:2798] T 2e3d34c0d09843d2be54e47f3ccc16d8 P 235ad6af04c34c88a5261daa3604fada [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:58:41.321498 24331 raft_consensus.cc:695] T 2e3d34c0d09843d2be54e47f3ccc16d8 P 235ad6af04c34c88a5261daa3604fada [term 2 LEADER]: Becoming Leader. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Running, Role: LEADER
I20250109 08:58:41.322316 24331 consensus_queue.cc:237] T 2e3d34c0d09843d2be54e47f3ccc16d8 P 235ad6af04c34c88a5261daa3604fada [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:41.322762 23446 raft_consensus.cc:2463] T 2e3d34c0d09843d2be54e47f3ccc16d8 P cf55383bef6649eb9e78cb2d2512dedc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 235ad6af04c34c88a5261daa3604fada in term 2.
I20250109 08:58:41.331418 23202 catalog_manager.cc:5526] T 2e3d34c0d09843d2be54e47f3ccc16d8 P 235ad6af04c34c88a5261daa3604fada reported cstate change: term changed from 1 to 2, leader changed from cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195) to 235ad6af04c34c88a5261daa3604fada (127.19.226.193). New cstate: current_term: 2 leader_uuid: "235ad6af04c34c88a5261daa3604fada" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:41.402730 24318 raft_consensus.cc:988] T 940653b6d55c46439827697b9aadf16b P ed1efbce9ee3401e99a5260edeae4e7c: : Instructing follower 05a74e1eb2474b3d8b3a2da1ef127e01 to start an election
I20250109 08:58:41.403234 24317 raft_consensus.cc:1076] T 940653b6d55c46439827697b9aadf16b P ed1efbce9ee3401e99a5260edeae4e7c [term 1 LEADER]: Signalling peer 05a74e1eb2474b3d8b3a2da1ef127e01 to start an election
I20250109 08:58:41.404476 23522 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "940653b6d55c46439827697b9aadf16b"
dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01"
from {username='slave'} at 127.0.0.1:59000
I20250109 08:58:41.404875 23522 raft_consensus.cc:491] T 940653b6d55c46439827697b9aadf16b P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:58:41.405090 23522 raft_consensus.cc:3054] T 940653b6d55c46439827697b9aadf16b P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:41.410468 23522 raft_consensus.cc:513] T 940653b6d55c46439827697b9aadf16b P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:41.412272 23522 leader_election.cc:290] T 940653b6d55c46439827697b9aadf16b P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 2 election: Requested vote from peers ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365), cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551)
I20250109 08:58:41.413105 23446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "940653b6d55c46439827697b9aadf16b" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc"
I20250109 08:58:41.413425 23374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "940653b6d55c46439827697b9aadf16b" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c"
I20250109 08:58:41.414028 23446 raft_consensus.cc:3054] T 940653b6d55c46439827697b9aadf16b P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:41.414136 23374 raft_consensus.cc:3049] T 940653b6d55c46439827697b9aadf16b P ed1efbce9ee3401e99a5260edeae4e7c [term 1 LEADER]: Stepping down as leader of term 1
I20250109 08:58:41.414538 23374 raft_consensus.cc:738] T 940653b6d55c46439827697b9aadf16b P ed1efbce9ee3401e99a5260edeae4e7c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ed1efbce9ee3401e99a5260edeae4e7c, State: Running, Role: LEADER
I20250109 08:58:41.415117 23374 consensus_queue.cc:260] T 940653b6d55c46439827697b9aadf16b P ed1efbce9ee3401e99a5260edeae4e7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:41.416608 23374 raft_consensus.cc:3054] T 940653b6d55c46439827697b9aadf16b P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:41.419433 23446 raft_consensus.cc:2463] T 940653b6d55c46439827697b9aadf16b P cf55383bef6649eb9e78cb2d2512dedc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05a74e1eb2474b3d8b3a2da1ef127e01 in term 2.
I20250109 08:58:41.420583 23494 leader_election.cc:304] T 940653b6d55c46439827697b9aadf16b P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01, cf55383bef6649eb9e78cb2d2512dedc; no voters:
I20250109 08:58:41.421289 24319 raft_consensus.cc:2798] T 940653b6d55c46439827697b9aadf16b P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:58:41.421649 24319 raft_consensus.cc:695] T 940653b6d55c46439827697b9aadf16b P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 LEADER]: Becoming Leader. State: Replica: 05a74e1eb2474b3d8b3a2da1ef127e01, State: Running, Role: LEADER
I20250109 08:58:41.421772 23374 raft_consensus.cc:2463] T 940653b6d55c46439827697b9aadf16b P ed1efbce9ee3401e99a5260edeae4e7c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05a74e1eb2474b3d8b3a2da1ef127e01 in term 2.
I20250109 08:58:41.422433 24319 consensus_queue.cc:237] T 940653b6d55c46439827697b9aadf16b P 05a74e1eb2474b3d8b3a2da1ef127e01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:41.433578 23202 catalog_manager.cc:5526] T 940653b6d55c46439827697b9aadf16b P 05a74e1eb2474b3d8b3a2da1ef127e01 reported cstate change: term changed from 1 to 2, leader changed from ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194) to 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196). New cstate: current_term: 2 leader_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } health_report { overall_health: HEALTHY } } }
I20250109 08:58:41.533382 23522 raft_consensus.cc:1270] T 970a82e754db47b6b1a19551d4ba43d8 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 3 FOLLOWER]: Refusing update from remote peer 235ad6af04c34c88a5261daa3604fada: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20250109 08:58:41.534360 24331 consensus_queue.cc:1035] T 970a82e754db47b6b1a19551d4ba43d8 P 235ad6af04c34c88a5261daa3604fada [LEADER]: Connected to new peer: Peer: permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:58:41.543309 23374 raft_consensus.cc:1270] T 970a82e754db47b6b1a19551d4ba43d8 P ed1efbce9ee3401e99a5260edeae4e7c [term 3 FOLLOWER]: Refusing update from remote peer 235ad6af04c34c88a5261daa3604fada: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20250109 08:58:41.547879 24330 consensus_queue.cc:1035] T 970a82e754db47b6b1a19551d4ba43d8 P 235ad6af04c34c88a5261daa3604fada [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:58:41.622507 23522 raft_consensus.cc:1270] T 0dab6c23ce444dda9427901a98f07b15 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 FOLLOWER]: Refusing update from remote peer cf55383bef6649eb9e78cb2d2512dedc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:58:41.624399 24316 consensus_queue.cc:1035] T 0dab6c23ce444dda9427901a98f07b15 P cf55383bef6649eb9e78cb2d2512dedc [LEADER]: Connected to new peer: Peer: permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:58:41.636142 23374 raft_consensus.cc:1270] T 0dab6c23ce444dda9427901a98f07b15 P ed1efbce9ee3401e99a5260edeae4e7c [term 2 FOLLOWER]: Refusing update from remote peer cf55383bef6649eb9e78cb2d2512dedc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:58:41.637881 23526 raft_consensus.cc:1270] T b572a6a3dfc04ea199bc703ddad8af67 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 FOLLOWER]: Refusing update from remote peer 235ad6af04c34c88a5261daa3604fada: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:58:41.639565 24330 consensus_queue.cc:1035] T b572a6a3dfc04ea199bc703ddad8af67 P 235ad6af04c34c88a5261daa3604fada [LEADER]: Connected to new peer: Peer: permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:58:41.641223 24316 consensus_queue.cc:1035] T 0dab6c23ce444dda9427901a98f07b15 P cf55383bef6649eb9e78cb2d2512dedc [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:58:41.659160 23450 raft_consensus.cc:1270] T b572a6a3dfc04ea199bc703ddad8af67 P cf55383bef6649eb9e78cb2d2512dedc [term 2 FOLLOWER]: Refusing update from remote peer 235ad6af04c34c88a5261daa3604fada: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:58:41.661088 24331 consensus_queue.cc:1035] T b572a6a3dfc04ea199bc703ddad8af67 P 235ad6af04c34c88a5261daa3604fada [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:58:41.717972 23526 raft_consensus.cc:1270] T 2ab70546dda143b5b5db15c3875aa348 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 3 FOLLOWER]: Refusing update from remote peer 235ad6af04c34c88a5261daa3604fada: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20250109 08:58:41.718566 23450 raft_consensus.cc:1270] T 2e3d34c0d09843d2be54e47f3ccc16d8 P cf55383bef6649eb9e78cb2d2512dedc [term 2 FOLLOWER]: Refusing update from remote peer 235ad6af04c34c88a5261daa3604fada: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:58:41.719137 24331 consensus_queue.cc:1035] T 2ab70546dda143b5b5db15c3875aa348 P 235ad6af04c34c88a5261daa3604fada [LEADER]: Connected to new peer: Peer: permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:58:41.719944 24334 consensus_queue.cc:1035] T 2e3d34c0d09843d2be54e47f3ccc16d8 P 235ad6af04c34c88a5261daa3604fada [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:58:41.732638 23374 raft_consensus.cc:1270] T 2e3d34c0d09843d2be54e47f3ccc16d8 P ed1efbce9ee3401e99a5260edeae4e7c [term 2 FOLLOWER]: Refusing update from remote peer 235ad6af04c34c88a5261daa3604fada: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:58:41.734845 24331 consensus_queue.cc:1035] T 2e3d34c0d09843d2be54e47f3ccc16d8 P 235ad6af04c34c88a5261daa3604fada [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:58:41.750419 23373 raft_consensus.cc:1270] T 2ab70546dda143b5b5db15c3875aa348 P ed1efbce9ee3401e99a5260edeae4e7c [term 3 FOLLOWER]: Refusing update from remote peer 235ad6af04c34c88a5261daa3604fada: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20250109 08:58:41.754946 24334 consensus_queue.cc:1035] T 2ab70546dda143b5b5db15c3875aa348 P 235ad6af04c34c88a5261daa3604fada [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:58:41.845211 23446 raft_consensus.cc:1270] T 940653b6d55c46439827697b9aadf16b P cf55383bef6649eb9e78cb2d2512dedc [term 2 FOLLOWER]: Refusing update from remote peer 05a74e1eb2474b3d8b3a2da1ef127e01: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:58:41.846275 24326 consensus_queue.cc:1035] T 940653b6d55c46439827697b9aadf16b P 05a74e1eb2474b3d8b3a2da1ef127e01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:58:41.850570 23526 raft_consensus.cc:1270] T ea2b0762940e44dbbf69b9a163ace846 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 3 FOLLOWER]: Refusing update from remote peer 235ad6af04c34c88a5261daa3604fada: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20250109 08:58:41.852099 24331 consensus_queue.cc:1035] T ea2b0762940e44dbbf69b9a163ace846 P 235ad6af04c34c88a5261daa3604fada [LEADER]: Connected to new peer: Peer: permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20250109 08:58:41.865316 23374 raft_consensus.cc:1270] T ea2b0762940e44dbbf69b9a163ace846 P ed1efbce9ee3401e99a5260edeae4e7c [term 3 FOLLOWER]: Refusing update from remote peer 235ad6af04c34c88a5261daa3604fada: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20250109 08:58:41.866586 23373 raft_consensus.cc:1270] T 940653b6d55c46439827697b9aadf16b P ed1efbce9ee3401e99a5260edeae4e7c [term 2 FOLLOWER]: Refusing update from remote peer 05a74e1eb2474b3d8b3a2da1ef127e01: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:58:41.866847 24331 consensus_queue.cc:1035] T ea2b0762940e44dbbf69b9a163ace846 P 235ad6af04c34c88a5261daa3604fada [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:58:41.868585 24319 consensus_queue.cc:1035] T 940653b6d55c46439827697b9aadf16b P 05a74e1eb2474b3d8b3a2da1ef127e01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:58:42.064049 20363 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250109 08:58:42.098739 23450 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "0dab6c23ce444dda9427901a98f07b15"
dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc"
mode: GRACEFUL
new_leader_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01"
from {username='slave'} at 127.0.0.1:33404
I20250109 08:58:42.099367 23450 raft_consensus.cc:604] T 0dab6c23ce444dda9427901a98f07b15 P cf55383bef6649eb9e78cb2d2512dedc [term 2 LEADER]: Received request to transfer leadership to TS 05a74e1eb2474b3d8b3a2da1ef127e01
I20250109 08:58:42.101816 23374 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "11aa76a3a30643708fececed660a5250"
dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c"
mode: GRACEFUL
new_leader_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01"
from {username='slave'} at 127.0.0.1:51604
I20250109 08:58:42.102269 23374 raft_consensus.cc:604] T 11aa76a3a30643708fececed660a5250 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 LEADER]: Received request to transfer leadership to TS 05a74e1eb2474b3d8b3a2da1ef127e01
I20250109 08:58:42.104614 23374 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "1ac4317290b24209a76d2e19e68ee45b"
dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c"
mode: GRACEFUL
new_leader_uuid: "cf55383bef6649eb9e78cb2d2512dedc"
from {username='slave'} at 127.0.0.1:51604
I20250109 08:58:42.105115 23374 raft_consensus.cc:604] T 1ac4317290b24209a76d2e19e68ee45b P ed1efbce9ee3401e99a5260edeae4e7c [term 2 LEADER]: Received request to transfer leadership to TS cf55383bef6649eb9e78cb2d2512dedc
I20250109 08:58:42.107003 23526 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "940653b6d55c46439827697b9aadf16b"
dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01"
mode: GRACEFUL
new_leader_uuid: "cf55383bef6649eb9e78cb2d2512dedc"
from {username='slave'} at 127.0.0.1:45238
I20250109 08:58:42.107441 23526 raft_consensus.cc:604] T 940653b6d55c46439827697b9aadf16b P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 LEADER]: Received request to transfer leadership to TS cf55383bef6649eb9e78cb2d2512dedc
I20250109 08:58:42.109570 23374 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "a686bf4d947c4b638049e6b88c59ae09"
dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c"
mode: GRACEFUL
new_leader_uuid: "cf55383bef6649eb9e78cb2d2512dedc"
from {username='slave'} at 127.0.0.1:51604
I20250109 08:58:42.110018 23374 raft_consensus.cc:604] T a686bf4d947c4b638049e6b88c59ae09 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 LEADER]: Received request to transfer leadership to TS cf55383bef6649eb9e78cb2d2512dedc
I20250109 08:58:42.110980 20363 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 5
I20250109 08:58:42.112057 20363 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250109 08:58:42.112572 20363 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250109 08:58:42.239681 24351 raft_consensus.cc:988] T 0dab6c23ce444dda9427901a98f07b15 P cf55383bef6649eb9e78cb2d2512dedc: : Instructing follower 05a74e1eb2474b3d8b3a2da1ef127e01 to start an election
I20250109 08:58:42.240123 24320 raft_consensus.cc:1076] T 0dab6c23ce444dda9427901a98f07b15 P cf55383bef6649eb9e78cb2d2512dedc [term 2 LEADER]: Signalling peer 05a74e1eb2474b3d8b3a2da1ef127e01 to start an election
I20250109 08:58:42.241422 23526 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "0dab6c23ce444dda9427901a98f07b15"
dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01"
from {username='slave'} at 127.0.0.1:58988
I20250109 08:58:42.241847 23526 raft_consensus.cc:491] T 0dab6c23ce444dda9427901a98f07b15 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:58:42.242102 23526 raft_consensus.cc:3054] T 0dab6c23ce444dda9427901a98f07b15 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 FOLLOWER]: Advancing to term 3
I20250109 08:58:42.246457 23526 raft_consensus.cc:513] T 0dab6c23ce444dda9427901a98f07b15 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:42.247920 23526 leader_election.cc:290] T 0dab6c23ce444dda9427901a98f07b15 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 3 election: Requested vote from peers ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365), cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551)
I20250109 08:58:42.248605 23374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0dab6c23ce444dda9427901a98f07b15" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c"
I20250109 08:58:42.248823 23450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0dab6c23ce444dda9427901a98f07b15" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc"
I20250109 08:58:42.249240 23374 raft_consensus.cc:3054] T 0dab6c23ce444dda9427901a98f07b15 P ed1efbce9ee3401e99a5260edeae4e7c [term 2 FOLLOWER]: Advancing to term 3
I20250109 08:58:42.249344 23450 raft_consensus.cc:3049] T 0dab6c23ce444dda9427901a98f07b15 P cf55383bef6649eb9e78cb2d2512dedc [term 2 LEADER]: Stepping down as leader of term 2
I20250109 08:58:42.249694 23450 raft_consensus.cc:738] T 0dab6c23ce444dda9427901a98f07b15 P cf55383bef6649eb9e78cb2d2512dedc [term 2 LEADER]: Becoming Follower/Learner. State: Replica: cf55383bef6649eb9e78cb2d2512dedc, State: Running, Role: LEADER
I20250109 08:58:42.250346 23450 consensus_queue.cc:260] T 0dab6c23ce444dda9427901a98f07b15 P cf55383bef6649eb9e78cb2d2512dedc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:42.251524 23450 raft_consensus.cc:3054] T 0dab6c23ce444dda9427901a98f07b15 P cf55383bef6649eb9e78cb2d2512dedc [term 2 FOLLOWER]: Advancing to term 3
I20250109 08:58:42.255766 23374 raft_consensus.cc:2463] T 0dab6c23ce444dda9427901a98f07b15 P ed1efbce9ee3401e99a5260edeae4e7c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05a74e1eb2474b3d8b3a2da1ef127e01 in term 3.
I20250109 08:58:42.256827 23494 leader_election.cc:304] T 0dab6c23ce444dda9427901a98f07b15 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01, ed1efbce9ee3401e99a5260edeae4e7c; no voters:
I20250109 08:58:42.257349 23450 raft_consensus.cc:2463] T 0dab6c23ce444dda9427901a98f07b15 P cf55383bef6649eb9e78cb2d2512dedc [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05a74e1eb2474b3d8b3a2da1ef127e01 in term 3.
I20250109 08:58:42.257573 24361 raft_consensus.cc:2798] T 0dab6c23ce444dda9427901a98f07b15 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 3 FOLLOWER]: Leader election won for term 3
I20250109 08:58:42.258066 24361 raft_consensus.cc:695] T 0dab6c23ce444dda9427901a98f07b15 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 3 LEADER]: Becoming Leader. State: Replica: 05a74e1eb2474b3d8b3a2da1ef127e01, State: Running, Role: LEADER
I20250109 08:58:42.258731 24361 consensus_queue.cc:237] T 0dab6c23ce444dda9427901a98f07b15 P 05a74e1eb2474b3d8b3a2da1ef127e01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:42.274436 24318 raft_consensus.cc:988] T 11aa76a3a30643708fececed660a5250 P ed1efbce9ee3401e99a5260edeae4e7c: : Instructing follower 05a74e1eb2474b3d8b3a2da1ef127e01 to start an election
I20250109 08:58:42.275108 24353 raft_consensus.cc:1076] T 11aa76a3a30643708fececed660a5250 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 LEADER]: Signalling peer 05a74e1eb2474b3d8b3a2da1ef127e01 to start an election
I20250109 08:58:42.276594 23526 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "11aa76a3a30643708fececed660a5250"
dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01"
from {username='slave'} at 127.0.0.1:59000
I20250109 08:58:42.269109 23200 catalog_manager.cc:5526] T 0dab6c23ce444dda9427901a98f07b15 P 05a74e1eb2474b3d8b3a2da1ef127e01 reported cstate change: term changed from 2 to 3, leader changed from cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195) to 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196). New cstate: current_term: 3 leader_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } health_report { overall_health: HEALTHY } } }
I20250109 08:58:42.277067 23526 raft_consensus.cc:491] T 11aa76a3a30643708fececed660a5250 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:58:42.277426 23526 raft_consensus.cc:3054] T 11aa76a3a30643708fececed660a5250 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:42.283378 23526 raft_consensus.cc:513] T 11aa76a3a30643708fececed660a5250 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:42.285148 23526 leader_election.cc:290] T 11aa76a3a30643708fececed660a5250 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 2 election: Requested vote from peers ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365), cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551)
I20250109 08:58:42.288720 23372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11aa76a3a30643708fececed660a5250" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c"
I20250109 08:58:42.288976 23446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11aa76a3a30643708fececed660a5250" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc"
I20250109 08:58:42.289340 23372 raft_consensus.cc:3049] T 11aa76a3a30643708fececed660a5250 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 LEADER]: Stepping down as leader of term 1
I20250109 08:58:42.289567 23446 raft_consensus.cc:3054] T 11aa76a3a30643708fececed660a5250 P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:42.289783 23372 raft_consensus.cc:738] T 11aa76a3a30643708fececed660a5250 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ed1efbce9ee3401e99a5260edeae4e7c, State: Running, Role: LEADER
I20250109 08:58:42.290573 23372 consensus_queue.cc:260] T 11aa76a3a30643708fececed660a5250 P ed1efbce9ee3401e99a5260edeae4e7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:42.291872 23372 raft_consensus.cc:3054] T 11aa76a3a30643708fececed660a5250 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:42.298756 23446 raft_consensus.cc:2463] T 11aa76a3a30643708fececed660a5250 P cf55383bef6649eb9e78cb2d2512dedc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05a74e1eb2474b3d8b3a2da1ef127e01 in term 2.
I20250109 08:58:42.303418 23494 leader_election.cc:304] T 11aa76a3a30643708fececed660a5250 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01, cf55383bef6649eb9e78cb2d2512dedc; no voters:
I20250109 08:58:42.305243 24361 raft_consensus.cc:2798] T 11aa76a3a30643708fececed660a5250 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:58:42.305783 24361 raft_consensus.cc:695] T 11aa76a3a30643708fececed660a5250 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 LEADER]: Becoming Leader. State: Replica: 05a74e1eb2474b3d8b3a2da1ef127e01, State: Running, Role: LEADER
I20250109 08:58:42.307729 24361 consensus_queue.cc:237] T 11aa76a3a30643708fececed660a5250 P 05a74e1eb2474b3d8b3a2da1ef127e01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:42.305689 23372 raft_consensus.cc:2463] T 11aa76a3a30643708fececed660a5250 P ed1efbce9ee3401e99a5260edeae4e7c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05a74e1eb2474b3d8b3a2da1ef127e01 in term 2.
I20250109 08:58:42.326310 23200 catalog_manager.cc:5526] T 11aa76a3a30643708fececed660a5250 P 05a74e1eb2474b3d8b3a2da1ef127e01 reported cstate change: term changed from 1 to 2, leader changed from ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194) to 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196). New cstate: current_term: 2 leader_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } health_report { overall_health: HEALTHY } } }
I20250109 08:58:42.348846 24353 raft_consensus.cc:988] T 1ac4317290b24209a76d2e19e68ee45b P ed1efbce9ee3401e99a5260edeae4e7c: : Instructing follower cf55383bef6649eb9e78cb2d2512dedc to start an election
I20250109 08:58:42.349385 24317 raft_consensus.cc:1076] T 1ac4317290b24209a76d2e19e68ee45b P ed1efbce9ee3401e99a5260edeae4e7c [term 2 LEADER]: Signalling peer cf55383bef6649eb9e78cb2d2512dedc to start an election
I20250109 08:58:42.350630 23446 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "1ac4317290b24209a76d2e19e68ee45b"
dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc"
from {username='slave'} at 127.0.0.1:58546
I20250109 08:58:42.351052 23446 raft_consensus.cc:491] T 1ac4317290b24209a76d2e19e68ee45b P cf55383bef6649eb9e78cb2d2512dedc [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:58:42.351329 23446 raft_consensus.cc:3054] T 1ac4317290b24209a76d2e19e68ee45b P cf55383bef6649eb9e78cb2d2512dedc [term 2 FOLLOWER]: Advancing to term 3
I20250109 08:58:42.355892 23446 raft_consensus.cc:513] T 1ac4317290b24209a76d2e19e68ee45b P cf55383bef6649eb9e78cb2d2512dedc [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:42.357224 23446 leader_election.cc:290] T 1ac4317290b24209a76d2e19e68ee45b P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 3 election: Requested vote from peers ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365), 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829)
I20250109 08:58:42.358119 23526 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ac4317290b24209a76d2e19e68ee45b" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01"
I20250109 08:58:42.358120 23373 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ac4317290b24209a76d2e19e68ee45b" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c"
I20250109 08:58:42.358888 23373 raft_consensus.cc:3049] T 1ac4317290b24209a76d2e19e68ee45b P ed1efbce9ee3401e99a5260edeae4e7c [term 2 LEADER]: Stepping down as leader of term 2
I20250109 08:58:42.359100 23373 raft_consensus.cc:738] T 1ac4317290b24209a76d2e19e68ee45b P ed1efbce9ee3401e99a5260edeae4e7c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ed1efbce9ee3401e99a5260edeae4e7c, State: Running, Role: LEADER
I20250109 08:58:42.359491 23526 raft_consensus.cc:3054] T 1ac4317290b24209a76d2e19e68ee45b P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 FOLLOWER]: Advancing to term 3
I20250109 08:58:42.359737 23373 consensus_queue.cc:260] T 1ac4317290b24209a76d2e19e68ee45b P ed1efbce9ee3401e99a5260edeae4e7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:42.360934 23373 raft_consensus.cc:3054] T 1ac4317290b24209a76d2e19e68ee45b P ed1efbce9ee3401e99a5260edeae4e7c [term 2 FOLLOWER]: Advancing to term 3
I20250109 08:58:42.365761 23526 raft_consensus.cc:2463] T 1ac4317290b24209a76d2e19e68ee45b P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf55383bef6649eb9e78cb2d2512dedc in term 3.
I20250109 08:58:42.367324 23418 leader_election.cc:304] T 1ac4317290b24209a76d2e19e68ee45b P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01, cf55383bef6649eb9e78cb2d2512dedc; no voters:
I20250109 08:58:42.368167 24351 raft_consensus.cc:2798] T 1ac4317290b24209a76d2e19e68ee45b P cf55383bef6649eb9e78cb2d2512dedc [term 3 FOLLOWER]: Leader election won for term 3
I20250109 08:58:42.369829 24351 raft_consensus.cc:695] T 1ac4317290b24209a76d2e19e68ee45b P cf55383bef6649eb9e78cb2d2512dedc [term 3 LEADER]: Becoming Leader. State: Replica: cf55383bef6649eb9e78cb2d2512dedc, State: Running, Role: LEADER
I20250109 08:58:42.370515 24351 consensus_queue.cc:237] T 1ac4317290b24209a76d2e19e68ee45b P cf55383bef6649eb9e78cb2d2512dedc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:42.370816 23373 raft_consensus.cc:2463] T 1ac4317290b24209a76d2e19e68ee45b P ed1efbce9ee3401e99a5260edeae4e7c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf55383bef6649eb9e78cb2d2512dedc in term 3.
I20250109 08:58:42.381000 23200 catalog_manager.cc:5526] T 1ac4317290b24209a76d2e19e68ee45b P cf55383bef6649eb9e78cb2d2512dedc reported cstate change: term changed from 2 to 3, leader changed from ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194) to cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195). New cstate: current_term: 3 leader_uuid: "cf55383bef6649eb9e78cb2d2512dedc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:42.436127 24362 raft_consensus.cc:988] T 940653b6d55c46439827697b9aadf16b P 05a74e1eb2474b3d8b3a2da1ef127e01: : Instructing follower cf55383bef6649eb9e78cb2d2512dedc to start an election
I20250109 08:58:42.436581 24361 raft_consensus.cc:1076] T 940653b6d55c46439827697b9aadf16b P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 LEADER]: Signalling peer cf55383bef6649eb9e78cb2d2512dedc to start an election
I20250109 08:58:42.437778 23446 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "940653b6d55c46439827697b9aadf16b"
dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc"
from {username='slave'} at 127.0.0.1:58530
I20250109 08:58:42.438180 23446 raft_consensus.cc:491] T 940653b6d55c46439827697b9aadf16b P cf55383bef6649eb9e78cb2d2512dedc [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:58:42.438409 23446 raft_consensus.cc:3054] T 940653b6d55c46439827697b9aadf16b P cf55383bef6649eb9e78cb2d2512dedc [term 2 FOLLOWER]: Advancing to term 3
I20250109 08:58:42.442675 23446 raft_consensus.cc:513] T 940653b6d55c46439827697b9aadf16b P cf55383bef6649eb9e78cb2d2512dedc [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:42.444015 23446 leader_election.cc:290] T 940653b6d55c46439827697b9aadf16b P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 3 election: Requested vote from peers ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365), 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829)
I20250109 08:58:42.444815 23373 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "940653b6d55c46439827697b9aadf16b" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c"
I20250109 08:58:42.444950 23522 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "940653b6d55c46439827697b9aadf16b" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01"
I20250109 08:58:42.445345 23373 raft_consensus.cc:3054] T 940653b6d55c46439827697b9aadf16b P ed1efbce9ee3401e99a5260edeae4e7c [term 2 FOLLOWER]: Advancing to term 3
I20250109 08:58:42.445462 23522 raft_consensus.cc:3049] T 940653b6d55c46439827697b9aadf16b P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 LEADER]: Stepping down as leader of term 2
I20250109 08:58:42.445787 23522 raft_consensus.cc:738] T 940653b6d55c46439827697b9aadf16b P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 05a74e1eb2474b3d8b3a2da1ef127e01, State: Running, Role: LEADER
I20250109 08:58:42.446369 23522 consensus_queue.cc:260] T 940653b6d55c46439827697b9aadf16b P 05a74e1eb2474b3d8b3a2da1ef127e01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:42.447391 23522 raft_consensus.cc:3054] T 940653b6d55c46439827697b9aadf16b P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 FOLLOWER]: Advancing to term 3
I20250109 08:58:42.449687 23373 raft_consensus.cc:2463] T 940653b6d55c46439827697b9aadf16b P ed1efbce9ee3401e99a5260edeae4e7c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf55383bef6649eb9e78cb2d2512dedc in term 3.
I20250109 08:58:42.450646 23420 leader_election.cc:304] T 940653b6d55c46439827697b9aadf16b P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf55383bef6649eb9e78cb2d2512dedc, ed1efbce9ee3401e99a5260edeae4e7c; no voters:
I20250109 08:58:42.451305 24351 raft_consensus.cc:2798] T 940653b6d55c46439827697b9aadf16b P cf55383bef6649eb9e78cb2d2512dedc [term 3 FOLLOWER]: Leader election won for term 3
I20250109 08:58:42.451347 23522 raft_consensus.cc:2463] T 940653b6d55c46439827697b9aadf16b P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf55383bef6649eb9e78cb2d2512dedc in term 3.
I20250109 08:58:42.451995 24351 raft_consensus.cc:695] T 940653b6d55c46439827697b9aadf16b P cf55383bef6649eb9e78cb2d2512dedc [term 3 LEADER]: Becoming Leader. State: Replica: cf55383bef6649eb9e78cb2d2512dedc, State: Running, Role: LEADER
I20250109 08:58:42.453253 24351 consensus_queue.cc:237] T 940653b6d55c46439827697b9aadf16b P cf55383bef6649eb9e78cb2d2512dedc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:42.460947 23200 catalog_manager.cc:5526] T 940653b6d55c46439827697b9aadf16b P cf55383bef6649eb9e78cb2d2512dedc reported cstate change: term changed from 2 to 3, leader changed from 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196) to cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195). New cstate: current_term: 3 leader_uuid: "cf55383bef6649eb9e78cb2d2512dedc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:42.592695 24317 raft_consensus.cc:988] T a686bf4d947c4b638049e6b88c59ae09 P ed1efbce9ee3401e99a5260edeae4e7c: : Instructing follower cf55383bef6649eb9e78cb2d2512dedc to start an election
I20250109 08:58:42.593092 24353 raft_consensus.cc:1076] T a686bf4d947c4b638049e6b88c59ae09 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 LEADER]: Signalling peer cf55383bef6649eb9e78cb2d2512dedc to start an election
I20250109 08:58:42.594288 23446 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "a686bf4d947c4b638049e6b88c59ae09"
dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc"
from {username='slave'} at 127.0.0.1:58546
I20250109 08:58:42.594715 23446 raft_consensus.cc:491] T a686bf4d947c4b638049e6b88c59ae09 P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:58:42.594956 23446 raft_consensus.cc:3054] T a686bf4d947c4b638049e6b88c59ae09 P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:42.601727 23446 raft_consensus.cc:513] T a686bf4d947c4b638049e6b88c59ae09 P cf55383bef6649eb9e78cb2d2512dedc [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:42.603225 23446 leader_election.cc:290] T a686bf4d947c4b638049e6b88c59ae09 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 2 election: Requested vote from peers ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365), 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829)
I20250109 08:58:42.603842 23373 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a686bf4d947c4b638049e6b88c59ae09" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c"
I20250109 08:58:42.604143 23522 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a686bf4d947c4b638049e6b88c59ae09" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01"
I20250109 08:58:42.604561 23373 raft_consensus.cc:3049] T a686bf4d947c4b638049e6b88c59ae09 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 LEADER]: Stepping down as leader of term 1
I20250109 08:58:42.604825 23522 raft_consensus.cc:3054] T a686bf4d947c4b638049e6b88c59ae09 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:42.604921 23373 raft_consensus.cc:738] T a686bf4d947c4b638049e6b88c59ae09 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ed1efbce9ee3401e99a5260edeae4e7c, State: Running, Role: LEADER
I20250109 08:58:42.605741 23373 consensus_queue.cc:260] T a686bf4d947c4b638049e6b88c59ae09 P ed1efbce9ee3401e99a5260edeae4e7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:42.606647 23373 raft_consensus.cc:3054] T a686bf4d947c4b638049e6b88c59ae09 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:42.609498 23522 raft_consensus.cc:2463] T a686bf4d947c4b638049e6b88c59ae09 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf55383bef6649eb9e78cb2d2512dedc in term 2.
I20250109 08:58:42.610414 23418 leader_election.cc:304] T a686bf4d947c4b638049e6b88c59ae09 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01, cf55383bef6649eb9e78cb2d2512dedc; no voters:
I20250109 08:58:42.611070 23373 raft_consensus.cc:2463] T a686bf4d947c4b638049e6b88c59ae09 P ed1efbce9ee3401e99a5260edeae4e7c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf55383bef6649eb9e78cb2d2512dedc in term 2.
I20250109 08:58:42.611256 24351 raft_consensus.cc:2798] T a686bf4d947c4b638049e6b88c59ae09 P cf55383bef6649eb9e78cb2d2512dedc [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:58:42.612496 24351 raft_consensus.cc:695] T a686bf4d947c4b638049e6b88c59ae09 P cf55383bef6649eb9e78cb2d2512dedc [term 2 LEADER]: Becoming Leader. State: Replica: cf55383bef6649eb9e78cb2d2512dedc, State: Running, Role: LEADER
I20250109 08:58:42.613348 24351 consensus_queue.cc:237] T a686bf4d947c4b638049e6b88c59ae09 P cf55383bef6649eb9e78cb2d2512dedc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:42.621562 23200 catalog_manager.cc:5526] T a686bf4d947c4b638049e6b88c59ae09 P cf55383bef6649eb9e78cb2d2512dedc reported cstate change: term changed from 1 to 2, leader changed from ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194) to cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195). New cstate: current_term: 2 leader_uuid: "cf55383bef6649eb9e78cb2d2512dedc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:42.646952 23446 raft_consensus.cc:1270] T 0dab6c23ce444dda9427901a98f07b15 P cf55383bef6649eb9e78cb2d2512dedc [term 3 FOLLOWER]: Refusing update from remote peer 05a74e1eb2474b3d8b3a2da1ef127e01: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20250109 08:58:42.648017 24361 consensus_queue.cc:1035] T 0dab6c23ce444dda9427901a98f07b15 P 05a74e1eb2474b3d8b3a2da1ef127e01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:58:42.651126 23372 raft_consensus.cc:1270] T 0dab6c23ce444dda9427901a98f07b15 P ed1efbce9ee3401e99a5260edeae4e7c [term 3 FOLLOWER]: Refusing update from remote peer 05a74e1eb2474b3d8b3a2da1ef127e01: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20250109 08:58:42.652542 24361 consensus_queue.cc:1035] T 0dab6c23ce444dda9427901a98f07b15 P 05a74e1eb2474b3d8b3a2da1ef127e01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:58:42.863379 23372 raft_consensus.cc:1270] T 1ac4317290b24209a76d2e19e68ee45b P ed1efbce9ee3401e99a5260edeae4e7c [term 3 FOLLOWER]: Refusing update from remote peer cf55383bef6649eb9e78cb2d2512dedc: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20250109 08:58:42.864751 24372 consensus_queue.cc:1035] T 1ac4317290b24209a76d2e19e68ee45b P cf55383bef6649eb9e78cb2d2512dedc [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:58:42.872917 23522 raft_consensus.cc:1270] T 1ac4317290b24209a76d2e19e68ee45b P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 3 FOLLOWER]: Refusing update from remote peer cf55383bef6649eb9e78cb2d2512dedc: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20250109 08:58:42.875221 24320 consensus_queue.cc:1035] T 1ac4317290b24209a76d2e19e68ee45b P cf55383bef6649eb9e78cb2d2512dedc [LEADER]: Connected to new peer: Peer: permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:58:42.917176 23450 raft_consensus.cc:1270] T 11aa76a3a30643708fececed660a5250 P cf55383bef6649eb9e78cb2d2512dedc [term 2 FOLLOWER]: Refusing update from remote peer 05a74e1eb2474b3d8b3a2da1ef127e01: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:58:42.918171 24319 consensus_queue.cc:1035] T 11aa76a3a30643708fececed660a5250 P 05a74e1eb2474b3d8b3a2da1ef127e01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:58:42.925127 23372 raft_consensus.cc:1270] T 11aa76a3a30643708fececed660a5250 P ed1efbce9ee3401e99a5260edeae4e7c [term 2 FOLLOWER]: Refusing update from remote peer 05a74e1eb2474b3d8b3a2da1ef127e01: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:58:42.925127 23522 raft_consensus.cc:1270] T 940653b6d55c46439827697b9aadf16b P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 3 FOLLOWER]: Refusing update from remote peer cf55383bef6649eb9e78cb2d2512dedc: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20250109 08:58:42.926712 24319 consensus_queue.cc:1035] T 11aa76a3a30643708fececed660a5250 P 05a74e1eb2474b3d8b3a2da1ef127e01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:58:42.927114 24320 consensus_queue.cc:1035] T 940653b6d55c46439827697b9aadf16b P cf55383bef6649eb9e78cb2d2512dedc [LEADER]: Connected to new peer: Peer: permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:58:42.937618 23372 raft_consensus.cc:1270] T 940653b6d55c46439827697b9aadf16b P ed1efbce9ee3401e99a5260edeae4e7c [term 3 FOLLOWER]: Refusing update from remote peer cf55383bef6649eb9e78cb2d2512dedc: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20250109 08:58:42.939002 24372 consensus_queue.cc:1035] T 940653b6d55c46439827697b9aadf16b P cf55383bef6649eb9e78cb2d2512dedc [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:58:43.035630 23522 raft_consensus.cc:1270] T a686bf4d947c4b638049e6b88c59ae09 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 FOLLOWER]: Refusing update from remote peer cf55383bef6649eb9e78cb2d2512dedc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:58:43.036787 24351 consensus_queue.cc:1035] T a686bf4d947c4b638049e6b88c59ae09 P cf55383bef6649eb9e78cb2d2512dedc [LEADER]: Connected to new peer: Peer: permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:58:43.045055 23373 raft_consensus.cc:1270] T a686bf4d947c4b638049e6b88c59ae09 P ed1efbce9ee3401e99a5260edeae4e7c [term 2 FOLLOWER]: Refusing update from remote peer cf55383bef6649eb9e78cb2d2512dedc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:58:43.046892 24372 consensus_queue.cc:1035] T a686bf4d947c4b638049e6b88c59ae09 P cf55383bef6649eb9e78cb2d2512dedc [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:58:43.152781 20363 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250109 08:58:43.189402 23450 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "1ac4317290b24209a76d2e19e68ee45b"
dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc"
mode: GRACEFUL
new_leader_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01"
from {username='slave'} at 127.0.0.1:33404
I20250109 08:58:43.189896 23450 raft_consensus.cc:604] T 1ac4317290b24209a76d2e19e68ee45b P cf55383bef6649eb9e78cb2d2512dedc [term 3 LEADER]: Received request to transfer leadership to TS 05a74e1eb2474b3d8b3a2da1ef127e01
I20250109 08:58:43.191859 23522 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "41e7f0cc38c4414ab88c92c1cdeb3984"
dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01"
mode: GRACEFUL
new_leader_uuid: "235ad6af04c34c88a5261daa3604fada"
from {username='slave'} at 127.0.0.1:45238
I20250109 08:58:43.192310 23522 raft_consensus.cc:604] T 41e7f0cc38c4414ab88c92c1cdeb3984 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 LEADER]: Received request to transfer leadership to TS 235ad6af04c34c88a5261daa3604fada
I20250109 08:58:43.194310 23373 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "6eef9bb3c645428d9f1cb843bae9a264"
dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c"
mode: GRACEFUL
new_leader_uuid: "235ad6af04c34c88a5261daa3604fada"
from {username='slave'} at 127.0.0.1:51604
I20250109 08:58:43.194726 23373 raft_consensus.cc:604] T 6eef9bb3c645428d9f1cb843bae9a264 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 LEADER]: Received request to transfer leadership to TS 235ad6af04c34c88a5261daa3604fada
I20250109 08:58:43.196601 23522 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "950e07e9cc794b8ab9ec5bd97d7e8e59"
dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01"
mode: GRACEFUL
new_leader_uuid: "235ad6af04c34c88a5261daa3604fada"
from {username='slave'} at 127.0.0.1:45238
I20250109 08:58:43.196995 23522 raft_consensus.cc:604] T 950e07e9cc794b8ab9ec5bd97d7e8e59 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 LEADER]: Received request to transfer leadership to TS 235ad6af04c34c88a5261daa3604fada
I20250109 08:58:43.198786 23450 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "bd6fcd838b0249cabd5f1f95e5538ab6"
dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc"
mode: GRACEFUL
new_leader_uuid: "235ad6af04c34c88a5261daa3604fada"
from {username='slave'} at 127.0.0.1:33404
I20250109 08:58:43.199275 23450 raft_consensus.cc:604] T bd6fcd838b0249cabd5f1f95e5538ab6 P cf55383bef6649eb9e78cb2d2512dedc [term 1 LEADER]: Received request to transfer leadership to TS 235ad6af04c34c88a5261daa3604fada
I20250109 08:58:43.201108 23450 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "bfb140bb3a8b44bd8b4e1fcf0946162b"
dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc"
mode: GRACEFUL
new_leader_uuid: "235ad6af04c34c88a5261daa3604fada"
from {username='slave'} at 127.0.0.1:33404
I20250109 08:58:43.201460 23450 raft_consensus.cc:604] T bfb140bb3a8b44bd8b4e1fcf0946162b P cf55383bef6649eb9e78cb2d2512dedc [term 1 LEADER]: Received request to transfer leadership to TS 235ad6af04c34c88a5261daa3604fada
I20250109 08:58:43.202293 20363 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 6
I20250109 08:58:43.203372 20363 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250109 08:58:43.203815 20363 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250109 08:58:43.319564 24318 raft_consensus.cc:988] T 6eef9bb3c645428d9f1cb843bae9a264 P ed1efbce9ee3401e99a5260edeae4e7c: : Instructing follower 235ad6af04c34c88a5261daa3604fada to start an election
I20250109 08:58:43.319993 24317 raft_consensus.cc:1076] T 6eef9bb3c645428d9f1cb843bae9a264 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 LEADER]: Signalling peer 235ad6af04c34c88a5261daa3604fada to start an election
I20250109 08:58:43.321308 24156 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6eef9bb3c645428d9f1cb843bae9a264"
dest_uuid: "235ad6af04c34c88a5261daa3604fada"
from {username='slave'} at 127.0.0.1:43328
I20250109 08:58:43.321738 24156 raft_consensus.cc:491] T 6eef9bb3c645428d9f1cb843bae9a264 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:58:43.321981 24156 raft_consensus.cc:3054] T 6eef9bb3c645428d9f1cb843bae9a264 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:43.327003 24156 raft_consensus.cc:513] T 6eef9bb3c645428d9f1cb843bae9a264 P 235ad6af04c34c88a5261daa3604fada [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:43.328402 24156 leader_election.cc:290] T 6eef9bb3c645428d9f1cb843bae9a264 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 2 election: Requested vote from peers ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365), cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551)
I20250109 08:58:43.329107 23373 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6eef9bb3c645428d9f1cb843bae9a264" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c"
I20250109 08:58:43.329393 23446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6eef9bb3c645428d9f1cb843bae9a264" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc"
I20250109 08:58:43.329641 23373 raft_consensus.cc:3049] T 6eef9bb3c645428d9f1cb843bae9a264 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 LEADER]: Stepping down as leader of term 1
I20250109 08:58:43.329892 23446 raft_consensus.cc:3054] T 6eef9bb3c645428d9f1cb843bae9a264 P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:43.329933 23373 raft_consensus.cc:738] T 6eef9bb3c645428d9f1cb843bae9a264 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ed1efbce9ee3401e99a5260edeae4e7c, State: Running, Role: LEADER
I20250109 08:58:43.330678 23373 consensus_queue.cc:260] T 6eef9bb3c645428d9f1cb843bae9a264 P ed1efbce9ee3401e99a5260edeae4e7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:43.331549 23373 raft_consensus.cc:3054] T 6eef9bb3c645428d9f1cb843bae9a264 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:43.334842 23446 raft_consensus.cc:2463] T 6eef9bb3c645428d9f1cb843bae9a264 P cf55383bef6649eb9e78cb2d2512dedc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 235ad6af04c34c88a5261daa3604fada in term 2.
I20250109 08:58:43.336030 24112 leader_election.cc:304] T 6eef9bb3c645428d9f1cb843bae9a264 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 235ad6af04c34c88a5261daa3604fada, cf55383bef6649eb9e78cb2d2512dedc; no voters:
I20250109 08:58:43.336858 24334 raft_consensus.cc:2798] T 6eef9bb3c645428d9f1cb843bae9a264 P 235ad6af04c34c88a5261daa3604fada [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:58:43.337198 23373 raft_consensus.cc:2463] T 6eef9bb3c645428d9f1cb843bae9a264 P ed1efbce9ee3401e99a5260edeae4e7c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 235ad6af04c34c88a5261daa3604fada in term 2.
I20250109 08:58:43.337786 24334 raft_consensus.cc:695] T 6eef9bb3c645428d9f1cb843bae9a264 P 235ad6af04c34c88a5261daa3604fada [term 2 LEADER]: Becoming Leader. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Running, Role: LEADER
I20250109 08:58:43.338665 24334 consensus_queue.cc:237] T 6eef9bb3c645428d9f1cb843bae9a264 P 235ad6af04c34c88a5261daa3604fada [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:43.349169 23201 catalog_manager.cc:5526] T 6eef9bb3c645428d9f1cb843bae9a264 P 235ad6af04c34c88a5261daa3604fada reported cstate change: term changed from 1 to 2, leader changed from ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194) to 235ad6af04c34c88a5261daa3604fada (127.19.226.193). New cstate: current_term: 2 leader_uuid: "235ad6af04c34c88a5261daa3604fada" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:43.365556 24351 raft_consensus.cc:988] T bd6fcd838b0249cabd5f1f95e5538ab6 P cf55383bef6649eb9e78cb2d2512dedc: : Instructing follower 235ad6af04c34c88a5261daa3604fada to start an election
I20250109 08:58:43.366039 24351 raft_consensus.cc:1076] T bd6fcd838b0249cabd5f1f95e5538ab6 P cf55383bef6649eb9e78cb2d2512dedc [term 1 LEADER]: Signalling peer 235ad6af04c34c88a5261daa3604fada to start an election
I20250109 08:58:43.367854 24156 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "bd6fcd838b0249cabd5f1f95e5538ab6"
dest_uuid: "235ad6af04c34c88a5261daa3604fada"
from {username='slave'} at 127.0.0.1:43338
I20250109 08:58:43.374186 24156 raft_consensus.cc:491] T bd6fcd838b0249cabd5f1f95e5538ab6 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:58:43.374630 24156 raft_consensus.cc:3054] T bd6fcd838b0249cabd5f1f95e5538ab6 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:43.377028 24351 raft_consensus.cc:988] T 1ac4317290b24209a76d2e19e68ee45b P cf55383bef6649eb9e78cb2d2512dedc: : Instructing follower 05a74e1eb2474b3d8b3a2da1ef127e01 to start an election
I20250109 08:58:43.377632 24351 raft_consensus.cc:1076] T 1ac4317290b24209a76d2e19e68ee45b P cf55383bef6649eb9e78cb2d2512dedc [term 3 LEADER]: Signalling peer 05a74e1eb2474b3d8b3a2da1ef127e01 to start an election
I20250109 08:58:43.379002 23522 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "1ac4317290b24209a76d2e19e68ee45b"
dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01"
from {username='slave'} at 127.0.0.1:58988
I20250109 08:58:43.379509 23522 raft_consensus.cc:491] T 1ac4317290b24209a76d2e19e68ee45b P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:58:43.379832 23522 raft_consensus.cc:3054] T 1ac4317290b24209a76d2e19e68ee45b P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 3 FOLLOWER]: Advancing to term 4
I20250109 08:58:43.386119 23522 raft_consensus.cc:513] T 1ac4317290b24209a76d2e19e68ee45b P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:43.387910 23522 leader_election.cc:290] T 1ac4317290b24209a76d2e19e68ee45b P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 4 election: Requested vote from peers ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365), cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551)
I20250109 08:58:43.388742 23446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ac4317290b24209a76d2e19e68ee45b" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc"
I20250109 08:58:43.388794 23373 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ac4317290b24209a76d2e19e68ee45b" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c"
I20250109 08:58:43.389343 23446 raft_consensus.cc:3049] T 1ac4317290b24209a76d2e19e68ee45b P cf55383bef6649eb9e78cb2d2512dedc [term 3 LEADER]: Stepping down as leader of term 3
I20250109 08:58:43.389485 23373 raft_consensus.cc:3054] T 1ac4317290b24209a76d2e19e68ee45b P ed1efbce9ee3401e99a5260edeae4e7c [term 3 FOLLOWER]: Advancing to term 4
I20250109 08:58:43.389633 23446 raft_consensus.cc:738] T 1ac4317290b24209a76d2e19e68ee45b P cf55383bef6649eb9e78cb2d2512dedc [term 3 LEADER]: Becoming Follower/Learner. State: Replica: cf55383bef6649eb9e78cb2d2512dedc, State: Running, Role: LEADER
I20250109 08:58:43.390323 23446 consensus_queue.cc:260] T 1ac4317290b24209a76d2e19e68ee45b P cf55383bef6649eb9e78cb2d2512dedc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:43.391155 23446 raft_consensus.cc:3054] T 1ac4317290b24209a76d2e19e68ee45b P cf55383bef6649eb9e78cb2d2512dedc [term 3 FOLLOWER]: Advancing to term 4
I20250109 08:58:43.392549 24156 raft_consensus.cc:513] T bd6fcd838b0249cabd5f1f95e5538ab6 P 235ad6af04c34c88a5261daa3604fada [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:43.395430 24156 leader_election.cc:290] T bd6fcd838b0249cabd5f1f95e5538ab6 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 2 election: Requested vote from peers 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829), cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551)
I20250109 08:58:43.395757 23453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd6fcd838b0249cabd5f1f95e5538ab6" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc"
I20250109 08:58:43.396337 23453 raft_consensus.cc:3049] T bd6fcd838b0249cabd5f1f95e5538ab6 P cf55383bef6649eb9e78cb2d2512dedc [term 1 LEADER]: Stepping down as leader of term 1
I20250109 08:58:43.396638 23453 raft_consensus.cc:738] T bd6fcd838b0249cabd5f1f95e5538ab6 P cf55383bef6649eb9e78cb2d2512dedc [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cf55383bef6649eb9e78cb2d2512dedc, State: Running, Role: LEADER
I20250109 08:58:43.397353 23453 consensus_queue.cc:260] T bd6fcd838b0249cabd5f1f95e5538ab6 P cf55383bef6649eb9e78cb2d2512dedc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:43.398625 23453 raft_consensus.cc:3054] T bd6fcd838b0249cabd5f1f95e5538ab6 P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:43.398880 23373 raft_consensus.cc:2463] T 1ac4317290b24209a76d2e19e68ee45b P ed1efbce9ee3401e99a5260edeae4e7c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05a74e1eb2474b3d8b3a2da1ef127e01 in term 4.
I20250109 08:58:43.399641 23522 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd6fcd838b0249cabd5f1f95e5538ab6" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01"
I20250109 08:58:43.400170 23494 leader_election.cc:304] T 1ac4317290b24209a76d2e19e68ee45b P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01, ed1efbce9ee3401e99a5260edeae4e7c; no voters:
I20250109 08:58:43.400225 23522 raft_consensus.cc:3054] T bd6fcd838b0249cabd5f1f95e5538ab6 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:43.401091 24381 raft_consensus.cc:2798] T 1ac4317290b24209a76d2e19e68ee45b P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 4 FOLLOWER]: Leader election won for term 4
I20250109 08:58:43.401470 24381 raft_consensus.cc:695] T 1ac4317290b24209a76d2e19e68ee45b P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 4 LEADER]: Becoming Leader. State: Replica: 05a74e1eb2474b3d8b3a2da1ef127e01, State: Running, Role: LEADER
I20250109 08:58:43.402197 24381 consensus_queue.cc:237] T 1ac4317290b24209a76d2e19e68ee45b P 05a74e1eb2474b3d8b3a2da1ef127e01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:43.408116 23522 raft_consensus.cc:2463] T bd6fcd838b0249cabd5f1f95e5538ab6 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 235ad6af04c34c88a5261daa3604fada in term 2.
I20250109 08:58:43.400399 23446 raft_consensus.cc:2463] T 1ac4317290b24209a76d2e19e68ee45b P cf55383bef6649eb9e78cb2d2512dedc [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05a74e1eb2474b3d8b3a2da1ef127e01 in term 4.
I20250109 08:58:43.409415 23453 raft_consensus.cc:2463] T bd6fcd838b0249cabd5f1f95e5538ab6 P cf55383bef6649eb9e78cb2d2512dedc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 235ad6af04c34c88a5261daa3604fada in term 2.
I20250109 08:58:43.409781 24110 leader_election.cc:304] T bd6fcd838b0249cabd5f1f95e5538ab6 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01, 235ad6af04c34c88a5261daa3604fada; no voters:
I20250109 08:58:43.411181 24334 raft_consensus.cc:2798] T bd6fcd838b0249cabd5f1f95e5538ab6 P 235ad6af04c34c88a5261daa3604fada [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:58:43.411888 23201 catalog_manager.cc:5526] T 1ac4317290b24209a76d2e19e68ee45b P 05a74e1eb2474b3d8b3a2da1ef127e01 reported cstate change: term changed from 3 to 4, leader changed from cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195) to 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196). New cstate: current_term: 4 leader_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } health_report { overall_health: HEALTHY } } }
I20250109 08:58:43.413162 24334 raft_consensus.cc:695] T bd6fcd838b0249cabd5f1f95e5538ab6 P 235ad6af04c34c88a5261daa3604fada [term 2 LEADER]: Becoming Leader. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Running, Role: LEADER
I20250109 08:58:43.414117 24334 consensus_queue.cc:237] T bd6fcd838b0249cabd5f1f95e5538ab6 P 235ad6af04c34c88a5261daa3604fada [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:43.426904 23201 catalog_manager.cc:5526] T bd6fcd838b0249cabd5f1f95e5538ab6 P 235ad6af04c34c88a5261daa3604fada reported cstate change: term changed from 1 to 2, leader changed from cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195) to 235ad6af04c34c88a5261daa3604fada (127.19.226.193). New cstate: current_term: 2 leader_uuid: "235ad6af04c34c88a5261daa3604fada" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:43.513947 24372 raft_consensus.cc:988] T bfb140bb3a8b44bd8b4e1fcf0946162b P cf55383bef6649eb9e78cb2d2512dedc: : Instructing follower 235ad6af04c34c88a5261daa3604fada to start an election
I20250109 08:58:43.514405 24351 raft_consensus.cc:1076] T bfb140bb3a8b44bd8b4e1fcf0946162b P cf55383bef6649eb9e78cb2d2512dedc [term 1 LEADER]: Signalling peer 235ad6af04c34c88a5261daa3604fada to start an election
I20250109 08:58:43.515650 24156 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "bfb140bb3a8b44bd8b4e1fcf0946162b"
dest_uuid: "235ad6af04c34c88a5261daa3604fada"
from {username='slave'} at 127.0.0.1:43338
I20250109 08:58:43.516079 24156 raft_consensus.cc:491] T bfb140bb3a8b44bd8b4e1fcf0946162b P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:58:43.516324 24156 raft_consensus.cc:3054] T bfb140bb3a8b44bd8b4e1fcf0946162b P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:43.520792 24156 raft_consensus.cc:513] T bfb140bb3a8b44bd8b4e1fcf0946162b P 235ad6af04c34c88a5261daa3604fada [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:43.522096 24156 leader_election.cc:290] T bfb140bb3a8b44bd8b4e1fcf0946162b P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 2 election: Requested vote from peers cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551), 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829)
I20250109 08:58:43.522894 23446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfb140bb3a8b44bd8b4e1fcf0946162b" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc"
I20250109 08:58:43.523020 23522 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfb140bb3a8b44bd8b4e1fcf0946162b" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01"
I20250109 08:58:43.523553 23446 raft_consensus.cc:3049] T bfb140bb3a8b44bd8b4e1fcf0946162b P cf55383bef6649eb9e78cb2d2512dedc [term 1 LEADER]: Stepping down as leader of term 1
I20250109 08:58:43.523950 23522 raft_consensus.cc:3054] T bfb140bb3a8b44bd8b4e1fcf0946162b P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:43.524124 23446 raft_consensus.cc:738] T bfb140bb3a8b44bd8b4e1fcf0946162b P cf55383bef6649eb9e78cb2d2512dedc [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cf55383bef6649eb9e78cb2d2512dedc, State: Running, Role: LEADER
I20250109 08:58:43.525014 23446 consensus_queue.cc:260] T bfb140bb3a8b44bd8b4e1fcf0946162b P cf55383bef6649eb9e78cb2d2512dedc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:43.526297 23446 raft_consensus.cc:3054] T bfb140bb3a8b44bd8b4e1fcf0946162b P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:43.532769 23522 raft_consensus.cc:2463] T bfb140bb3a8b44bd8b4e1fcf0946162b P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 235ad6af04c34c88a5261daa3604fada in term 2.
I20250109 08:58:43.533143 23446 raft_consensus.cc:2463] T bfb140bb3a8b44bd8b4e1fcf0946162b P cf55383bef6649eb9e78cb2d2512dedc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 235ad6af04c34c88a5261daa3604fada in term 2.
I20250109 08:58:43.533732 24110 leader_election.cc:304] T bfb140bb3a8b44bd8b4e1fcf0946162b P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01, 235ad6af04c34c88a5261daa3604fada; no voters:
I20250109 08:58:43.534572 24334 raft_consensus.cc:2798] T bfb140bb3a8b44bd8b4e1fcf0946162b P 235ad6af04c34c88a5261daa3604fada [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:58:43.535041 24334 raft_consensus.cc:695] T bfb140bb3a8b44bd8b4e1fcf0946162b P 235ad6af04c34c88a5261daa3604fada [term 2 LEADER]: Becoming Leader. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Running, Role: LEADER
I20250109 08:58:43.535816 24334 consensus_queue.cc:237] T bfb140bb3a8b44bd8b4e1fcf0946162b P 235ad6af04c34c88a5261daa3604fada [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:43.544613 23201 catalog_manager.cc:5526] T bfb140bb3a8b44bd8b4e1fcf0946162b P 235ad6af04c34c88a5261daa3604fada reported cstate change: term changed from 1 to 2, leader changed from cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195) to 235ad6af04c34c88a5261daa3604fada (127.19.226.193). New cstate: current_term: 2 leader_uuid: "235ad6af04c34c88a5261daa3604fada" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } health_report { overall_health: HEALTHY } } }
I20250109 08:58:43.555949 24319 raft_consensus.cc:988] T 950e07e9cc794b8ab9ec5bd97d7e8e59 P 05a74e1eb2474b3d8b3a2da1ef127e01: : Instructing follower 235ad6af04c34c88a5261daa3604fada to start an election
I20250109 08:58:43.556347 24381 raft_consensus.cc:1076] T 950e07e9cc794b8ab9ec5bd97d7e8e59 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 LEADER]: Signalling peer 235ad6af04c34c88a5261daa3604fada to start an election
I20250109 08:58:43.557582 24156 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "950e07e9cc794b8ab9ec5bd97d7e8e59"
dest_uuid: "235ad6af04c34c88a5261daa3604fada"
from {username='slave'} at 127.0.0.1:43346
I20250109 08:58:43.558003 24156 raft_consensus.cc:491] T 950e07e9cc794b8ab9ec5bd97d7e8e59 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:58:43.558243 24156 raft_consensus.cc:3054] T 950e07e9cc794b8ab9ec5bd97d7e8e59 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:43.563408 24156 raft_consensus.cc:513] T 950e07e9cc794b8ab9ec5bd97d7e8e59 P 235ad6af04c34c88a5261daa3604fada [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:43.564790 24156 leader_election.cc:290] T 950e07e9cc794b8ab9ec5bd97d7e8e59 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 2 election: Requested vote from peers cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551), 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829)
I20250109 08:58:43.565434 23446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "950e07e9cc794b8ab9ec5bd97d7e8e59" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc"
I20250109 08:58:43.565788 23522 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "950e07e9cc794b8ab9ec5bd97d7e8e59" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01"
I20250109 08:58:43.565980 23446 raft_consensus.cc:3054] T 950e07e9cc794b8ab9ec5bd97d7e8e59 P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:43.566293 23522 raft_consensus.cc:3049] T 950e07e9cc794b8ab9ec5bd97d7e8e59 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 LEADER]: Stepping down as leader of term 1
I20250109 08:58:43.566533 23522 raft_consensus.cc:738] T 950e07e9cc794b8ab9ec5bd97d7e8e59 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 05a74e1eb2474b3d8b3a2da1ef127e01, State: Running, Role: LEADER
I20250109 08:58:43.567224 23522 consensus_queue.cc:260] T 950e07e9cc794b8ab9ec5bd97d7e8e59 P 05a74e1eb2474b3d8b3a2da1ef127e01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:43.568301 23522 raft_consensus.cc:3054] T 950e07e9cc794b8ab9ec5bd97d7e8e59 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:43.571297 23446 raft_consensus.cc:2463] T 950e07e9cc794b8ab9ec5bd97d7e8e59 P cf55383bef6649eb9e78cb2d2512dedc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 235ad6af04c34c88a5261daa3604fada in term 2.
I20250109 08:58:43.572328 24112 leader_election.cc:304] T 950e07e9cc794b8ab9ec5bd97d7e8e59 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 235ad6af04c34c88a5261daa3604fada, cf55383bef6649eb9e78cb2d2512dedc; no voters:
I20250109 08:58:43.574074 24334 raft_consensus.cc:2798] T 950e07e9cc794b8ab9ec5bd97d7e8e59 P 235ad6af04c34c88a5261daa3604fada [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:58:43.574642 24334 raft_consensus.cc:695] T 950e07e9cc794b8ab9ec5bd97d7e8e59 P 235ad6af04c34c88a5261daa3604fada [term 2 LEADER]: Becoming Leader. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Running, Role: LEADER
I20250109 08:58:43.574414 23522 raft_consensus.cc:2463] T 950e07e9cc794b8ab9ec5bd97d7e8e59 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 235ad6af04c34c88a5261daa3604fada in term 2.
I20250109 08:58:43.575775 24334 consensus_queue.cc:237] T 950e07e9cc794b8ab9ec5bd97d7e8e59 P 235ad6af04c34c88a5261daa3604fada [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:44.416361 23372 raft_consensus.cc:1270] T 6eef9bb3c645428d9f1cb843bae9a264 P ed1efbce9ee3401e99a5260edeae4e7c [term 2 FOLLOWER]: Refusing update from remote peer 235ad6af04c34c88a5261daa3604fada: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:58:44.425369 24334 consensus_queue.cc:1035] T 6eef9bb3c645428d9f1cb843bae9a264 P 235ad6af04c34c88a5261daa3604fada [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20250109 08:58:44.470084 24112 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.226.194:41365, user_credentials={real_user=slave}} blocked reactor thread for 46713.9us
W20250109 08:58:44.544909 23494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.226.195:35551, user_credentials={real_user=slave}} blocked reactor thread for 103336us
I20250109 08:58:44.562974 24413 raft_consensus.cc:491] T db111e41c1be497e8e673a2c42b9ba66 P cf55383bef6649eb9e78cb2d2512dedc [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ed1efbce9ee3401e99a5260edeae4e7c)
I20250109 08:58:44.576217 24413 raft_consensus.cc:513] T db111e41c1be497e8e673a2c42b9ba66 P cf55383bef6649eb9e78cb2d2512dedc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:44.581843 24413 leader_election.cc:290] T db111e41c1be497e8e673a2c42b9ba66 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691), ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365)
I20250109 08:58:44.586632 23374 raft_consensus.cc:1270] T 1ac4317290b24209a76d2e19e68ee45b P ed1efbce9ee3401e99a5260edeae4e7c [term 4 FOLLOWER]: Refusing update from remote peer 05a74e1eb2474b3d8b3a2da1ef127e01: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20250109 08:58:44.602597 24425 raft_consensus.cc:491] T 1c5b5beea4a840eb9677097cacee3408 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ed1efbce9ee3401e99a5260edeae4e7c)
I20250109 08:58:44.603175 24425 raft_consensus.cc:513] T 1c5b5beea4a840eb9677097cacee3408 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:44.605232 24425 leader_election.cc:290] T 1c5b5beea4a840eb9677097cacee3408 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691), ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365)
I20250109 08:58:44.606300 24414 consensus_queue.cc:1035] T 1ac4317290b24209a76d2e19e68ee45b P 05a74e1eb2474b3d8b3a2da1ef127e01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:58:44.603684 23201 catalog_manager.cc:5526] T 950e07e9cc794b8ab9ec5bd97d7e8e59 P 235ad6af04c34c88a5261daa3604fada reported cstate change: term changed from 1 to 2, leader changed from 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196) to 235ad6af04c34c88a5261daa3604fada (127.19.226.193). New cstate: current_term: 2 leader_uuid: "235ad6af04c34c88a5261daa3604fada" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } health_report { overall_health: HEALTHY } } }
I20250109 08:58:44.611861 23374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5b5beea4a840eb9677097cacee3408" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" is_pre_election: true
I20250109 08:58:44.618048 24153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5b5beea4a840eb9677097cacee3408" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada" is_pre_election: true
I20250109 08:58:44.637919 24425 raft_consensus.cc:491] T 9b0bbf2e4a424fb5ac13a3b55627b048 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ed1efbce9ee3401e99a5260edeae4e7c)
I20250109 08:58:44.641929 24425 raft_consensus.cc:513] T 9b0bbf2e4a424fb5ac13a3b55627b048 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:44.643971 24425 leader_election.cc:290] T 9b0bbf2e4a424fb5ac13a3b55627b048 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365), cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551)
I20250109 08:58:44.651417 24412 raft_consensus.cc:491] T db111e41c1be497e8e673a2c42b9ba66 P 235ad6af04c34c88a5261daa3604fada [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ed1efbce9ee3401e99a5260edeae4e7c)
I20250109 08:58:44.652135 24412 raft_consensus.cc:513] T db111e41c1be497e8e673a2c42b9ba66 P 235ad6af04c34c88a5261daa3604fada [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:44.654539 24412 leader_election.cc:290] T db111e41c1be497e8e673a2c42b9ba66 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365), cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551)
I20250109 08:58:44.656710 23376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db111e41c1be497e8e673a2c42b9ba66" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" is_pre_election: true
I20250109 08:58:44.660703 24402 raft_consensus.cc:491] T 5831288451724d4ca1114ec5e5f01b78 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cf55383bef6649eb9e78cb2d2512dedc)
I20250109 08:58:44.661295 24402 raft_consensus.cc:513] T 5831288451724d4ca1114ec5e5f01b78 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } }
I20250109 08:58:44.664021 24402 leader_election.cc:290] T 5831288451724d4ca1114ec5e5f01b78 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551), 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829)
I20250109 08:58:44.666719 23446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5831288451724d4ca1114ec5e5f01b78" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc" is_pre_election: true
I20250109 08:58:44.682408 23525 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5831288451724d4ca1114ec5e5f01b78" candidate_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" is_pre_election: true
I20250109 08:58:44.681860 23494 leader_election.cc:304] T 1c5b5beea4a840eb9677097cacee3408 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01; no voters: 235ad6af04c34c88a5261daa3604fada, ed1efbce9ee3401e99a5260edeae4e7c
I20250109 08:58:44.684825 23372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b0bbf2e4a424fb5ac13a3b55627b048" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" is_pre_election: true
I20250109 08:58:44.685626 24432 raft_consensus.cc:2743] T 1c5b5beea4a840eb9677097cacee3408 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250109 08:58:44.695631 23525 raft_consensus.cc:1270] T bd6fcd838b0249cabd5f1f95e5538ab6 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 FOLLOWER]: Refusing update from remote peer 235ad6af04c34c88a5261daa3604fada: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:58:44.696857 23344 leader_election.cc:304] T 5831288451724d4ca1114ec5e5f01b78 P ed1efbce9ee3401e99a5260edeae4e7c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ed1efbce9ee3401e99a5260edeae4e7c; no voters: 05a74e1eb2474b3d8b3a2da1ef127e01, cf55383bef6649eb9e78cb2d2512dedc
I20250109 08:58:44.700412 24409 raft_consensus.cc:2743] T 5831288451724d4ca1114ec5e5f01b78 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250109 08:58:44.703437 24412 raft_consensus.cc:491] T 1365130ef97c4581b623c22c39753831 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ed1efbce9ee3401e99a5260edeae4e7c)
I20250109 08:58:44.704157 24412 raft_consensus.cc:513] T 1365130ef97c4581b623c22c39753831 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:44.713778 24412 leader_election.cc:290] T 1365130ef97c4581b623c22c39753831 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365), cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551)
I20250109 08:58:44.714347 24415 raft_consensus.cc:491] T 4df64aae69ca41dc91286cae1603000f P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ed1efbce9ee3401e99a5260edeae4e7c)
I20250109 08:58:44.714609 23374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1365130ef97c4581b623c22c39753831" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" is_pre_election: true
I20250109 08:58:44.714956 24415 raft_consensus.cc:513] T 4df64aae69ca41dc91286cae1603000f P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:44.717868 24415 leader_election.cc:290] T 4df64aae69ca41dc91286cae1603000f P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691), ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365)
I20250109 08:58:44.726467 23521 raft_consensus.cc:1270] T bfb140bb3a8b44bd8b4e1fcf0946162b P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 FOLLOWER]: Refusing update from remote peer 235ad6af04c34c88a5261daa3604fada: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:58:44.726588 24392 raft_consensus.cc:491] T ed79047393d2425f8ecff31ea08b9cad P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ed1efbce9ee3401e99a5260edeae4e7c)
I20250109 08:58:44.732067 24392 raft_consensus.cc:513] T ed79047393d2425f8ecff31ea08b9cad P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:44.734190 24392 leader_election.cc:290] T ed79047393d2425f8ecff31ea08b9cad P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829), ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365)
I20250109 08:58:44.743693 24156 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db111e41c1be497e8e673a2c42b9ba66" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada" is_pre_election: true
I20250109 08:58:44.748198 23520 raft_consensus.cc:1270] T 950e07e9cc794b8ab9ec5bd97d7e8e59 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 FOLLOWER]: Refusing update from remote peer 235ad6af04c34c88a5261daa3604fada: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:58:44.761466 24152 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4df64aae69ca41dc91286cae1603000f" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "235ad6af04c34c88a5261daa3604fada" is_pre_election: true
I20250109 08:58:44.763931 24392 raft_consensus.cc:491] T e5251927614642639bd46ecd5373d2a4 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cf55383bef6649eb9e78cb2d2512dedc)
I20250109 08:58:44.764551 24392 raft_consensus.cc:513] T e5251927614642639bd46ecd5373d2a4 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:44.764748 24387 consensus_queue.cc:1035] T bd6fcd838b0249cabd5f1f95e5538ab6 P 235ad6af04c34c88a5261daa3604fada [LEADER]: Connected to new peer: Peer: permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20250109 08:58:44.766569 24392 leader_election.cc:290] T e5251927614642639bd46ecd5373d2a4 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551), 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829)
I20250109 08:58:44.769307 24334 consensus_queue.cc:1035] T bfb140bb3a8b44bd8b4e1fcf0946162b P 235ad6af04c34c88a5261daa3604fada [LEADER]: Connected to new peer: Peer: permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:58:44.773676 23376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db111e41c1be497e8e673a2c42b9ba66" candidate_uuid: "cf55383bef6649eb9e78cb2d2512dedc" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" is_pre_election: true
I20250109 08:58:44.779829 23374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed79047393d2425f8ecff31ea08b9cad" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" is_pre_election: true
I20250109 08:58:44.786531 23525 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed79047393d2425f8ecff31ea08b9cad" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" is_pre_election: true
I20250109 08:58:44.782156 23376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4df64aae69ca41dc91286cae1603000f" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" is_pre_election: true
I20250109 08:58:44.794546 23453 raft_consensus.cc:1270] T 1ac4317290b24209a76d2e19e68ee45b P cf55383bef6649eb9e78cb2d2512dedc [term 4 FOLLOWER]: Refusing update from remote peer 05a74e1eb2474b3d8b3a2da1ef127e01: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20250109 08:58:44.799044 23453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b0bbf2e4a424fb5ac13a3b55627b048" candidate_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc" is_pre_election: true
I20250109 08:58:44.808244 23420 leader_election.cc:304] T db111e41c1be497e8e673a2c42b9ba66 P cf55383bef6649eb9e78cb2d2512dedc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf55383bef6649eb9e78cb2d2512dedc; no voters: 235ad6af04c34c88a5261daa3604fada, ed1efbce9ee3401e99a5260edeae4e7c
I20250109 08:58:44.818806 24413 raft_consensus.cc:2743] T db111e41c1be497e8e673a2c42b9ba66 P cf55383bef6649eb9e78cb2d2512dedc [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20250109 08:58:44.810592 23446 raft_consensus.cc:1270] T 6eef9bb3c645428d9f1cb843bae9a264 P cf55383bef6649eb9e78cb2d2512dedc [term 2 FOLLOWER]: Refusing update from remote peer 235ad6af04c34c88a5261daa3604fada: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:58:44.810858 24392 consensus_queue.cc:1035] T 950e07e9cc794b8ab9ec5bd97d7e8e59 P 235ad6af04c34c88a5261daa3604fada [LEADER]: Connected to new peer: Peer: permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:58:44.811656 24110 leader_election.cc:304] T ed79047393d2425f8ecff31ea08b9cad P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 235ad6af04c34c88a5261daa3604fada; no voters: 05a74e1eb2474b3d8b3a2da1ef127e01, ed1efbce9ee3401e99a5260edeae4e7c
I20250109 08:58:44.812175 23450 raft_consensus.cc:1270] T 950e07e9cc794b8ab9ec5bd97d7e8e59 P cf55383bef6649eb9e78cb2d2512dedc [term 2 FOLLOWER]: Refusing update from remote peer 235ad6af04c34c88a5261daa3604fada: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:58:44.813666 23448 raft_consensus.cc:1270] T bfb140bb3a8b44bd8b4e1fcf0946162b P cf55383bef6649eb9e78cb2d2512dedc [term 2 FOLLOWER]: Refusing update from remote peer 235ad6af04c34c88a5261daa3604fada: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:58:44.814741 23447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1365130ef97c4581b623c22c39753831" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc" is_pre_election: true
I20250109 08:58:44.816416 23453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5251927614642639bd46ecd5373d2a4" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc" is_pre_election: true
I20250109 08:58:44.816735 23445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db111e41c1be497e8e673a2c42b9ba66" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc" is_pre_election: true
I20250109 08:58:44.832069 24412 consensus_queue.cc:1035] T bfb140bb3a8b44bd8b4e1fcf0946162b P 235ad6af04c34c88a5261daa3604fada [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:58:44.831954 24392 consensus_queue.cc:1035] T 950e07e9cc794b8ab9ec5bd97d7e8e59 P 235ad6af04c34c88a5261daa3604fada [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20250109 08:58:44.816776 24411 consensus_queue.cc:1035] T 1ac4317290b24209a76d2e19e68ee45b P 05a74e1eb2474b3d8b3a2da1ef127e01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20250109 08:58:44.808357 23454 raft_consensus.cc:1270] T bd6fcd838b0249cabd5f1f95e5538ab6 P cf55383bef6649eb9e78cb2d2512dedc [term 2 FOLLOWER]: Refusing update from remote peer 235ad6af04c34c88a5261daa3604fada: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:58:44.831799 24442 consensus_queue.cc:1035] T 6eef9bb3c645428d9f1cb843bae9a264 P 235ad6af04c34c88a5261daa3604fada [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20250109 08:58:44.834518 24392 raft_consensus.cc:2743] T ed79047393d2425f8ecff31ea08b9cad P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250109 08:58:44.832605 24112 leader_election.cc:304] T 1365130ef97c4581b623c22c39753831 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 235ad6af04c34c88a5261daa3604fada; no voters: cf55383bef6649eb9e78cb2d2512dedc, ed1efbce9ee3401e99a5260edeae4e7c
I20250109 08:58:44.835587 23494 leader_election.cc:304] T 9b0bbf2e4a424fb5ac13a3b55627b048 P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01; no voters: cf55383bef6649eb9e78cb2d2512dedc, ed1efbce9ee3401e99a5260edeae4e7c
I20250109 08:58:44.838382 24422 raft_consensus.cc:2743] T 9b0bbf2e4a424fb5ac13a3b55627b048 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250109 08:58:44.835687 23525 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5251927614642639bd46ecd5373d2a4" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" is_pre_election: true
I20250109 08:58:44.837030 24334 raft_consensus.cc:2743] T 1365130ef97c4581b623c22c39753831 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250109 08:58:44.844190 23494 leader_election.cc:304] T 4df64aae69ca41dc91286cae1603000f P 05a74e1eb2474b3d8b3a2da1ef127e01 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01; no voters: 235ad6af04c34c88a5261daa3604fada, ed1efbce9ee3401e99a5260edeae4e7c
I20250109 08:58:44.845660 24430 raft_consensus.cc:2743] T 4df64aae69ca41dc91286cae1603000f P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20250109 08:58:44.839026 24112 leader_election.cc:304] T db111e41c1be497e8e673a2c42b9ba66 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 235ad6af04c34c88a5261daa3604fada; no voters: cf55383bef6649eb9e78cb2d2512dedc, ed1efbce9ee3401e99a5260edeae4e7c
I20250109 08:58:44.851289 24442 raft_consensus.cc:2743] T db111e41c1be497e8e673a2c42b9ba66 P 235ad6af04c34c88a5261daa3604fada [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20250109 08:58:44.851907 24439 consensus_queue.cc:1035] T bd6fcd838b0249cabd5f1f95e5538ab6 P 235ad6af04c34c88a5261daa3604fada [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20250109 08:58:44.854297 24110 leader_election.cc:304] T e5251927614642639bd46ecd5373d2a4 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 235ad6af04c34c88a5261daa3604fada; no voters: 05a74e1eb2474b3d8b3a2da1ef127e01, cf55383bef6649eb9e78cb2d2512dedc
I20250109 08:58:44.862149 24392 raft_consensus.cc:2743] T e5251927614642639bd46ecd5373d2a4 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250109 08:58:45.076267 20363 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250109 08:58:45.112278 23521 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "060daed1d3334c6188413dd381b82bf9"
dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01"
mode: GRACEFUL
new_leader_uuid: "235ad6af04c34c88a5261daa3604fada"
from {username='slave'} at 127.0.0.1:45238
I20250109 08:58:45.112871 23521 raft_consensus.cc:604] T 060daed1d3334c6188413dd381b82bf9 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 LEADER]: Received request to transfer leadership to TS 235ad6af04c34c88a5261daa3604fada
I20250109 08:58:45.115113 23372 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "1340cf8ee2cb423f875276137da17e8b"
dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c"
mode: GRACEFUL
new_leader_uuid: "235ad6af04c34c88a5261daa3604fada"
from {username='slave'} at 127.0.0.1:51604
I20250109 08:58:45.115630 23372 raft_consensus.cc:604] T 1340cf8ee2cb423f875276137da17e8b P ed1efbce9ee3401e99a5260edeae4e7c [term 2 LEADER]: Received request to transfer leadership to TS 235ad6af04c34c88a5261daa3604fada
I20250109 08:58:45.117885 23521 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "5f0d500a91b54851bda52e5ccf741fd6"
dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01"
mode: GRACEFUL
new_leader_uuid: "235ad6af04c34c88a5261daa3604fada"
from {username='slave'} at 127.0.0.1:45238
I20250109 08:58:45.118464 23521 raft_consensus.cc:604] T 5f0d500a91b54851bda52e5ccf741fd6 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 LEADER]: Received request to transfer leadership to TS 235ad6af04c34c88a5261daa3604fada
I20250109 08:58:45.120432 23372 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "db111e41c1be497e8e673a2c42b9ba66"
dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c"
mode: GRACEFUL
new_leader_uuid: "235ad6af04c34c88a5261daa3604fada"
from {username='slave'} at 127.0.0.1:51604
I20250109 08:58:45.120877 23372 raft_consensus.cc:604] T db111e41c1be497e8e673a2c42b9ba66 P ed1efbce9ee3401e99a5260edeae4e7c [term 2 LEADER]: Received request to transfer leadership to TS 235ad6af04c34c88a5261daa3604fada
I20250109 08:58:45.125320 23372 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "ed79047393d2425f8ecff31ea08b9cad"
dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c"
mode: GRACEFUL
new_leader_uuid: "235ad6af04c34c88a5261daa3604fada"
from {username='slave'} at 127.0.0.1:51604
I20250109 08:58:45.126647 23372 raft_consensus.cc:604] T ed79047393d2425f8ecff31ea08b9cad P ed1efbce9ee3401e99a5260edeae4e7c [term 1 LEADER]: Received request to transfer leadership to TS 235ad6af04c34c88a5261daa3604fada
I20250109 08:58:45.127691 20363 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 5
I20250109 08:58:45.128952 20363 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250109 08:58:45.129515 20363 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250109 08:58:45.150523 24422 raft_consensus.cc:988] T 060daed1d3334c6188413dd381b82bf9 P 05a74e1eb2474b3d8b3a2da1ef127e01: : Instructing follower 235ad6af04c34c88a5261daa3604fada to start an election
I20250109 08:58:45.151119 24430 raft_consensus.cc:1076] T 060daed1d3334c6188413dd381b82bf9 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 LEADER]: Signalling peer 235ad6af04c34c88a5261daa3604fada to start an election
I20250109 08:58:45.152612 24152 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "060daed1d3334c6188413dd381b82bf9"
dest_uuid: "235ad6af04c34c88a5261daa3604fada"
from {username='slave'} at 127.0.0.1:43346
I20250109 08:58:45.153127 24152 raft_consensus.cc:491] T 060daed1d3334c6188413dd381b82bf9 P 235ad6af04c34c88a5261daa3604fada [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:58:45.153467 24152 raft_consensus.cc:3054] T 060daed1d3334c6188413dd381b82bf9 P 235ad6af04c34c88a5261daa3604fada [term 2 FOLLOWER]: Advancing to term 3
I20250109 08:58:45.158496 24152 raft_consensus.cc:513] T 060daed1d3334c6188413dd381b82bf9 P 235ad6af04c34c88a5261daa3604fada [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:45.161439 24152 leader_election.cc:290] T 060daed1d3334c6188413dd381b82bf9 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 3 election: Requested vote from peers 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829), ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365)
I20250109 08:58:45.162647 23521 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "060daed1d3334c6188413dd381b82bf9" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01"
I20250109 08:58:45.163316 23521 raft_consensus.cc:3049] T 060daed1d3334c6188413dd381b82bf9 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 LEADER]: Stepping down as leader of term 2
I20250109 08:58:45.163829 23521 raft_consensus.cc:738] T 060daed1d3334c6188413dd381b82bf9 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 05a74e1eb2474b3d8b3a2da1ef127e01, State: Running, Role: LEADER
I20250109 08:58:45.164618 23521 consensus_queue.cc:260] T 060daed1d3334c6188413dd381b82bf9 P 05a74e1eb2474b3d8b3a2da1ef127e01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:45.165756 23521 raft_consensus.cc:3054] T 060daed1d3334c6188413dd381b82bf9 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 FOLLOWER]: Advancing to term 3
I20250109 08:58:45.163296 23373 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "060daed1d3334c6188413dd381b82bf9" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c"
I20250109 08:58:45.166759 23373 raft_consensus.cc:3054] T 060daed1d3334c6188413dd381b82bf9 P ed1efbce9ee3401e99a5260edeae4e7c [term 2 FOLLOWER]: Advancing to term 3
I20250109 08:58:45.171038 23521 raft_consensus.cc:2463] T 060daed1d3334c6188413dd381b82bf9 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 235ad6af04c34c88a5261daa3604fada in term 3.
I20250109 08:58:45.171537 23373 raft_consensus.cc:2463] T 060daed1d3334c6188413dd381b82bf9 P ed1efbce9ee3401e99a5260edeae4e7c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 235ad6af04c34c88a5261daa3604fada in term 3.
I20250109 08:58:45.172470 24112 leader_election.cc:304] T 060daed1d3334c6188413dd381b82bf9 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 235ad6af04c34c88a5261daa3604fada, ed1efbce9ee3401e99a5260edeae4e7c; no voters:
I20250109 08:58:45.173242 24412 raft_consensus.cc:2798] T 060daed1d3334c6188413dd381b82bf9 P 235ad6af04c34c88a5261daa3604fada [term 3 FOLLOWER]: Leader election won for term 3
I20250109 08:58:45.173684 24412 raft_consensus.cc:695] T 060daed1d3334c6188413dd381b82bf9 P 235ad6af04c34c88a5261daa3604fada [term 3 LEADER]: Becoming Leader. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Running, Role: LEADER
I20250109 08:58:45.174396 24412 consensus_queue.cc:237] T 060daed1d3334c6188413dd381b82bf9 P 235ad6af04c34c88a5261daa3604fada [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:45.184840 23202 catalog_manager.cc:5526] T 060daed1d3334c6188413dd381b82bf9 P 235ad6af04c34c88a5261daa3604fada reported cstate change: term changed from 2 to 3, leader changed from 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196) to 235ad6af04c34c88a5261daa3604fada (127.19.226.193). New cstate: current_term: 3 leader_uuid: "235ad6af04c34c88a5261daa3604fada" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:45.251605 24405 raft_consensus.cc:988] T 1340cf8ee2cb423f875276137da17e8b P ed1efbce9ee3401e99a5260edeae4e7c: : Instructing follower 235ad6af04c34c88a5261daa3604fada to start an election
I20250109 08:58:45.252081 24398 raft_consensus.cc:1076] T 1340cf8ee2cb423f875276137da17e8b P ed1efbce9ee3401e99a5260edeae4e7c [term 2 LEADER]: Signalling peer 235ad6af04c34c88a5261daa3604fada to start an election
I20250109 08:58:45.253340 24152 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "1340cf8ee2cb423f875276137da17e8b"
dest_uuid: "235ad6af04c34c88a5261daa3604fada"
from {username='slave'} at 127.0.0.1:43328
I20250109 08:58:45.254099 24152 raft_consensus.cc:491] T 1340cf8ee2cb423f875276137da17e8b P 235ad6af04c34c88a5261daa3604fada [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:58:45.254449 24152 raft_consensus.cc:3054] T 1340cf8ee2cb423f875276137da17e8b P 235ad6af04c34c88a5261daa3604fada [term 2 FOLLOWER]: Advancing to term 3
I20250109 08:58:45.258725 24152 raft_consensus.cc:513] T 1340cf8ee2cb423f875276137da17e8b P 235ad6af04c34c88a5261daa3604fada [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:45.260098 24152 leader_election.cc:290] T 1340cf8ee2cb423f875276137da17e8b P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 3 election: Requested vote from peers ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365), cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551)
I20250109 08:58:45.260830 23372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1340cf8ee2cb423f875276137da17e8b" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c"
I20250109 08:58:45.261032 23450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1340cf8ee2cb423f875276137da17e8b" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc"
I20250109 08:58:45.261453 23372 raft_consensus.cc:3049] T 1340cf8ee2cb423f875276137da17e8b P ed1efbce9ee3401e99a5260edeae4e7c [term 2 LEADER]: Stepping down as leader of term 2
I20250109 08:58:45.261606 23450 raft_consensus.cc:3054] T 1340cf8ee2cb423f875276137da17e8b P cf55383bef6649eb9e78cb2d2512dedc [term 2 FOLLOWER]: Advancing to term 3
I20250109 08:58:45.261770 23372 raft_consensus.cc:738] T 1340cf8ee2cb423f875276137da17e8b P ed1efbce9ee3401e99a5260edeae4e7c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ed1efbce9ee3401e99a5260edeae4e7c, State: Running, Role: LEADER
I20250109 08:58:45.262544 23372 consensus_queue.cc:260] T 1340cf8ee2cb423f875276137da17e8b P ed1efbce9ee3401e99a5260edeae4e7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:45.263521 23372 raft_consensus.cc:3054] T 1340cf8ee2cb423f875276137da17e8b P ed1efbce9ee3401e99a5260edeae4e7c [term 2 FOLLOWER]: Advancing to term 3
I20250109 08:58:45.267346 23450 raft_consensus.cc:2463] T 1340cf8ee2cb423f875276137da17e8b P cf55383bef6649eb9e78cb2d2512dedc [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 235ad6af04c34c88a5261daa3604fada in term 3.
I20250109 08:58:45.268471 24112 leader_election.cc:304] T 1340cf8ee2cb423f875276137da17e8b P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 235ad6af04c34c88a5261daa3604fada, cf55383bef6649eb9e78cb2d2512dedc; no voters:
I20250109 08:58:45.268945 23372 raft_consensus.cc:2463] T 1340cf8ee2cb423f875276137da17e8b P ed1efbce9ee3401e99a5260edeae4e7c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 235ad6af04c34c88a5261daa3604fada in term 3.
I20250109 08:58:45.269323 24412 raft_consensus.cc:2798] T 1340cf8ee2cb423f875276137da17e8b P 235ad6af04c34c88a5261daa3604fada [term 3 FOLLOWER]: Leader election won for term 3
I20250109 08:58:45.270468 24412 raft_consensus.cc:695] T 1340cf8ee2cb423f875276137da17e8b P 235ad6af04c34c88a5261daa3604fada [term 3 LEADER]: Becoming Leader. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Running, Role: LEADER
I20250109 08:58:45.271270 24412 consensus_queue.cc:237] T 1340cf8ee2cb423f875276137da17e8b P 235ad6af04c34c88a5261daa3604fada [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:45.282729 23202 catalog_manager.cc:5526] T 1340cf8ee2cb423f875276137da17e8b P 235ad6af04c34c88a5261daa3604fada reported cstate change: term changed from 2 to 3, leader changed from ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194) to 235ad6af04c34c88a5261daa3604fada (127.19.226.193). New cstate: current_term: 3 leader_uuid: "235ad6af04c34c88a5261daa3604fada" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:45.316498 24405 raft_consensus.cc:988] T db111e41c1be497e8e673a2c42b9ba66 P ed1efbce9ee3401e99a5260edeae4e7c: : Instructing follower 235ad6af04c34c88a5261daa3604fada to start an election
I20250109 08:58:45.316939 24398 raft_consensus.cc:1076] T db111e41c1be497e8e673a2c42b9ba66 P ed1efbce9ee3401e99a5260edeae4e7c [term 2 LEADER]: Signalling peer 235ad6af04c34c88a5261daa3604fada to start an election
I20250109 08:58:45.318576 24157 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "db111e41c1be497e8e673a2c42b9ba66"
dest_uuid: "235ad6af04c34c88a5261daa3604fada"
from {username='slave'} at 127.0.0.1:43328
I20250109 08:58:45.319123 24157 raft_consensus.cc:491] T db111e41c1be497e8e673a2c42b9ba66 P 235ad6af04c34c88a5261daa3604fada [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:58:45.319561 24157 raft_consensus.cc:3054] T db111e41c1be497e8e673a2c42b9ba66 P 235ad6af04c34c88a5261daa3604fada [term 2 FOLLOWER]: Advancing to term 3
I20250109 08:58:45.326694 24157 raft_consensus.cc:513] T db111e41c1be497e8e673a2c42b9ba66 P 235ad6af04c34c88a5261daa3604fada [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:45.329538 24157 leader_election.cc:290] T db111e41c1be497e8e673a2c42b9ba66 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 3 election: Requested vote from peers ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365), cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551)
I20250109 08:58:45.330237 23372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db111e41c1be497e8e673a2c42b9ba66" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c"
I20250109 08:58:45.330662 23450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db111e41c1be497e8e673a2c42b9ba66" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc"
I20250109 08:58:45.330857 23372 raft_consensus.cc:3049] T db111e41c1be497e8e673a2c42b9ba66 P ed1efbce9ee3401e99a5260edeae4e7c [term 2 LEADER]: Stepping down as leader of term 2
I20250109 08:58:45.331228 23372 raft_consensus.cc:738] T db111e41c1be497e8e673a2c42b9ba66 P ed1efbce9ee3401e99a5260edeae4e7c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ed1efbce9ee3401e99a5260edeae4e7c, State: Running, Role: LEADER
I20250109 08:58:45.331261 23450 raft_consensus.cc:3054] T db111e41c1be497e8e673a2c42b9ba66 P cf55383bef6649eb9e78cb2d2512dedc [term 2 FOLLOWER]: Advancing to term 3
I20250109 08:58:45.332016 23372 consensus_queue.cc:260] T db111e41c1be497e8e673a2c42b9ba66 P ed1efbce9ee3401e99a5260edeae4e7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:45.333245 23372 raft_consensus.cc:3054] T db111e41c1be497e8e673a2c42b9ba66 P ed1efbce9ee3401e99a5260edeae4e7c [term 2 FOLLOWER]: Advancing to term 3
I20250109 08:58:45.339017 23450 raft_consensus.cc:2463] T db111e41c1be497e8e673a2c42b9ba66 P cf55383bef6649eb9e78cb2d2512dedc [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 235ad6af04c34c88a5261daa3604fada in term 3.
I20250109 08:58:45.339391 23372 raft_consensus.cc:2463] T db111e41c1be497e8e673a2c42b9ba66 P ed1efbce9ee3401e99a5260edeae4e7c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 235ad6af04c34c88a5261daa3604fada in term 3.
I20250109 08:58:45.340181 24112 leader_election.cc:304] T db111e41c1be497e8e673a2c42b9ba66 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 235ad6af04c34c88a5261daa3604fada, cf55383bef6649eb9e78cb2d2512dedc; no voters:
I20250109 08:58:45.340898 24442 raft_consensus.cc:2798] T db111e41c1be497e8e673a2c42b9ba66 P 235ad6af04c34c88a5261daa3604fada [term 3 FOLLOWER]: Leader election won for term 3
I20250109 08:58:45.341372 24442 raft_consensus.cc:695] T db111e41c1be497e8e673a2c42b9ba66 P 235ad6af04c34c88a5261daa3604fada [term 3 LEADER]: Becoming Leader. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Running, Role: LEADER
I20250109 08:58:45.342738 24442 consensus_queue.cc:237] T db111e41c1be497e8e673a2c42b9ba66 P 235ad6af04c34c88a5261daa3604fada [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } }
I20250109 08:58:45.352073 23202 catalog_manager.cc:5526] T db111e41c1be497e8e673a2c42b9ba66 P 235ad6af04c34c88a5261daa3604fada reported cstate change: term changed from 2 to 3, leader changed from ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194) to 235ad6af04c34c88a5261daa3604fada (127.19.226.193). New cstate: current_term: 3 leader_uuid: "235ad6af04c34c88a5261daa3604fada" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:45.423182 24405 raft_consensus.cc:988] T ed79047393d2425f8ecff31ea08b9cad P ed1efbce9ee3401e99a5260edeae4e7c: : Instructing follower 235ad6af04c34c88a5261daa3604fada to start an election
I20250109 08:58:45.423617 24398 raft_consensus.cc:1076] T ed79047393d2425f8ecff31ea08b9cad P ed1efbce9ee3401e99a5260edeae4e7c [term 1 LEADER]: Signalling peer 235ad6af04c34c88a5261daa3604fada to start an election
I20250109 08:58:45.424934 24157 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "ed79047393d2425f8ecff31ea08b9cad"
dest_uuid: "235ad6af04c34c88a5261daa3604fada"
from {username='slave'} at 127.0.0.1:43328
I20250109 08:58:45.425372 24157 raft_consensus.cc:491] T ed79047393d2425f8ecff31ea08b9cad P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:58:45.425629 24157 raft_consensus.cc:3054] T ed79047393d2425f8ecff31ea08b9cad P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:45.429908 24157 raft_consensus.cc:513] T ed79047393d2425f8ecff31ea08b9cad P 235ad6af04c34c88a5261daa3604fada [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:45.431214 24157 leader_election.cc:290] T ed79047393d2425f8ecff31ea08b9cad P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 2 election: Requested vote from peers 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829), ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365)
I20250109 08:58:45.432181 23521 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed79047393d2425f8ecff31ea08b9cad" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01"
I20250109 08:58:45.432181 23373 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed79047393d2425f8ecff31ea08b9cad" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ed1efbce9ee3401e99a5260edeae4e7c"
I20250109 08:58:45.432972 23521 raft_consensus.cc:3054] T ed79047393d2425f8ecff31ea08b9cad P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:45.433010 23373 raft_consensus.cc:3049] T ed79047393d2425f8ecff31ea08b9cad P ed1efbce9ee3401e99a5260edeae4e7c [term 1 LEADER]: Stepping down as leader of term 1
I20250109 08:58:45.433435 23373 raft_consensus.cc:738] T ed79047393d2425f8ecff31ea08b9cad P ed1efbce9ee3401e99a5260edeae4e7c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ed1efbce9ee3401e99a5260edeae4e7c, State: Running, Role: LEADER
I20250109 08:58:45.434155 23373 consensus_queue.cc:260] T ed79047393d2425f8ecff31ea08b9cad P ed1efbce9ee3401e99a5260edeae4e7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:45.435297 23373 raft_consensus.cc:3054] T ed79047393d2425f8ecff31ea08b9cad P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:45.439441 23521 raft_consensus.cc:2463] T ed79047393d2425f8ecff31ea08b9cad P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 235ad6af04c34c88a5261daa3604fada in term 2.
I20250109 08:58:45.440475 24110 leader_election.cc:304] T ed79047393d2425f8ecff31ea08b9cad P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05a74e1eb2474b3d8b3a2da1ef127e01, 235ad6af04c34c88a5261daa3604fada; no voters:
I20250109 08:58:45.441097 24442 raft_consensus.cc:2798] T ed79047393d2425f8ecff31ea08b9cad P 235ad6af04c34c88a5261daa3604fada [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:58:45.441141 23373 raft_consensus.cc:2463] T ed79047393d2425f8ecff31ea08b9cad P ed1efbce9ee3401e99a5260edeae4e7c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 235ad6af04c34c88a5261daa3604fada in term 2.
I20250109 08:58:45.442611 24442 raft_consensus.cc:695] T ed79047393d2425f8ecff31ea08b9cad P 235ad6af04c34c88a5261daa3604fada [term 2 LEADER]: Becoming Leader. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Running, Role: LEADER
I20250109 08:58:45.443468 24442 consensus_queue.cc:237] T ed79047393d2425f8ecff31ea08b9cad P 235ad6af04c34c88a5261daa3604fada [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } }
I20250109 08:58:45.452548 23202 catalog_manager.cc:5526] T ed79047393d2425f8ecff31ea08b9cad P 235ad6af04c34c88a5261daa3604fada reported cstate change: term changed from 1 to 2, leader changed from ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194) to 235ad6af04c34c88a5261daa3604fada (127.19.226.193). New cstate: current_term: 2 leader_uuid: "235ad6af04c34c88a5261daa3604fada" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 } health_report { overall_health: UNKNOWN } } }
W20250109 08:58:45.562335 23549 debug-util.cc:398] Leaking SignalData structure 0x7b08002fce20 after lost signal to thread 20366
W20250109 08:58:45.563225 23549 debug-util.cc:398] Leaking SignalData structure 0x7b08002b49c0 after lost signal to thread 23237
W20250109 08:58:45.564146 23549 debug-util.cc:398] Leaking SignalData structure 0x7b0800176380 after lost signal to thread 23404
W20250109 08:58:45.564980 23549 debug-util.cc:398] Leaking SignalData structure 0x7b080032e220 after lost signal to thread 23478
W20250109 08:58:45.565762 23549 debug-util.cc:398] Leaking SignalData structure 0x7b080024a920 after lost signal to thread 23552
W20250109 08:58:45.566691 23549 debug-util.cc:398] Leaking SignalData structure 0x7b08002481a0 after lost signal to thread 24184
W20250109 08:58:45.567003 23549 debug-util.cc:398] Leaking SignalData structure 0x7b08002926a0 after lost signal to thread 24335
W20250109 08:58:45.567313 23549 debug-util.cc:398] Leaking SignalData structure 0x7b0800374040 after lost signal to thread 24394
I20250109 08:58:45.611295 23372 raft_consensus.cc:1270] T 060daed1d3334c6188413dd381b82bf9 P ed1efbce9ee3401e99a5260edeae4e7c [term 3 FOLLOWER]: Refusing update from remote peer 235ad6af04c34c88a5261daa3604fada: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20250109 08:58:45.614480 24442 consensus_queue.cc:1035] T 060daed1d3334c6188413dd381b82bf9 P 235ad6af04c34c88a5261daa3604fada [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:58:45.628070 23520 raft_consensus.cc:1270] T 060daed1d3334c6188413dd381b82bf9 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 3 FOLLOWER]: Refusing update from remote peer 235ad6af04c34c88a5261daa3604fada: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20250109 08:58:45.636910 24442 consensus_queue.cc:1035] T 060daed1d3334c6188413dd381b82bf9 P 235ad6af04c34c88a5261daa3604fada [LEADER]: Connected to new peer: Peer: permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:58:45.660077 23450 raft_consensus.cc:1270] T 1340cf8ee2cb423f875276137da17e8b P cf55383bef6649eb9e78cb2d2512dedc [term 3 FOLLOWER]: Refusing update from remote peer 235ad6af04c34c88a5261daa3604fada: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20250109 08:58:45.661592 24442 consensus_queue.cc:1035] T 1340cf8ee2cb423f875276137da17e8b P 235ad6af04c34c88a5261daa3604fada [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:58:45.669852 24411 raft_consensus.cc:988] T 5f0d500a91b54851bda52e5ccf741fd6 P 05a74e1eb2474b3d8b3a2da1ef127e01: : Instructing follower 235ad6af04c34c88a5261daa3604fada to start an election
I20250109 08:58:45.670356 24411 raft_consensus.cc:1076] T 5f0d500a91b54851bda52e5ccf741fd6 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 LEADER]: Signalling peer 235ad6af04c34c88a5261daa3604fada to start an election
I20250109 08:58:45.671780 24152 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "5f0d500a91b54851bda52e5ccf741fd6"
dest_uuid: "235ad6af04c34c88a5261daa3604fada"
from {username='slave'} at 127.0.0.1:43346
I20250109 08:58:45.672305 24152 raft_consensus.cc:491] T 5f0d500a91b54851bda52e5ccf741fd6 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:58:45.672659 24152 raft_consensus.cc:3054] T 5f0d500a91b54851bda52e5ccf741fd6 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:45.691152 23373 raft_consensus.cc:1270] T 1340cf8ee2cb423f875276137da17e8b P ed1efbce9ee3401e99a5260edeae4e7c [term 3 FOLLOWER]: Refusing update from remote peer 235ad6af04c34c88a5261daa3604fada: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20250109 08:58:45.692721 24465 consensus_queue.cc:1035] T 1340cf8ee2cb423f875276137da17e8b P 235ad6af04c34c88a5261daa3604fada [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:58:45.680951 24152 raft_consensus.cc:513] T 5f0d500a91b54851bda52e5ccf741fd6 P 235ad6af04c34c88a5261daa3604fada [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:45.704164 23453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f0d500a91b54851bda52e5ccf741fd6" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cf55383bef6649eb9e78cb2d2512dedc"
I20250109 08:58:45.704823 23453 raft_consensus.cc:3054] T 5f0d500a91b54851bda52e5ccf741fd6 P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:45.706923 23520 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f0d500a91b54851bda52e5ccf741fd6" candidate_uuid: "235ad6af04c34c88a5261daa3604fada" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01"
I20250109 08:58:45.707692 23520 raft_consensus.cc:3049] T 5f0d500a91b54851bda52e5ccf741fd6 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 LEADER]: Stepping down as leader of term 1
I20250109 08:58:45.708020 23520 raft_consensus.cc:738] T 5f0d500a91b54851bda52e5ccf741fd6 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 05a74e1eb2474b3d8b3a2da1ef127e01, State: Running, Role: LEADER
I20250109 08:58:45.708981 23520 consensus_queue.cc:260] T 5f0d500a91b54851bda52e5ccf741fd6 P 05a74e1eb2474b3d8b3a2da1ef127e01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:45.711305 23520 raft_consensus.cc:3054] T 5f0d500a91b54851bda52e5ccf741fd6 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:45.714233 24152 leader_election.cc:290] T 5f0d500a91b54851bda52e5ccf741fd6 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 2 election: Requested vote from peers cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551), 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196:41829)
I20250109 08:58:45.712275 23453 raft_consensus.cc:2463] T 5f0d500a91b54851bda52e5ccf741fd6 P cf55383bef6649eb9e78cb2d2512dedc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 235ad6af04c34c88a5261daa3604fada in term 2.
I20250109 08:58:45.723649 24112 leader_election.cc:304] T 5f0d500a91b54851bda52e5ccf741fd6 P 235ad6af04c34c88a5261daa3604fada [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 235ad6af04c34c88a5261daa3604fada, cf55383bef6649eb9e78cb2d2512dedc; no voters:
I20250109 08:58:45.727269 24465 raft_consensus.cc:2798] T 5f0d500a91b54851bda52e5ccf741fd6 P 235ad6af04c34c88a5261daa3604fada [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:58:45.727808 24465 raft_consensus.cc:695] T 5f0d500a91b54851bda52e5ccf741fd6 P 235ad6af04c34c88a5261daa3604fada [term 2 LEADER]: Becoming Leader. State: Replica: 235ad6af04c34c88a5261daa3604fada, State: Running, Role: LEADER
I20250109 08:58:45.728686 24465 consensus_queue.cc:237] T 5f0d500a91b54851bda52e5ccf741fd6 P 235ad6af04c34c88a5261daa3604fada [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } }
I20250109 08:58:45.738983 23520 raft_consensus.cc:2463] T 5f0d500a91b54851bda52e5ccf741fd6 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 235ad6af04c34c88a5261daa3604fada in term 2.
I20250109 08:58:45.765786 23202 catalog_manager.cc:5526] T 5f0d500a91b54851bda52e5ccf741fd6 P 235ad6af04c34c88a5261daa3604fada reported cstate change: term changed from 1 to 2, leader changed from 05a74e1eb2474b3d8b3a2da1ef127e01 (127.19.226.196) to 235ad6af04c34c88a5261daa3604fada (127.19.226.193). New cstate: current_term: 2 leader_uuid: "235ad6af04c34c88a5261daa3604fada" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "235ad6af04c34c88a5261daa3604fada" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 42691 } health_report { overall_health: HEALTHY } } }
I20250109 08:58:45.800719 23373 raft_consensus.cc:1270] T db111e41c1be497e8e673a2c42b9ba66 P ed1efbce9ee3401e99a5260edeae4e7c [term 3 FOLLOWER]: Refusing update from remote peer 235ad6af04c34c88a5261daa3604fada: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20250109 08:58:45.801815 24412 consensus_queue.cc:1035] T db111e41c1be497e8e673a2c42b9ba66 P 235ad6af04c34c88a5261daa3604fada [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:58:45.818905 23450 raft_consensus.cc:1270] T db111e41c1be497e8e673a2c42b9ba66 P cf55383bef6649eb9e78cb2d2512dedc [term 3 FOLLOWER]: Refusing update from remote peer 235ad6af04c34c88a5261daa3604fada: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20250109 08:58:45.825094 24412 consensus_queue.cc:1035] T db111e41c1be497e8e673a2c42b9ba66 P 235ad6af04c34c88a5261daa3604fada [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:58:45.870463 23372 raft_consensus.cc:1270] T ed79047393d2425f8ecff31ea08b9cad P ed1efbce9ee3401e99a5260edeae4e7c [term 2 FOLLOWER]: Refusing update from remote peer 235ad6af04c34c88a5261daa3604fada: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:58:45.871805 24477 consensus_queue.cc:1035] T ed79047393d2425f8ecff31ea08b9cad P 235ad6af04c34c88a5261daa3604fada [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed1efbce9ee3401e99a5260edeae4e7c" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 41365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:58:45.889777 23520 raft_consensus.cc:1270] T ed79047393d2425f8ecff31ea08b9cad P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 FOLLOWER]: Refusing update from remote peer 235ad6af04c34c88a5261daa3604fada: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:58:45.891212 24412 consensus_queue.cc:1035] T ed79047393d2425f8ecff31ea08b9cad P 235ad6af04c34c88a5261daa3604fada [LEADER]: Connected to new peer: Peer: permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:58:46.167264 20363 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250109 08:58:46.204692 20363 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250109 08:58:46.205942 20363 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250109 08:58:46.206455 20363 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250109 08:58:46.229885 23520 raft_consensus.cc:1270] T 5f0d500a91b54851bda52e5ccf741fd6 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 FOLLOWER]: Refusing update from remote peer 235ad6af04c34c88a5261daa3604fada: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:58:46.232882 24412 consensus_queue.cc:1035] T 5f0d500a91b54851bda52e5ccf741fd6 P 235ad6af04c34c88a5261daa3604fada [LEADER]: Connected to new peer: Peer: permanent_uuid: "05a74e1eb2474b3d8b3a2da1ef127e01" member_type: VOTER last_known_addr { host: "127.19.226.196" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:58:46.262441 20363 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250109 08:58:46.262996 23447 raft_consensus.cc:1270] T 5f0d500a91b54851bda52e5ccf741fd6 P cf55383bef6649eb9e78cb2d2512dedc [term 2 FOLLOWER]: Refusing update from remote peer 235ad6af04c34c88a5261daa3604fada: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:58:46.266184 24442 consensus_queue.cc:1035] T 5f0d500a91b54851bda52e5ccf741fd6 P 235ad6af04c34c88a5261daa3604fada [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf55383bef6649eb9e78cb2d2512dedc" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 35551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:58:46.314301 20363 tablet_server.cc:178] TabletServer@127.19.226.193:42691 shutting down...
W20250109 08:58:46.334363 23420 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 81 similar messages]
W20250109 08:58:46.337152 23420 consensus_peers.cc:487] T 8d3947300cfc4523acdab19fc992f26a P cf55383bef6649eb9e78cb2d2512dedc -> Peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Couldn't send request to peer 235ad6af04c34c88a5261daa3604fada. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:46.377015 20363 ts_tablet_manager.cc:1500] Shutting down tablet manager...
W20250109 08:58:46.380513 23420 consensus_peers.cc:487] T de98c70e25494d2a8e581443be836bd3 P cf55383bef6649eb9e78cb2d2512dedc -> Peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Couldn't send request to peer 235ad6af04c34c88a5261daa3604fada. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:46.381228 20363 tablet_replica.cc:331] stopping tablet replica
I20250109 08:58:46.382388 20363 raft_consensus.cc:2238] T 1c5b5beea4a840eb9677097cacee3408 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:46.382863 20363 raft_consensus.cc:2267] T 1c5b5beea4a840eb9677097cacee3408 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus is shut down!
W20250109 08:58:46.384361 23346 consensus_peers.cc:487] T 4df64aae69ca41dc91286cae1603000f P ed1efbce9ee3401e99a5260edeae4e7c -> Peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Couldn't send request to peer 235ad6af04c34c88a5261daa3604fada. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:46.386071 20363 tablet_replica.cc:331] stopping tablet replica
I20250109 08:58:46.387013 20363 raft_consensus.cc:2238] T a7d2ab61d238471c9f287ac1d5db4ed0 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:46.387493 20363 raft_consensus.cc:2267] T a7d2ab61d238471c9f287ac1d5db4ed0 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:46.390236 20363 tablet_replica.cc:331] stopping tablet replica
I20250109 08:58:46.391038 20363 raft_consensus.cc:2238] T 29828d9befb845d5bd49a80e018502e3 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:46.391525 20363 raft_consensus.cc:2267] T 29828d9befb845d5bd49a80e018502e3 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:46.394182 20363 tablet_replica.cc:331] stopping tablet replica
I20250109 08:58:46.394994 20363 raft_consensus.cc:2238] T 3208a99957554f6db341654856980c8a P 235ad6af04c34c88a5261daa3604fada [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:46.395460 20363 raft_consensus.cc:2267] T 3208a99957554f6db341654856980c8a P 235ad6af04c34c88a5261daa3604fada [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:46.397884 20363 tablet_replica.cc:331] stopping tablet replica
I20250109 08:58:46.398398 20363 raft_consensus.cc:2238] T 86d4074509744900abe94402467a7d85 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:46.398762 20363 raft_consensus.cc:2267] T 86d4074509744900abe94402467a7d85 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:46.400720 20363 tablet_replica.cc:331] stopping tablet replica
I20250109 08:58:46.401243 20363 raft_consensus.cc:2238] T 1365130ef97c4581b623c22c39753831 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:46.401654 20363 raft_consensus.cc:2267] T 1365130ef97c4581b623c22c39753831 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:46.404292 20363 tablet_replica.cc:331] stopping tablet replica
I20250109 08:58:46.404983 20363 raft_consensus.cc:2238] T de98c70e25494d2a8e581443be836bd3 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:46.405494 20363 raft_consensus.cc:2267] T de98c70e25494d2a8e581443be836bd3 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:46.407557 20363 tablet_replica.cc:331] stopping tablet replica
I20250109 08:58:46.408347 20363 raft_consensus.cc:2238] T 9b245cb3666c421c884cf20161a22831 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:46.408856 20363 raft_consensus.cc:2267] T 9b245cb3666c421c884cf20161a22831 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus is shut down!
W20250109 08:58:46.409900 23494 consensus_peers.cc:487] T 82ea7dc042dd4d05b260d508f4422c3c P 05a74e1eb2474b3d8b3a2da1ef127e01 -> Peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Couldn't send request to peer 235ad6af04c34c88a5261daa3604fada. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:46.410636 20363 tablet_replica.cc:331] stopping tablet replica
I20250109 08:58:46.411264 20363 raft_consensus.cc:2238] T fc1038255dd64cb5ba2425037d22f1f7 P 235ad6af04c34c88a5261daa3604fada [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:46.411643 20363 raft_consensus.cc:2267] T fc1038255dd64cb5ba2425037d22f1f7 P 235ad6af04c34c88a5261daa3604fada [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:46.413480 20363 tablet_replica.cc:331] stopping tablet replica
I20250109 08:58:46.413987 20363 raft_consensus.cc:2238] T bd6fcd838b0249cabd5f1f95e5538ab6 P 235ad6af04c34c88a5261daa3604fada [term 2 LEADER]: Raft consensus shutting down.
I20250109 08:58:46.414841 20363 raft_consensus.cc:2267] T bd6fcd838b0249cabd5f1f95e5538ab6 P 235ad6af04c34c88a5261daa3604fada [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:46.416795 20363 tablet_replica.cc:331] stopping tablet replica
I20250109 08:58:46.417378 20363 raft_consensus.cc:2238] T 5f0d500a91b54851bda52e5ccf741fd6 P 235ad6af04c34c88a5261daa3604fada [term 2 LEADER]: Raft consensus shutting down.
I20250109 08:58:46.418087 20363 raft_consensus.cc:2267] T 5f0d500a91b54851bda52e5ccf741fd6 P 235ad6af04c34c88a5261daa3604fada [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:46.420442 20363 tablet_replica.cc:331] stopping tablet replica
I20250109 08:58:46.421092 20363 raft_consensus.cc:2238] T a09d8dffc2e446f1acd3c7f6cc1705bb P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:46.421561 20363 raft_consensus.cc:2267] T a09d8dffc2e446f1acd3c7f6cc1705bb P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:46.424294 20363 tablet_replica.cc:331] stopping tablet replica
I20250109 08:58:46.425060 20363 raft_consensus.cc:2238] T 2e3d34c0d09843d2be54e47f3ccc16d8 P 235ad6af04c34c88a5261daa3604fada [term 2 LEADER]: Raft consensus shutting down.
I20250109 08:58:46.425922 20363 raft_consensus.cc:2267] T 2e3d34c0d09843d2be54e47f3ccc16d8 P 235ad6af04c34c88a5261daa3604fada [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:46.428112 20363 tablet_replica.cc:331] stopping tablet replica
I20250109 08:58:46.428807 20363 raft_consensus.cc:2238] T 970a82e754db47b6b1a19551d4ba43d8 P 235ad6af04c34c88a5261daa3604fada [term 3 LEADER]: Raft consensus shutting down.
I20250109 08:58:46.429700 20363 raft_consensus.cc:2267] T 970a82e754db47b6b1a19551d4ba43d8 P 235ad6af04c34c88a5261daa3604fada [term 3 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:46.432137 20363 tablet_replica.cc:331] stopping tablet replica
I20250109 08:58:46.433010 20363 raft_consensus.cc:2238] T 82ea7dc042dd4d05b260d508f4422c3c P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:46.433445 20363 raft_consensus.cc:2267] T 82ea7dc042dd4d05b260d508f4422c3c P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus is shut down!
W20250109 08:58:46.433624 23420 consensus_peers.cc:487] T a7d2ab61d238471c9f287ac1d5db4ed0 P cf55383bef6649eb9e78cb2d2512dedc -> Peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Couldn't send request to peer 235ad6af04c34c88a5261daa3604fada. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:46.435679 20363 tablet_replica.cc:331] stopping tablet replica
W20250109 08:58:46.435678 23494 consensus_peers.cc:487] T 9b245cb3666c421c884cf20161a22831 P 05a74e1eb2474b3d8b3a2da1ef127e01 -> Peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Couldn't send request to peer 235ad6af04c34c88a5261daa3604fada. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:46.436695 20363 raft_consensus.cc:2238] T cc9886333c0343798226d59758a1ea5e P 235ad6af04c34c88a5261daa3604fada [term 3 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:46.437072 20363 raft_consensus.cc:2267] T cc9886333c0343798226d59758a1ea5e P 235ad6af04c34c88a5261daa3604fada [term 3 FOLLOWER]: Raft consensus is shut down!
W20250109 08:58:46.437397 23346 consensus_peers.cc:487] T d714271c7ea64c0b8b13fcf63591ec07 P ed1efbce9ee3401e99a5260edeae4e7c -> Peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Couldn't send request to peer 235ad6af04c34c88a5261daa3604fada. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:46.439584 20363 tablet_replica.cc:331] stopping tablet replica
I20250109 08:58:46.440412 20363 raft_consensus.cc:2238] T 060daed1d3334c6188413dd381b82bf9 P 235ad6af04c34c88a5261daa3604fada [term 3 LEADER]: Raft consensus shutting down.
I20250109 08:58:46.441298 20363 raft_consensus.cc:2267] T 060daed1d3334c6188413dd381b82bf9 P 235ad6af04c34c88a5261daa3604fada [term 3 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:46.444342 20363 tablet_replica.cc:331] stopping tablet replica
I20250109 08:58:46.444921 20363 raft_consensus.cc:2238] T 24ce131f7bb143bea1581d5b0c2a7109 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:46.445291 20363 raft_consensus.cc:2267] T 24ce131f7bb143bea1581d5b0c2a7109 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:46.447285 20363 tablet_replica.cc:331] stopping tablet replica
I20250109 08:58:46.448040 20363 raft_consensus.cc:2238] T ea2b0762940e44dbbf69b9a163ace846 P 235ad6af04c34c88a5261daa3604fada [term 3 LEADER]: Raft consensus shutting down.
I20250109 08:58:46.448961 20363 raft_consensus.cc:2267] T ea2b0762940e44dbbf69b9a163ace846 P 235ad6af04c34c88a5261daa3604fada [term 3 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:46.451723 20363 tablet_replica.cc:331] stopping tablet replica
I20250109 08:58:46.452531 20363 raft_consensus.cc:2238] T e5251927614642639bd46ecd5373d2a4 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:46.452996 20363 raft_consensus.cc:2267] T e5251927614642639bd46ecd5373d2a4 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:46.455422 20363 tablet_replica.cc:331] stopping tablet replica
W20250109 08:58:46.455493 23420 consensus_peers.cc:487] T e5251927614642639bd46ecd5373d2a4 P cf55383bef6649eb9e78cb2d2512dedc -> Peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Couldn't send request to peer 235ad6af04c34c88a5261daa3604fada. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:46.456187 20363 raft_consensus.cc:2238] T 41e7f0cc38c4414ab88c92c1cdeb3984 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:46.456705 20363 raft_consensus.cc:2267] T 41e7f0cc38c4414ab88c92c1cdeb3984 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:46.459472 20363 tablet_replica.cc:331] stopping tablet replica
I20250109 08:58:46.460193 20363 raft_consensus.cc:2238] T 94f2be307f704c75bceb213e52307cfb P 235ad6af04c34c88a5261daa3604fada [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:46.460691 20363 raft_consensus.cc:2267] T 94f2be307f704c75bceb213e52307cfb P 235ad6af04c34c88a5261daa3604fada [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:46.463229 20363 tablet_replica.cc:331] stopping tablet replica
I20250109 08:58:46.463811 20363 raft_consensus.cc:2238] T 08214ee9c4c14bda92bdc6fa8231c313 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:46.464208 20363 raft_consensus.cc:2267] T 08214ee9c4c14bda92bdc6fa8231c313 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus is shut down!
W20250109 08:58:46.464632 23346 consensus_peers.cc:487] T 86b82e19759c4fd0b471a0dbff839aa5 P ed1efbce9ee3401e99a5260edeae4e7c -> Peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Couldn't send request to peer 235ad6af04c34c88a5261daa3604fada. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:46.466437 20363 tablet_replica.cc:331] stopping tablet replica
W20250109 08:58:46.466468 23346 consensus_peers.cc:487] T 86d4074509744900abe94402467a7d85 P ed1efbce9ee3401e99a5260edeae4e7c -> Peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Couldn't send request to peer 235ad6af04c34c88a5261daa3604fada. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:46.467185 20363 raft_consensus.cc:2238] T bfb140bb3a8b44bd8b4e1fcf0946162b P 235ad6af04c34c88a5261daa3604fada [term 2 LEADER]: Raft consensus shutting down.
I20250109 08:58:46.467928 20363 raft_consensus.cc:2267] T bfb140bb3a8b44bd8b4e1fcf0946162b P 235ad6af04c34c88a5261daa3604fada [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:46.469695 20363 tablet_replica.cc:331] stopping tablet replica
I20250109 08:58:46.470247 20363 raft_consensus.cc:2238] T 2ab70546dda143b5b5db15c3875aa348 P 235ad6af04c34c88a5261daa3604fada [term 3 LEADER]: Raft consensus shutting down.
I20250109 08:58:46.471136 20363 raft_consensus.cc:2267] T 2ab70546dda143b5b5db15c3875aa348 P 235ad6af04c34c88a5261daa3604fada [term 3 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:46.472978 20363 tablet_replica.cc:331] stopping tablet replica
I20250109 08:58:46.473506 20363 raft_consensus.cc:2238] T ed79047393d2425f8ecff31ea08b9cad P 235ad6af04c34c88a5261daa3604fada [term 2 LEADER]: Raft consensus shutting down.
I20250109 08:58:46.474177 20363 raft_consensus.cc:2267] T ed79047393d2425f8ecff31ea08b9cad P 235ad6af04c34c88a5261daa3604fada [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:46.476270 20363 tablet_replica.cc:331] stopping tablet replica
I20250109 08:58:46.476842 20363 raft_consensus.cc:2238] T 950e07e9cc794b8ab9ec5bd97d7e8e59 P 235ad6af04c34c88a5261daa3604fada [term 2 LEADER]: Raft consensus shutting down.
I20250109 08:58:46.477629 20363 raft_consensus.cc:2267] T 950e07e9cc794b8ab9ec5bd97d7e8e59 P 235ad6af04c34c88a5261daa3604fada [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:46.479862 20363 tablet_replica.cc:331] stopping tablet replica
I20250109 08:58:46.480522 20363 raft_consensus.cc:2238] T 6eef9bb3c645428d9f1cb843bae9a264 P 235ad6af04c34c88a5261daa3604fada [term 2 LEADER]: Raft consensus shutting down.
W20250109 08:58:46.481123 23494 consensus_peers.cc:487] T 444507f7812441ceaeebc44bc8f06293 P 05a74e1eb2474b3d8b3a2da1ef127e01 -> Peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Couldn't send request to peer 235ad6af04c34c88a5261daa3604fada. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:46.481323 20363 raft_consensus.cc:2267] T 6eef9bb3c645428d9f1cb843bae9a264 P 235ad6af04c34c88a5261daa3604fada [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:46.483392 20363 tablet_replica.cc:331] stopping tablet replica
I20250109 08:58:46.483882 20363 raft_consensus.cc:2238] T 77597d68423749e7b87f9f0db7a1ff3b P 235ad6af04c34c88a5261daa3604fada [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:46.484274 20363 raft_consensus.cc:2267] T 77597d68423749e7b87f9f0db7a1ff3b P 235ad6af04c34c88a5261daa3604fada [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:46.486173 20363 tablet_replica.cc:331] stopping tablet replica
I20250109 08:58:46.486804 20363 raft_consensus.cc:2238] T a54ca9f0d6e844e9a8b1d9cd0bb2a026 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:46.487177 20363 raft_consensus.cc:2267] T a54ca9f0d6e844e9a8b1d9cd0bb2a026 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:46.489187 20363 tablet_replica.cc:331] stopping tablet replica
I20250109 08:58:46.489717 20363 raft_consensus.cc:2238] T db111e41c1be497e8e673a2c42b9ba66 P 235ad6af04c34c88a5261daa3604fada [term 3 LEADER]: Raft consensus shutting down.
I20250109 08:58:46.490443 20363 raft_consensus.cc:2267] T db111e41c1be497e8e673a2c42b9ba66 P 235ad6af04c34c88a5261daa3604fada [term 3 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:46.492796 20363 tablet_replica.cc:331] stopping tablet replica
I20250109 08:58:46.493400 20363 raft_consensus.cc:2238] T d714271c7ea64c0b8b13fcf63591ec07 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:46.493773 20363 raft_consensus.cc:2267] T d714271c7ea64c0b8b13fcf63591ec07 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:46.496191 20363 tablet_replica.cc:331] stopping tablet replica
I20250109 08:58:46.496871 20363 raft_consensus.cc:2238] T d87d3b4755344b82b0aec7ffd1ce785c P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:46.497294 20363 raft_consensus.cc:2267] T d87d3b4755344b82b0aec7ffd1ce785c P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:46.499397 20363 tablet_replica.cc:331] stopping tablet replica
I20250109 08:58:46.499884 20363 raft_consensus.cc:2238] T 8d3947300cfc4523acdab19fc992f26a P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:46.500346 20363 raft_consensus.cc:2267] T 8d3947300cfc4523acdab19fc992f26a P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:46.502517 20363 tablet_replica.cc:331] stopping tablet replica
I20250109 08:58:46.503068 20363 raft_consensus.cc:2238] T 1340cf8ee2cb423f875276137da17e8b P 235ad6af04c34c88a5261daa3604fada [term 3 LEADER]: Raft consensus shutting down.
I20250109 08:58:46.503849 20363 raft_consensus.cc:2267] T 1340cf8ee2cb423f875276137da17e8b P 235ad6af04c34c88a5261daa3604fada [term 3 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:46.506023 20363 tablet_replica.cc:331] stopping tablet replica
I20250109 08:58:46.506547 20363 raft_consensus.cc:2238] T 4df64aae69ca41dc91286cae1603000f P 235ad6af04c34c88a5261daa3604fada [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:46.506970 20363 raft_consensus.cc:2267] T 4df64aae69ca41dc91286cae1603000f P 235ad6af04c34c88a5261daa3604fada [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:46.509145 20363 tablet_replica.cc:331] stopping tablet replica
I20250109 08:58:46.509860 20363 raft_consensus.cc:2238] T f1f9274aeda748f493f5752814fb262d P 235ad6af04c34c88a5261daa3604fada [term 2 FOLLOWER]: Raft consensus shutting down.
W20250109 08:58:46.511951 23346 consensus_peers.cc:487] T ee703ea575234c659da04f21ef3cdd70 P ed1efbce9ee3401e99a5260edeae4e7c -> Peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Couldn't send request to peer 235ad6af04c34c88a5261daa3604fada. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20250109 08:58:46.520480 23494 consensus_peers.cc:487] T d87d3b4755344b82b0aec7ffd1ce785c P 05a74e1eb2474b3d8b3a2da1ef127e01 -> Peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Couldn't send request to peer 235ad6af04c34c88a5261daa3604fada. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:46.520529 20363 raft_consensus.cc:2267] T f1f9274aeda748f493f5752814fb262d P 235ad6af04c34c88a5261daa3604fada [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:46.524086 20363 tablet_replica.cc:331] stopping tablet replica
I20250109 08:58:46.524864 20363 raft_consensus.cc:2238] T 444507f7812441ceaeebc44bc8f06293 P 235ad6af04c34c88a5261daa3604fada [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:46.525349 20363 raft_consensus.cc:2267] T 444507f7812441ceaeebc44bc8f06293 P 235ad6af04c34c88a5261daa3604fada [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:46.527921 20363 tablet_replica.cc:331] stopping tablet replica
I20250109 08:58:46.528729 20363 raft_consensus.cc:2238] T 5c37c58abaa54656b31c72160b4dec9e P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:46.529211 20363 raft_consensus.cc:2267] T 5c37c58abaa54656b31c72160b4dec9e P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:46.531706 20363 tablet_replica.cc:331] stopping tablet replica
I20250109 08:58:46.532488 20363 raft_consensus.cc:2238] T b572a6a3dfc04ea199bc703ddad8af67 P 235ad6af04c34c88a5261daa3604fada [term 2 LEADER]: Raft consensus shutting down.
I20250109 08:58:46.533416 20363 raft_consensus.cc:2267] T b572a6a3dfc04ea199bc703ddad8af67 P 235ad6af04c34c88a5261daa3604fada [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:46.536043 20363 tablet_replica.cc:331] stopping tablet replica
I20250109 08:58:46.536733 20363 raft_consensus.cc:2238] T ee703ea575234c659da04f21ef3cdd70 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:46.537140 20363 raft_consensus.cc:2267] T ee703ea575234c659da04f21ef3cdd70 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:46.539541 20363 tablet_replica.cc:331] stopping tablet replica
I20250109 08:58:46.540191 20363 raft_consensus.cc:2238] T 86b82e19759c4fd0b471a0dbff839aa5 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:46.540647 20363 raft_consensus.cc:2267] T 86b82e19759c4fd0b471a0dbff839aa5 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus is shut down!
W20250109 08:58:46.541798 23346 consensus_peers.cc:487] T f1f9274aeda748f493f5752814fb262d P ed1efbce9ee3401e99a5260edeae4e7c -> Peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Couldn't send request to peer 235ad6af04c34c88a5261daa3604fada. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:46.542825 20363 tablet_replica.cc:331] stopping tablet replica
I20250109 08:58:46.543387 20363 raft_consensus.cc:2238] T 732e4aa69aae44e9b1153ac330a01a96 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:46.543740 20363 raft_consensus.cc:2267] T 732e4aa69aae44e9b1153ac330a01a96 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus is shut down!
W20250109 08:58:46.544044 23494 consensus_peers.cc:487] T 24ce131f7bb143bea1581d5b0c2a7109 P 05a74e1eb2474b3d8b3a2da1ef127e01 -> Peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Couldn't send request to peer 235ad6af04c34c88a5261daa3604fada. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:46.545548 20363 tablet_replica.cc:331] stopping tablet replica
I20250109 08:58:46.546053 20363 raft_consensus.cc:2238] T 5ccc0d42d4a148d7819366cea0b754a7 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:46.546427 20363 raft_consensus.cc:2267] T 5ccc0d42d4a148d7819366cea0b754a7 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:46.548552 20363 tablet_replica.cc:331] stopping tablet replica
I20250109 08:58:46.549103 20363 raft_consensus.cc:2238] T e6b9aaeda9a947d4a87fedd774301cc0 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:46.549531 20363 raft_consensus.cc:2267] T e6b9aaeda9a947d4a87fedd774301cc0 P 235ad6af04c34c88a5261daa3604fada [term 1 FOLLOWER]: Raft consensus is shut down!
W20250109 08:58:46.549747 23346 consensus_peers.cc:487] T 08214ee9c4c14bda92bdc6fa8231c313 P ed1efbce9ee3401e99a5260edeae4e7c -> Peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Couldn't send request to peer 235ad6af04c34c88a5261daa3604fada. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20250109 08:58:46.588613 23494 consensus_peers.cc:487] T cc9886333c0343798226d59758a1ea5e P 05a74e1eb2474b3d8b3a2da1ef127e01 -> Peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Couldn't send request to peer 235ad6af04c34c88a5261daa3604fada. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20250109 08:58:46.592661 23420 consensus_peers.cc:487] T a09d8dffc2e446f1acd3c7f6cc1705bb P cf55383bef6649eb9e78cb2d2512dedc -> Peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Couldn't send request to peer 235ad6af04c34c88a5261daa3604fada. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20250109 08:58:46.600051 24109 connection.cc:441] server connection from 127.0.0.1:43338 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43338 (request call id 324) could send its response
W20250109 08:58:46.600766 23346 consensus_peers.cc:487] T a54ca9f0d6e844e9a8b1d9cd0bb2a026 P ed1efbce9ee3401e99a5260edeae4e7c -> Peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Couldn't send request to peer 235ad6af04c34c88a5261daa3604fada. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20250109 08:58:46.603901 23420 consensus_peers.cc:487] T 5ccc0d42d4a148d7819366cea0b754a7 P cf55383bef6649eb9e78cb2d2512dedc -> Peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Couldn't send request to peer 235ad6af04c34c88a5261daa3604fada. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.193:42691: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:46.620940 20363 tablet_server.cc:195] TabletServer@127.19.226.193:42691 shutdown complete.
W20250109 08:58:46.632452 23346 consensus_peers.cc:487] T 1c5b5beea4a840eb9677097cacee3408 P ed1efbce9ee3401e99a5260edeae4e7c -> Peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Couldn't send request to peer 235ad6af04c34c88a5261daa3604fada. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.193:42691: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20250109 08:58:46.633101 23346 consensus_peers.cc:487] T 1365130ef97c4581b623c22c39753831 P ed1efbce9ee3401e99a5260edeae4e7c -> Peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Couldn't send request to peer 235ad6af04c34c88a5261daa3604fada. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.193:42691: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20250109 08:58:46.673978 23494 consensus_peers.cc:487] T fc1038255dd64cb5ba2425037d22f1f7 P 05a74e1eb2474b3d8b3a2da1ef127e01 -> Peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Couldn't send request to peer 235ad6af04c34c88a5261daa3604fada. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.193:42691: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20250109 08:58:46.676070 23494 consensus_peers.cc:487] T 77597d68423749e7b87f9f0db7a1ff3b P 05a74e1eb2474b3d8b3a2da1ef127e01 -> Peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Couldn't send request to peer 235ad6af04c34c88a5261daa3604fada. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.193:42691: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20250109 08:58:46.684939 23494 consensus_peers.cc:487] T 41e7f0cc38c4414ab88c92c1cdeb3984 P 05a74e1eb2474b3d8b3a2da1ef127e01 -> Peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Couldn't send request to peer 235ad6af04c34c88a5261daa3604fada. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.193:42691: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:46.692305 20363 tablet_server.cc:178] TabletServer@127.19.226.194:0 shutting down...
W20250109 08:58:46.698966 23346 consensus_peers.cc:487] T 732e4aa69aae44e9b1153ac330a01a96 P ed1efbce9ee3401e99a5260edeae4e7c -> Peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Couldn't send request to peer 235ad6af04c34c88a5261daa3604fada. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.193:42691: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20250109 08:58:46.720223 23494 consensus_peers.cc:487] T 29828d9befb845d5bd49a80e018502e3 P 05a74e1eb2474b3d8b3a2da1ef127e01 -> Peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Couldn't send request to peer 235ad6af04c34c88a5261daa3604fada. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.193:42691: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:46.726630 20363 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250109 08:58:46.727414 20363 tablet_replica.cc:331] T 060daed1d3334c6188413dd381b82bf9 P ed1efbce9ee3401e99a5260edeae4e7c: stopping tablet replica
I20250109 08:58:46.728055 20363 raft_consensus.cc:2238] T 060daed1d3334c6188413dd381b82bf9 P ed1efbce9ee3401e99a5260edeae4e7c [term 3 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:46.728585 20363 raft_consensus.cc:2267] T 060daed1d3334c6188413dd381b82bf9 P ed1efbce9ee3401e99a5260edeae4e7c [term 3 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:46.730854 20363 tablet_replica.cc:331] T 444507f7812441ceaeebc44bc8f06293 P ed1efbce9ee3401e99a5260edeae4e7c: stopping tablet replica
I20250109 08:58:46.731621 20363 raft_consensus.cc:2238] T 444507f7812441ceaeebc44bc8f06293 P ed1efbce9ee3401e99a5260edeae4e7c [term 2 FOLLOWER]: Raft consensus shutting down.
W20250109 08:58:46.732036 23420 consensus_peers.cc:487] T 940653b6d55c46439827697b9aadf16b P cf55383bef6649eb9e78cb2d2512dedc -> Peer ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365): Couldn't send request to peer ed1efbce9ee3401e99a5260edeae4e7c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:46.732117 20363 raft_consensus.cc:2267] T 444507f7812441ceaeebc44bc8f06293 P ed1efbce9ee3401e99a5260edeae4e7c [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:46.734275 20363 tablet_replica.cc:331] T 0dab6c23ce444dda9427901a98f07b15 P ed1efbce9ee3401e99a5260edeae4e7c: stopping tablet replica
W20250109 08:58:46.734637 23420 consensus_peers.cc:487] T 5831288451724d4ca1114ec5e5f01b78 P cf55383bef6649eb9e78cb2d2512dedc -> Peer ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365): Couldn't send request to peer ed1efbce9ee3401e99a5260edeae4e7c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:46.734855 20363 raft_consensus.cc:2238] T 0dab6c23ce444dda9427901a98f07b15 P ed1efbce9ee3401e99a5260edeae4e7c [term 3 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:46.735349 20363 raft_consensus.cc:2267] T 0dab6c23ce444dda9427901a98f07b15 P ed1efbce9ee3401e99a5260edeae4e7c [term 3 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:46.737668 20363 tablet_replica.cc:331] T 1ac4317290b24209a76d2e19e68ee45b P ed1efbce9ee3401e99a5260edeae4e7c: stopping tablet replica
I20250109 08:58:46.738240 20363 raft_consensus.cc:2238] T 1ac4317290b24209a76d2e19e68ee45b P ed1efbce9ee3401e99a5260edeae4e7c [term 4 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:46.738631 20363 raft_consensus.cc:2267] T 1ac4317290b24209a76d2e19e68ee45b P ed1efbce9ee3401e99a5260edeae4e7c [term 4 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:46.741072 20363 tablet_replica.cc:331] T 8d3947300cfc4523acdab19fc992f26a P ed1efbce9ee3401e99a5260edeae4e7c: stopping tablet replica
I20250109 08:58:46.741827 20363 raft_consensus.cc:2238] T 8d3947300cfc4523acdab19fc992f26a P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:46.742350 20363 raft_consensus.cc:2267] T 8d3947300cfc4523acdab19fc992f26a P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:46.744300 20363 tablet_replica.cc:331] T d87d3b4755344b82b0aec7ffd1ce785c P ed1efbce9ee3401e99a5260edeae4e7c: stopping tablet replica
I20250109 08:58:46.744895 20363 raft_consensus.cc:2238] T d87d3b4755344b82b0aec7ffd1ce785c P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:46.745287 20363 raft_consensus.cc:2267] T d87d3b4755344b82b0aec7ffd1ce785c P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:46.746965 20363 tablet_replica.cc:331] T 1365130ef97c4581b623c22c39753831 P ed1efbce9ee3401e99a5260edeae4e7c: stopping tablet replica
I20250109 08:58:46.747511 20363 raft_consensus.cc:2238] T 1365130ef97c4581b623c22c39753831 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 LEADER]: Raft consensus shutting down.
I20250109 08:58:46.748229 20363 raft_consensus.cc:2267] T 1365130ef97c4581b623c22c39753831 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:46.749963 20363 tablet_replica.cc:331] T 1f9c6525267145d6bb73dde32ed9b7d4 P ed1efbce9ee3401e99a5260edeae4e7c: stopping tablet replica
I20250109 08:58:46.750490 20363 raft_consensus.cc:2238] T 1f9c6525267145d6bb73dde32ed9b7d4 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:46.750874 20363 raft_consensus.cc:2267] T 1f9c6525267145d6bb73dde32ed9b7d4 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:46.752557 20363 tablet_replica.cc:331] T ecc833ad1afa4079a75e41b1a807c923 P ed1efbce9ee3401e99a5260edeae4e7c: stopping tablet replica
I20250109 08:58:46.753064 20363 raft_consensus.cc:2238] T ecc833ad1afa4079a75e41b1a807c923 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:46.753439 20363 raft_consensus.cc:2267] T ecc833ad1afa4079a75e41b1a807c923 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:46.755244 20363 tablet_replica.cc:331] T 732e4aa69aae44e9b1153ac330a01a96 P ed1efbce9ee3401e99a5260edeae4e7c: stopping tablet replica
I20250109 08:58:46.755738 20363 raft_consensus.cc:2238] T 732e4aa69aae44e9b1153ac330a01a96 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 LEADER]: Raft consensus shutting down.
I20250109 08:58:46.756523 20363 raft_consensus.cc:2267] T 732e4aa69aae44e9b1153ac330a01a96 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:46.758258 20363 tablet_replica.cc:331] T ee703ea575234c659da04f21ef3cdd70 P ed1efbce9ee3401e99a5260edeae4e7c: stopping tablet replica
I20250109 08:58:46.758884 20363 raft_consensus.cc:2238] T ee703ea575234c659da04f21ef3cdd70 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 LEADER]: Raft consensus shutting down.
I20250109 08:58:46.759768 20363 raft_consensus.cc:2267] T ee703ea575234c659da04f21ef3cdd70 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:46.761670 20363 tablet_replica.cc:331] T 86b82e19759c4fd0b471a0dbff839aa5 P ed1efbce9ee3401e99a5260edeae4e7c: stopping tablet replica
I20250109 08:58:46.762161 20363 raft_consensus.cc:2238] T 86b82e19759c4fd0b471a0dbff839aa5 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 LEADER]: Raft consensus shutting down.
I20250109 08:58:46.762859 20363 raft_consensus.cc:2267] T 86b82e19759c4fd0b471a0dbff839aa5 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:46.764631 20363 tablet_replica.cc:331] T de98c70e25494d2a8e581443be836bd3 P ed1efbce9ee3401e99a5260edeae4e7c: stopping tablet replica
I20250109 08:58:46.765151 20363 raft_consensus.cc:2238] T de98c70e25494d2a8e581443be836bd3 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:46.765525 20363 raft_consensus.cc:2267] T de98c70e25494d2a8e581443be836bd3 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:46.767249 20363 tablet_replica.cc:331] T fc1038255dd64cb5ba2425037d22f1f7 P ed1efbce9ee3401e99a5260edeae4e7c: stopping tablet replica
I20250109 08:58:46.767748 20363 raft_consensus.cc:2238] T fc1038255dd64cb5ba2425037d22f1f7 P ed1efbce9ee3401e99a5260edeae4e7c [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:46.768103 20363 raft_consensus.cc:2267] T fc1038255dd64cb5ba2425037d22f1f7 P ed1efbce9ee3401e99a5260edeae4e7c [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:46.769794 20363 tablet_replica.cc:331] T a09d8dffc2e446f1acd3c7f6cc1705bb P ed1efbce9ee3401e99a5260edeae4e7c: stopping tablet replica
I20250109 08:58:46.770298 20363 raft_consensus.cc:2238] T a09d8dffc2e446f1acd3c7f6cc1705bb P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:46.770659 20363 raft_consensus.cc:2267] T a09d8dffc2e446f1acd3c7f6cc1705bb P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:46.772639 20363 tablet_replica.cc:331] T 08214ee9c4c14bda92bdc6fa8231c313 P ed1efbce9ee3401e99a5260edeae4e7c: stopping tablet replica
I20250109 08:58:46.773216 20363 raft_consensus.cc:2238] T 08214ee9c4c14bda92bdc6fa8231c313 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 LEADER]: Raft consensus shutting down.
I20250109 08:58:46.774132 20363 raft_consensus.cc:2267] T 08214ee9c4c14bda92bdc6fa8231c313 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Raft consensus is shut down!
W20250109 08:58:46.775918 23494 consensus_peers.cc:487] T ecc833ad1afa4079a75e41b1a807c923 P 05a74e1eb2474b3d8b3a2da1ef127e01 -> Peer ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365): Couldn't send request to peer ed1efbce9ee3401e99a5260edeae4e7c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:46.776680 20363 tablet_replica.cc:331] T 750dd1e1055f4570b0aecdda3ebe8772 P ed1efbce9ee3401e99a5260edeae4e7c: stopping tablet replica
I20250109 08:58:46.777455 20363 raft_consensus.cc:2238] T 750dd1e1055f4570b0aecdda3ebe8772 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 LEADER]: Raft consensus shutting down.
I20250109 08:58:46.778318 20363 raft_consensus.cc:2267] T 750dd1e1055f4570b0aecdda3ebe8772 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:46.780762 20363 tablet_replica.cc:331] T e6b9aaeda9a947d4a87fedd774301cc0 P ed1efbce9ee3401e99a5260edeae4e7c: stopping tablet replica
I20250109 08:58:46.781348 20363 raft_consensus.cc:2238] T e6b9aaeda9a947d4a87fedd774301cc0 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Raft consensus shutting down.
W20250109 08:58:46.781553 23494 consensus_peers.cc:487] T fc1038255dd64cb5ba2425037d22f1f7 P 05a74e1eb2474b3d8b3a2da1ef127e01 -> Peer ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365): Couldn't send request to peer ed1efbce9ee3401e99a5260edeae4e7c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:46.781805 20363 raft_consensus.cc:2267] T e6b9aaeda9a947d4a87fedd774301cc0 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:46.783715 20363 tablet_replica.cc:331] T d714271c7ea64c0b8b13fcf63591ec07 P ed1efbce9ee3401e99a5260edeae4e7c: stopping tablet replica
I20250109 08:58:46.784219 20363 raft_consensus.cc:2238] T d714271c7ea64c0b8b13fcf63591ec07 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 LEADER]: Raft consensus shutting down.
I20250109 08:58:46.784950 20363 raft_consensus.cc:2267] T d714271c7ea64c0b8b13fcf63591ec07 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:46.786605 20363 tablet_replica.cc:331] T db111e41c1be497e8e673a2c42b9ba66 P ed1efbce9ee3401e99a5260edeae4e7c: stopping tablet replica
I20250109 08:58:46.787171 20363 raft_consensus.cc:2238] T db111e41c1be497e8e673a2c42b9ba66 P ed1efbce9ee3401e99a5260edeae4e7c [term 3 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:46.787562 20363 raft_consensus.cc:2267] T db111e41c1be497e8e673a2c42b9ba66 P ed1efbce9ee3401e99a5260edeae4e7c [term 3 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:46.790004 20363 tablet_replica.cc:331] T 6eef9bb3c645428d9f1cb843bae9a264 P ed1efbce9ee3401e99a5260edeae4e7c: stopping tablet replica
I20250109 08:58:46.790649 20363 raft_consensus.cc:2238] T 6eef9bb3c645428d9f1cb843bae9a264 P ed1efbce9ee3401e99a5260edeae4e7c [term 2 FOLLOWER]: Raft consensus shutting down.
W20250109 08:58:46.790928 23346 consensus_peers.cc:487] T 3208a99957554f6db341654856980c8a P ed1efbce9ee3401e99a5260edeae4e7c -> Peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Couldn't send request to peer 235ad6af04c34c88a5261daa3604fada. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.193:42691: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:46.791085 20363 raft_consensus.cc:2267] T 6eef9bb3c645428d9f1cb843bae9a264 P ed1efbce9ee3401e99a5260edeae4e7c [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:46.793006 20363 tablet_replica.cc:331] T a54ca9f0d6e844e9a8b1d9cd0bb2a026 P ed1efbce9ee3401e99a5260edeae4e7c: stopping tablet replica
I20250109 08:58:46.793494 20363 raft_consensus.cc:2238] T a54ca9f0d6e844e9a8b1d9cd0bb2a026 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 LEADER]: Raft consensus shutting down.
I20250109 08:58:46.794178 20363 raft_consensus.cc:2267] T a54ca9f0d6e844e9a8b1d9cd0bb2a026 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:46.795818 20363 tablet_replica.cc:331] T a686bf4d947c4b638049e6b88c59ae09 P ed1efbce9ee3401e99a5260edeae4e7c: stopping tablet replica
I20250109 08:58:46.796334 20363 raft_consensus.cc:2238] T a686bf4d947c4b638049e6b88c59ae09 P ed1efbce9ee3401e99a5260edeae4e7c [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:46.796741 20363 raft_consensus.cc:2267] T a686bf4d947c4b638049e6b88c59ae09 P ed1efbce9ee3401e99a5260edeae4e7c [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:46.798389 20363 tablet_replica.cc:331] T 8c5e3d0155d140c4b32f162d3173f5c9 P ed1efbce9ee3401e99a5260edeae4e7c: stopping tablet replica
I20250109 08:58:46.798879 20363 raft_consensus.cc:2238] T 8c5e3d0155d140c4b32f162d3173f5c9 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:46.799237 20363 raft_consensus.cc:2267] T 8c5e3d0155d140c4b32f162d3173f5c9 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:46.800921 20363 tablet_replica.cc:331] T 1c5b5beea4a840eb9677097cacee3408 P ed1efbce9ee3401e99a5260edeae4e7c: stopping tablet replica
I20250109 08:58:46.801429 20363 raft_consensus.cc:2238] T 1c5b5beea4a840eb9677097cacee3408 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 LEADER]: Raft consensus shutting down.
I20250109 08:58:46.802223 20363 raft_consensus.cc:2267] T 1c5b5beea4a840eb9677097cacee3408 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:46.804293 20363 tablet_replica.cc:331] T 5831288451724d4ca1114ec5e5f01b78 P ed1efbce9ee3401e99a5260edeae4e7c: stopping tablet replica
I20250109 08:58:46.804960 20363 raft_consensus.cc:2238] T 5831288451724d4ca1114ec5e5f01b78 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:46.805364 20363 raft_consensus.cc:2267] T 5831288451724d4ca1114ec5e5f01b78 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Raft consensus is shut down!
W20250109 08:58:46.806733 23420 consensus_peers.cc:487] T e6b9aaeda9a947d4a87fedd774301cc0 P cf55383bef6649eb9e78cb2d2512dedc -> Peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Couldn't send request to peer 235ad6af04c34c88a5261daa3604fada. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.193:42691: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:46.807611 20363 tablet_replica.cc:331] T 41e7f0cc38c4414ab88c92c1cdeb3984 P ed1efbce9ee3401e99a5260edeae4e7c: stopping tablet replica
I20250109 08:58:46.808452 20363 raft_consensus.cc:2238] T 41e7f0cc38c4414ab88c92c1cdeb3984 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:46.808931 20363 raft_consensus.cc:2267] T 41e7f0cc38c4414ab88c92c1cdeb3984 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:46.811112 20363 tablet_replica.cc:331] T f1f9274aeda748f493f5752814fb262d P ed1efbce9ee3401e99a5260edeae4e7c: stopping tablet replica
I20250109 08:58:46.811789 20363 raft_consensus.cc:2238] T f1f9274aeda748f493f5752814fb262d P ed1efbce9ee3401e99a5260edeae4e7c [term 2 LEADER]: Raft consensus shutting down.
W20250109 08:58:46.812638 23420 consensus_peers.cc:487] T 8de6b2fb09bc46ce9e4033af06dffd04 P cf55383bef6649eb9e78cb2d2512dedc -> Peer ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365): Couldn't send request to peer ed1efbce9ee3401e99a5260edeae4e7c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:46.812753 20363 raft_consensus.cc:2267] T f1f9274aeda748f493f5752814fb262d P ed1efbce9ee3401e99a5260edeae4e7c [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:46.815161 20363 tablet_replica.cc:331] T 1340cf8ee2cb423f875276137da17e8b P ed1efbce9ee3401e99a5260edeae4e7c: stopping tablet replica
W20250109 08:58:46.815299 23494 consensus_peers.cc:487] T 1ac4317290b24209a76d2e19e68ee45b P 05a74e1eb2474b3d8b3a2da1ef127e01 -> Peer ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365): Couldn't send request to peer ed1efbce9ee3401e99a5260edeae4e7c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:46.815853 20363 raft_consensus.cc:2238] T 1340cf8ee2cb423f875276137da17e8b P ed1efbce9ee3401e99a5260edeae4e7c [term 3 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:46.816260 20363 raft_consensus.cc:2267] T 1340cf8ee2cb423f875276137da17e8b P ed1efbce9ee3401e99a5260edeae4e7c [term 3 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:46.818454 20363 tablet_replica.cc:331] T 4df64aae69ca41dc91286cae1603000f P ed1efbce9ee3401e99a5260edeae4e7c: stopping tablet replica
I20250109 08:58:46.819095 20363 raft_consensus.cc:2238] T 4df64aae69ca41dc91286cae1603000f P ed1efbce9ee3401e99a5260edeae4e7c [term 2 LEADER]: Raft consensus shutting down.
I20250109 08:58:46.819825 20363 raft_consensus.cc:2267] T 4df64aae69ca41dc91286cae1603000f P ed1efbce9ee3401e99a5260edeae4e7c [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:46.821625 20363 tablet_replica.cc:331] T 2ab70546dda143b5b5db15c3875aa348 P ed1efbce9ee3401e99a5260edeae4e7c: stopping tablet replica
I20250109 08:58:46.822110 20363 raft_consensus.cc:2238] T 2ab70546dda143b5b5db15c3875aa348 P ed1efbce9ee3401e99a5260edeae4e7c [term 3 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:46.822501 20363 raft_consensus.cc:2267] T 2ab70546dda143b5b5db15c3875aa348 P ed1efbce9ee3401e99a5260edeae4e7c [term 3 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:46.824131 20363 tablet_replica.cc:331] T ed79047393d2425f8ecff31ea08b9cad P ed1efbce9ee3401e99a5260edeae4e7c: stopping tablet replica
I20250109 08:58:46.824757 20363 raft_consensus.cc:2238] T ed79047393d2425f8ecff31ea08b9cad P ed1efbce9ee3401e99a5260edeae4e7c [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:46.825224 20363 raft_consensus.cc:2267] T ed79047393d2425f8ecff31ea08b9cad P ed1efbce9ee3401e99a5260edeae4e7c [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:46.827806 20363 tablet_replica.cc:331] T 940653b6d55c46439827697b9aadf16b P ed1efbce9ee3401e99a5260edeae4e7c: stopping tablet replica
I20250109 08:58:46.828517 20363 raft_consensus.cc:2238] T 940653b6d55c46439827697b9aadf16b P ed1efbce9ee3401e99a5260edeae4e7c [term 3 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:46.828979 20363 raft_consensus.cc:2267] T 940653b6d55c46439827697b9aadf16b P ed1efbce9ee3401e99a5260edeae4e7c [term 3 FOLLOWER]: Raft consensus is shut down!
W20250109 08:58:46.829219 23494 consensus_peers.cc:487] T d87d3b4755344b82b0aec7ffd1ce785c P 05a74e1eb2474b3d8b3a2da1ef127e01 -> Peer ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365): Couldn't send request to peer ed1efbce9ee3401e99a5260edeae4e7c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:46.830864 20363 tablet_replica.cc:331] T 11aa76a3a30643708fececed660a5250 P ed1efbce9ee3401e99a5260edeae4e7c: stopping tablet replica
I20250109 08:58:46.831717 20363 raft_consensus.cc:2238] T 11aa76a3a30643708fececed660a5250 P ed1efbce9ee3401e99a5260edeae4e7c [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:46.832125 20363 raft_consensus.cc:2267] T 11aa76a3a30643708fececed660a5250 P ed1efbce9ee3401e99a5260edeae4e7c [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:46.834193 20363 tablet_replica.cc:331] T 86d4074509744900abe94402467a7d85 P ed1efbce9ee3401e99a5260edeae4e7c: stopping tablet replica
I20250109 08:58:46.834861 20363 raft_consensus.cc:2238] T 86d4074509744900abe94402467a7d85 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 LEADER]: Raft consensus shutting down.
I20250109 08:58:46.835757 20363 raft_consensus.cc:2267] T 86d4074509744900abe94402467a7d85 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Raft consensus is shut down!
W20250109 08:58:46.837638 23494 consensus_peers.cc:487] T 94f2be307f704c75bceb213e52307cfb P 05a74e1eb2474b3d8b3a2da1ef127e01 -> Peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Couldn't send request to peer 235ad6af04c34c88a5261daa3604fada. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.193:42691: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:46.838076 20363 tablet_replica.cc:331] T 2e3d34c0d09843d2be54e47f3ccc16d8 P ed1efbce9ee3401e99a5260edeae4e7c: stopping tablet replica
I20250109 08:58:46.838704 20363 raft_consensus.cc:2238] T 2e3d34c0d09843d2be54e47f3ccc16d8 P ed1efbce9ee3401e99a5260edeae4e7c [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:46.839046 20363 raft_consensus.cc:2267] T 2e3d34c0d09843d2be54e47f3ccc16d8 P ed1efbce9ee3401e99a5260edeae4e7c [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:46.840886 20363 tablet_replica.cc:331] T 3208a99957554f6db341654856980c8a P ed1efbce9ee3401e99a5260edeae4e7c: stopping tablet replica
I20250109 08:58:46.841439 20363 raft_consensus.cc:2238] T 3208a99957554f6db341654856980c8a P ed1efbce9ee3401e99a5260edeae4e7c [term 2 LEADER]: Raft consensus shutting down.
I20250109 08:58:46.842370 20363 raft_consensus.cc:2267] T 3208a99957554f6db341654856980c8a P ed1efbce9ee3401e99a5260edeae4e7c [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:46.844740 20363 tablet_replica.cc:331] T 8de6b2fb09bc46ce9e4033af06dffd04 P ed1efbce9ee3401e99a5260edeae4e7c: stopping tablet replica
I20250109 08:58:46.845427 20363 raft_consensus.cc:2238] T 8de6b2fb09bc46ce9e4033af06dffd04 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:46.845897 20363 raft_consensus.cc:2267] T 8de6b2fb09bc46ce9e4033af06dffd04 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Raft consensus is shut down!
W20250109 08:58:46.846899 23420 consensus_peers.cc:487] T 5c37c58abaa54656b31c72160b4dec9e P cf55383bef6649eb9e78cb2d2512dedc -> Peer 235ad6af04c34c88a5261daa3604fada (127.19.226.193:42691): Couldn't send request to peer 235ad6af04c34c88a5261daa3604fada. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.193:42691: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:46.848297 20363 tablet_replica.cc:331] T 34caa5988a734b65a2eb512747d833b7 P ed1efbce9ee3401e99a5260edeae4e7c: stopping tablet replica
I20250109 08:58:46.849033 20363 raft_consensus.cc:2238] T 34caa5988a734b65a2eb512747d833b7 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 LEADER]: Raft consensus shutting down.
I20250109 08:58:46.849959 20363 raft_consensus.cc:2267] T 34caa5988a734b65a2eb512747d833b7 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:46.851718 20363 tablet_replica.cc:331] T 29828d9befb845d5bd49a80e018502e3 P ed1efbce9ee3401e99a5260edeae4e7c: stopping tablet replica
I20250109 08:58:46.852178 20363 raft_consensus.cc:2238] T 29828d9befb845d5bd49a80e018502e3 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:46.852618 20363 raft_consensus.cc:2267] T 29828d9befb845d5bd49a80e018502e3 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:46.854957 20363 tablet_replica.cc:331] T 9b0bbf2e4a424fb5ac13a3b55627b048 P ed1efbce9ee3401e99a5260edeae4e7c: stopping tablet replica
I20250109 08:58:46.855623 20363 raft_consensus.cc:2238] T 9b0bbf2e4a424fb5ac13a3b55627b048 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 LEADER]: Raft consensus shutting down.
I20250109 08:58:46.856609 20363 raft_consensus.cc:2267] T 9b0bbf2e4a424fb5ac13a3b55627b048 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Raft consensus is shut down!
W20250109 08:58:46.857298 23420 consensus_peers.cc:487] T 8d3947300cfc4523acdab19fc992f26a P cf55383bef6649eb9e78cb2d2512dedc -> Peer ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365): Couldn't send request to peer ed1efbce9ee3401e99a5260edeae4e7c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:46.858994 20363 tablet_replica.cc:331] T d5fa255ab5f24750a0afa6d81ec869f2 P ed1efbce9ee3401e99a5260edeae4e7c: stopping tablet replica
I20250109 08:58:46.859782 20363 raft_consensus.cc:2238] T d5fa255ab5f24750a0afa6d81ec869f2 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:46.860222 20363 raft_consensus.cc:2267] T d5fa255ab5f24750a0afa6d81ec869f2 P ed1efbce9ee3401e99a5260edeae4e7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:46.862721 20363 tablet_replica.cc:331] T ea2b0762940e44dbbf69b9a163ace846 P ed1efbce9ee3401e99a5260edeae4e7c: stopping tablet replica
I20250109 08:58:46.863351 20363 raft_consensus.cc:2238] T ea2b0762940e44dbbf69b9a163ace846 P ed1efbce9ee3401e99a5260edeae4e7c [term 3 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:46.863806 20363 raft_consensus.cc:2267] T ea2b0762940e44dbbf69b9a163ace846 P ed1efbce9ee3401e99a5260edeae4e7c [term 3 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:46.866370 20363 tablet_replica.cc:331] T 970a82e754db47b6b1a19551d4ba43d8 P ed1efbce9ee3401e99a5260edeae4e7c: stopping tablet replica
I20250109 08:58:46.866935 20363 raft_consensus.cc:2238] T 970a82e754db47b6b1a19551d4ba43d8 P ed1efbce9ee3401e99a5260edeae4e7c [term 3 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:46.867368 20363 raft_consensus.cc:2267] T 970a82e754db47b6b1a19551d4ba43d8 P ed1efbce9ee3401e99a5260edeae4e7c [term 3 FOLLOWER]: Raft consensus is shut down!
W20250109 08:58:46.868340 23494 consensus_peers.cc:487] T 11aa76a3a30643708fececed660a5250 P 05a74e1eb2474b3d8b3a2da1ef127e01 -> Peer ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365): Couldn't send request to peer ed1efbce9ee3401e99a5260edeae4e7c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20250109 08:58:46.884820 23494 consensus_peers.cc:487] T 29828d9befb845d5bd49a80e018502e3 P 05a74e1eb2474b3d8b3a2da1ef127e01 -> Peer ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365): Couldn't send request to peer ed1efbce9ee3401e99a5260edeae4e7c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20250109 08:58:46.898267 23494 consensus_peers.cc:487] T 41e7f0cc38c4414ab88c92c1cdeb3984 P 05a74e1eb2474b3d8b3a2da1ef127e01 -> Peer ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365): Couldn't send request to peer ed1efbce9ee3401e99a5260edeae4e7c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20250109 08:58:46.924731 23420 consensus_peers.cc:487] T 8c5e3d0155d140c4b32f162d3173f5c9 P cf55383bef6649eb9e78cb2d2512dedc -> Peer ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365): Couldn't send request to peer ed1efbce9ee3401e99a5260edeae4e7c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:46.934495 20363 tablet_server.cc:195] TabletServer@127.19.226.194:0 shutdown complete.
W20250109 08:58:46.983721 23420 consensus_peers.cc:487] T a686bf4d947c4b638049e6b88c59ae09 P cf55383bef6649eb9e78cb2d2512dedc -> Peer ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365): Couldn't send request to peer ed1efbce9ee3401e99a5260edeae4e7c. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.194:41365: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20250109 08:58:46.993023 23420 consensus_peers.cc:487] T de98c70e25494d2a8e581443be836bd3 P cf55383bef6649eb9e78cb2d2512dedc -> Peer ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365): Couldn't send request to peer ed1efbce9ee3401e99a5260edeae4e7c. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.194:41365: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:46.999927 20363 tablet_server.cc:178] TabletServer@127.19.226.195:0 shutting down...
W20250109 08:58:47.002975 23420 consensus_peers.cc:487] T 1f9c6525267145d6bb73dde32ed9b7d4 P cf55383bef6649eb9e78cb2d2512dedc -> Peer ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365): Couldn't send request to peer ed1efbce9ee3401e99a5260edeae4e7c. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.194:41365: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20250109 08:58:47.024441 23420 consensus_peers.cc:487] T a09d8dffc2e446f1acd3c7f6cc1705bb P cf55383bef6649eb9e78cb2d2512dedc -> Peer ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365): Couldn't send request to peer ed1efbce9ee3401e99a5260edeae4e7c. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.194:41365: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:47.046224 20363 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250109 08:58:47.047037 20363 tablet_replica.cc:331] T bfb140bb3a8b44bd8b4e1fcf0946162b P cf55383bef6649eb9e78cb2d2512dedc: stopping tablet replica
I20250109 08:58:47.047814 20363 raft_consensus.cc:2238] T bfb140bb3a8b44bd8b4e1fcf0946162b P cf55383bef6649eb9e78cb2d2512dedc [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:47.048336 20363 raft_consensus.cc:2267] T bfb140bb3a8b44bd8b4e1fcf0946162b P cf55383bef6649eb9e78cb2d2512dedc [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:47.050205 20363 tablet_replica.cc:331] T 8d3947300cfc4523acdab19fc992f26a P cf55383bef6649eb9e78cb2d2512dedc: stopping tablet replica
I20250109 08:58:47.050736 20363 raft_consensus.cc:2238] T 8d3947300cfc4523acdab19fc992f26a P cf55383bef6649eb9e78cb2d2512dedc [term 1 LEADER]: Raft consensus shutting down.
I20250109 08:58:47.051437 20363 raft_consensus.cc:2267] T 8d3947300cfc4523acdab19fc992f26a P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:47.053246 20363 tablet_replica.cc:331] T 1ac4317290b24209a76d2e19e68ee45b P cf55383bef6649eb9e78cb2d2512dedc: stopping tablet replica
I20250109 08:58:47.053726 20363 raft_consensus.cc:2238] T 1ac4317290b24209a76d2e19e68ee45b P cf55383bef6649eb9e78cb2d2512dedc [term 4 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:47.054064 20363 raft_consensus.cc:2267] T 1ac4317290b24209a76d2e19e68ee45b P cf55383bef6649eb9e78cb2d2512dedc [term 4 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:47.055778 20363 tablet_replica.cc:331] T ecc833ad1afa4079a75e41b1a807c923 P cf55383bef6649eb9e78cb2d2512dedc: stopping tablet replica
I20250109 08:58:47.056516 20363 raft_consensus.cc:2238] T ecc833ad1afa4079a75e41b1a807c923 P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:47.056926 20363 raft_consensus.cc:2267] T ecc833ad1afa4079a75e41b1a807c923 P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Raft consensus is shut down!
W20250109 08:58:47.058888 23420 consensus_peers.cc:487] T d5fa255ab5f24750a0afa6d81ec869f2 P cf55383bef6649eb9e78cb2d2512dedc -> Peer ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365): Couldn't send request to peer ed1efbce9ee3401e99a5260edeae4e7c. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.194:41365: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:47.059263 20363 tablet_replica.cc:331] T 1f9c6525267145d6bb73dde32ed9b7d4 P cf55383bef6649eb9e78cb2d2512dedc: stopping tablet replica
I20250109 08:58:47.059947 20363 raft_consensus.cc:2238] T 1f9c6525267145d6bb73dde32ed9b7d4 P cf55383bef6649eb9e78cb2d2512dedc [term 1 LEADER]: Raft consensus shutting down.
I20250109 08:58:47.060693 20363 raft_consensus.cc:2267] T 1f9c6525267145d6bb73dde32ed9b7d4 P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:47.062420 20363 tablet_replica.cc:331] T 950e07e9cc794b8ab9ec5bd97d7e8e59 P cf55383bef6649eb9e78cb2d2512dedc: stopping tablet replica
I20250109 08:58:47.062968 20363 raft_consensus.cc:2238] T 950e07e9cc794b8ab9ec5bd97d7e8e59 P cf55383bef6649eb9e78cb2d2512dedc [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:47.063326 20363 raft_consensus.cc:2267] T 950e07e9cc794b8ab9ec5bd97d7e8e59 P cf55383bef6649eb9e78cb2d2512dedc [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:47.064976 20363 tablet_replica.cc:331] T 1365130ef97c4581b623c22c39753831 P cf55383bef6649eb9e78cb2d2512dedc: stopping tablet replica
I20250109 08:58:47.065486 20363 raft_consensus.cc:2238] T 1365130ef97c4581b623c22c39753831 P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:47.065848 20363 raft_consensus.cc:2267] T 1365130ef97c4581b623c22c39753831 P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:47.067545 20363 tablet_replica.cc:331] T a7d2ab61d238471c9f287ac1d5db4ed0 P cf55383bef6649eb9e78cb2d2512dedc: stopping tablet replica
I20250109 08:58:47.068058 20363 raft_consensus.cc:2238] T a7d2ab61d238471c9f287ac1d5db4ed0 P cf55383bef6649eb9e78cb2d2512dedc [term 1 LEADER]: Raft consensus shutting down.
I20250109 08:58:47.068769 20363 raft_consensus.cc:2267] T a7d2ab61d238471c9f287ac1d5db4ed0 P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:47.070467 20363 tablet_replica.cc:331] T 94f2be307f704c75bceb213e52307cfb P cf55383bef6649eb9e78cb2d2512dedc: stopping tablet replica
I20250109 08:58:47.070997 20363 raft_consensus.cc:2238] T 94f2be307f704c75bceb213e52307cfb P cf55383bef6649eb9e78cb2d2512dedc [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:47.071365 20363 raft_consensus.cc:2267] T 94f2be307f704c75bceb213e52307cfb P cf55383bef6649eb9e78cb2d2512dedc [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:47.073097 20363 tablet_replica.cc:331] T e5251927614642639bd46ecd5373d2a4 P cf55383bef6649eb9e78cb2d2512dedc: stopping tablet replica
I20250109 08:58:47.073625 20363 raft_consensus.cc:2238] T e5251927614642639bd46ecd5373d2a4 P cf55383bef6649eb9e78cb2d2512dedc [term 1 LEADER]: Raft consensus shutting down.
I20250109 08:58:47.074331 20363 raft_consensus.cc:2267] T e5251927614642639bd46ecd5373d2a4 P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:47.076076 20363 tablet_replica.cc:331] T 24ce131f7bb143bea1581d5b0c2a7109 P cf55383bef6649eb9e78cb2d2512dedc: stopping tablet replica
I20250109 08:58:47.076642 20363 raft_consensus.cc:2238] T 24ce131f7bb143bea1581d5b0c2a7109 P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:47.077032 20363 raft_consensus.cc:2267] T 24ce131f7bb143bea1581d5b0c2a7109 P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:47.078807 20363 tablet_replica.cc:331] T 5f0d500a91b54851bda52e5ccf741fd6 P cf55383bef6649eb9e78cb2d2512dedc: stopping tablet replica
I20250109 08:58:47.079327 20363 raft_consensus.cc:2238] T 5f0d500a91b54851bda52e5ccf741fd6 P cf55383bef6649eb9e78cb2d2512dedc [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:47.079681 20363 raft_consensus.cc:2267] T 5f0d500a91b54851bda52e5ccf741fd6 P cf55383bef6649eb9e78cb2d2512dedc [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:47.081987 20363 tablet_replica.cc:331] T a09d8dffc2e446f1acd3c7f6cc1705bb P cf55383bef6649eb9e78cb2d2512dedc: stopping tablet replica
I20250109 08:58:47.082592 20363 raft_consensus.cc:2238] T a09d8dffc2e446f1acd3c7f6cc1705bb P cf55383bef6649eb9e78cb2d2512dedc [term 1 LEADER]: Raft consensus shutting down.
I20250109 08:58:47.083284 20363 raft_consensus.cc:2267] T a09d8dffc2e446f1acd3c7f6cc1705bb P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:47.085003 20363 tablet_replica.cc:331] T 750dd1e1055f4570b0aecdda3ebe8772 P cf55383bef6649eb9e78cb2d2512dedc: stopping tablet replica
I20250109 08:58:47.085518 20363 raft_consensus.cc:2238] T 750dd1e1055f4570b0aecdda3ebe8772 P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:47.085863 20363 raft_consensus.cc:2267] T 750dd1e1055f4570b0aecdda3ebe8772 P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:47.087495 20363 tablet_replica.cc:331] T e6b9aaeda9a947d4a87fedd774301cc0 P cf55383bef6649eb9e78cb2d2512dedc: stopping tablet replica
I20250109 08:58:47.087988 20363 raft_consensus.cc:2238] T e6b9aaeda9a947d4a87fedd774301cc0 P cf55383bef6649eb9e78cb2d2512dedc [term 1 LEADER]: Raft consensus shutting down.
I20250109 08:58:47.088872 20363 raft_consensus.cc:2267] T e6b9aaeda9a947d4a87fedd774301cc0 P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:47.090799 20363 tablet_replica.cc:331] T d714271c7ea64c0b8b13fcf63591ec07 P cf55383bef6649eb9e78cb2d2512dedc: stopping tablet replica
I20250109 08:58:47.091465 20363 raft_consensus.cc:2238] T d714271c7ea64c0b8b13fcf63591ec07 P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:47.091848 20363 raft_consensus.cc:2267] T d714271c7ea64c0b8b13fcf63591ec07 P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:47.093641 20363 tablet_replica.cc:331] T 77597d68423749e7b87f9f0db7a1ff3b P cf55383bef6649eb9e78cb2d2512dedc: stopping tablet replica
I20250109 08:58:47.094149 20363 raft_consensus.cc:2238] T 77597d68423749e7b87f9f0db7a1ff3b P cf55383bef6649eb9e78cb2d2512dedc [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:47.094547 20363 raft_consensus.cc:2267] T 77597d68423749e7b87f9f0db7a1ff3b P cf55383bef6649eb9e78cb2d2512dedc [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:47.096470 20363 tablet_replica.cc:331] T db111e41c1be497e8e673a2c42b9ba66 P cf55383bef6649eb9e78cb2d2512dedc: stopping tablet replica
I20250109 08:58:47.096952 20363 raft_consensus.cc:2238] T db111e41c1be497e8e673a2c42b9ba66 P cf55383bef6649eb9e78cb2d2512dedc [term 3 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:47.097323 20363 raft_consensus.cc:2267] T db111e41c1be497e8e673a2c42b9ba66 P cf55383bef6649eb9e78cb2d2512dedc [term 3 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:47.098949 20363 tablet_replica.cc:331] T 6eef9bb3c645428d9f1cb843bae9a264 P cf55383bef6649eb9e78cb2d2512dedc: stopping tablet replica
I20250109 08:58:47.099452 20363 raft_consensus.cc:2238] T 6eef9bb3c645428d9f1cb843bae9a264 P cf55383bef6649eb9e78cb2d2512dedc [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:47.099799 20363 raft_consensus.cc:2267] T 6eef9bb3c645428d9f1cb843bae9a264 P cf55383bef6649eb9e78cb2d2512dedc [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:47.101394 20363 tablet_replica.cc:331] T a54ca9f0d6e844e9a8b1d9cd0bb2a026 P cf55383bef6649eb9e78cb2d2512dedc: stopping tablet replica
I20250109 08:58:47.101888 20363 raft_consensus.cc:2238] T a54ca9f0d6e844e9a8b1d9cd0bb2a026 P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:47.102264 20363 raft_consensus.cc:2267] T a54ca9f0d6e844e9a8b1d9cd0bb2a026 P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:47.103864 20363 tablet_replica.cc:331] T 5831288451724d4ca1114ec5e5f01b78 P cf55383bef6649eb9e78cb2d2512dedc: stopping tablet replica
I20250109 08:58:47.104358 20363 raft_consensus.cc:2238] T 5831288451724d4ca1114ec5e5f01b78 P cf55383bef6649eb9e78cb2d2512dedc [term 1 LEADER]: Raft consensus shutting down.
I20250109 08:58:47.105091 20363 raft_consensus.cc:2267] T 5831288451724d4ca1114ec5e5f01b78 P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:47.106841 20363 tablet_replica.cc:331] T a686bf4d947c4b638049e6b88c59ae09 P cf55383bef6649eb9e78cb2d2512dedc: stopping tablet replica
I20250109 08:58:47.107340 20363 raft_consensus.cc:2238] T a686bf4d947c4b638049e6b88c59ae09 P cf55383bef6649eb9e78cb2d2512dedc [term 2 LEADER]: Raft consensus shutting down.
I20250109 08:58:47.108048 20363 raft_consensus.cc:2267] T a686bf4d947c4b638049e6b88c59ae09 P cf55383bef6649eb9e78cb2d2512dedc [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:47.109725 20363 tablet_replica.cc:331] T 0dab6c23ce444dda9427901a98f07b15 P cf55383bef6649eb9e78cb2d2512dedc: stopping tablet replica
I20250109 08:58:47.110226 20363 raft_consensus.cc:2238] T 0dab6c23ce444dda9427901a98f07b15 P cf55383bef6649eb9e78cb2d2512dedc [term 3 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:47.110594 20363 raft_consensus.cc:2267] T 0dab6c23ce444dda9427901a98f07b15 P cf55383bef6649eb9e78cb2d2512dedc [term 3 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:47.112205 20363 tablet_replica.cc:331] T 5c37c58abaa54656b31c72160b4dec9e P cf55383bef6649eb9e78cb2d2512dedc: stopping tablet replica
I20250109 08:58:47.112691 20363 raft_consensus.cc:2238] T 5c37c58abaa54656b31c72160b4dec9e P cf55383bef6649eb9e78cb2d2512dedc [term 1 LEADER]: Raft consensus shutting down.
I20250109 08:58:47.113358 20363 raft_consensus.cc:2267] T 5c37c58abaa54656b31c72160b4dec9e P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:47.114985 20363 tablet_replica.cc:331] T b572a6a3dfc04ea199bc703ddad8af67 P cf55383bef6649eb9e78cb2d2512dedc: stopping tablet replica
I20250109 08:58:47.115469 20363 raft_consensus.cc:2238] T b572a6a3dfc04ea199bc703ddad8af67 P cf55383bef6649eb9e78cb2d2512dedc [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:47.115824 20363 raft_consensus.cc:2267] T b572a6a3dfc04ea199bc703ddad8af67 P cf55383bef6649eb9e78cb2d2512dedc [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:47.117434 20363 tablet_replica.cc:331] T 9b0bbf2e4a424fb5ac13a3b55627b048 P cf55383bef6649eb9e78cb2d2512dedc: stopping tablet replica
I20250109 08:58:47.117947 20363 raft_consensus.cc:2238] T 9b0bbf2e4a424fb5ac13a3b55627b048 P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:47.118325 20363 raft_consensus.cc:2267] T 9b0bbf2e4a424fb5ac13a3b55627b048 P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:47.119949 20363 tablet_replica.cc:331] T cc9886333c0343798226d59758a1ea5e P cf55383bef6649eb9e78cb2d2512dedc: stopping tablet replica
I20250109 08:58:47.120462 20363 raft_consensus.cc:2238] T cc9886333c0343798226d59758a1ea5e P cf55383bef6649eb9e78cb2d2512dedc [term 3 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:47.120816 20363 raft_consensus.cc:2267] T cc9886333c0343798226d59758a1ea5e P cf55383bef6649eb9e78cb2d2512dedc [term 3 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:47.122400 20363 tablet_replica.cc:331] T 82ea7dc042dd4d05b260d508f4422c3c P cf55383bef6649eb9e78cb2d2512dedc: stopping tablet replica
I20250109 08:58:47.122884 20363 raft_consensus.cc:2238] T 82ea7dc042dd4d05b260d508f4422c3c P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:47.123255 20363 raft_consensus.cc:2267] T 82ea7dc042dd4d05b260d508f4422c3c P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:47.125228 20363 tablet_replica.cc:331] T d5fa255ab5f24750a0afa6d81ec869f2 P cf55383bef6649eb9e78cb2d2512dedc: stopping tablet replica
I20250109 08:58:47.125715 20363 raft_consensus.cc:2238] T d5fa255ab5f24750a0afa6d81ec869f2 P cf55383bef6649eb9e78cb2d2512dedc [term 1 LEADER]: Raft consensus shutting down.
I20250109 08:58:47.126591 20363 raft_consensus.cc:2267] T d5fa255ab5f24750a0afa6d81ec869f2 P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:47.128466 20363 tablet_replica.cc:331] T de98c70e25494d2a8e581443be836bd3 P cf55383bef6649eb9e78cb2d2512dedc: stopping tablet replica
I20250109 08:58:47.128988 20363 raft_consensus.cc:2238] T de98c70e25494d2a8e581443be836bd3 P cf55383bef6649eb9e78cb2d2512dedc [term 1 LEADER]: Raft consensus shutting down.
I20250109 08:58:47.129732 20363 raft_consensus.cc:2267] T de98c70e25494d2a8e581443be836bd3 P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:47.131438 20363 tablet_replica.cc:331] T 9b245cb3666c421c884cf20161a22831 P cf55383bef6649eb9e78cb2d2512dedc: stopping tablet replica
I20250109 08:58:47.131924 20363 raft_consensus.cc:2238] T 9b245cb3666c421c884cf20161a22831 P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:47.132313 20363 raft_consensus.cc:2267] T 9b245cb3666c421c884cf20161a22831 P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:47.134119 20363 tablet_replica.cc:331] T f1f9274aeda748f493f5752814fb262d P cf55383bef6649eb9e78cb2d2512dedc: stopping tablet replica
I20250109 08:58:47.134658 20363 raft_consensus.cc:2238] T f1f9274aeda748f493f5752814fb262d P cf55383bef6649eb9e78cb2d2512dedc [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:47.135030 20363 raft_consensus.cc:2267] T f1f9274aeda748f493f5752814fb262d P cf55383bef6649eb9e78cb2d2512dedc [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:47.136724 20363 tablet_replica.cc:331] T 1340cf8ee2cb423f875276137da17e8b P cf55383bef6649eb9e78cb2d2512dedc: stopping tablet replica
I20250109 08:58:47.137264 20363 raft_consensus.cc:2238] T 1340cf8ee2cb423f875276137da17e8b P cf55383bef6649eb9e78cb2d2512dedc [term 3 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:47.137639 20363 raft_consensus.cc:2267] T 1340cf8ee2cb423f875276137da17e8b P cf55383bef6649eb9e78cb2d2512dedc [term 3 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:47.139519 20363 tablet_replica.cc:331] T bd6fcd838b0249cabd5f1f95e5538ab6 P cf55383bef6649eb9e78cb2d2512dedc: stopping tablet replica
I20250109 08:58:47.140061 20363 raft_consensus.cc:2238] T bd6fcd838b0249cabd5f1f95e5538ab6 P cf55383bef6649eb9e78cb2d2512dedc [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:47.140499 20363 raft_consensus.cc:2267] T bd6fcd838b0249cabd5f1f95e5538ab6 P cf55383bef6649eb9e78cb2d2512dedc [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:47.142284 20363 tablet_replica.cc:331] T 940653b6d55c46439827697b9aadf16b P cf55383bef6649eb9e78cb2d2512dedc: stopping tablet replica
I20250109 08:58:47.142793 20363 raft_consensus.cc:2238] T 940653b6d55c46439827697b9aadf16b P cf55383bef6649eb9e78cb2d2512dedc [term 3 LEADER]: Raft consensus shutting down.
W20250109 08:58:47.143083 23494 consensus_peers.cc:487] T 0dab6c23ce444dda9427901a98f07b15 P 05a74e1eb2474b3d8b3a2da1ef127e01 -> Peer ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365): Couldn't send request to peer ed1efbce9ee3401e99a5260edeae4e7c. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.194:41365: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:47.143524 20363 raft_consensus.cc:2267] T 940653b6d55c46439827697b9aadf16b P cf55383bef6649eb9e78cb2d2512dedc [term 3 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:47.145275 20363 tablet_replica.cc:331] T 8de6b2fb09bc46ce9e4033af06dffd04 P cf55383bef6649eb9e78cb2d2512dedc: stopping tablet replica
I20250109 08:58:47.145769 20363 raft_consensus.cc:2238] T 8de6b2fb09bc46ce9e4033af06dffd04 P cf55383bef6649eb9e78cb2d2512dedc [term 1 LEADER]: Raft consensus shutting down.
I20250109 08:58:47.146593 20363 raft_consensus.cc:2267] T 8de6b2fb09bc46ce9e4033af06dffd04 P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:47.148980 20363 tablet_replica.cc:331] T 34caa5988a734b65a2eb512747d833b7 P cf55383bef6649eb9e78cb2d2512dedc: stopping tablet replica
I20250109 08:58:47.149655 20363 raft_consensus.cc:2238] T 34caa5988a734b65a2eb512747d833b7 P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:47.150094 20363 raft_consensus.cc:2267] T 34caa5988a734b65a2eb512747d833b7 P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:47.152326 20363 tablet_replica.cc:331] T 732e4aa69aae44e9b1153ac330a01a96 P cf55383bef6649eb9e78cb2d2512dedc: stopping tablet replica
I20250109 08:58:47.153036 20363 raft_consensus.cc:2238] T 732e4aa69aae44e9b1153ac330a01a96 P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:47.153517 20363 raft_consensus.cc:2267] T 732e4aa69aae44e9b1153ac330a01a96 P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Raft consensus is shut down!
W20250109 08:58:47.154670 23494 consensus_peers.cc:487] T 444507f7812441ceaeebc44bc8f06293 P 05a74e1eb2474b3d8b3a2da1ef127e01 -> Peer ed1efbce9ee3401e99a5260edeae4e7c (127.19.226.194:41365): Couldn't send request to peer ed1efbce9ee3401e99a5260edeae4e7c. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.194:41365: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:47.155877 20363 tablet_replica.cc:331] T 5ccc0d42d4a148d7819366cea0b754a7 P cf55383bef6649eb9e78cb2d2512dedc: stopping tablet replica
I20250109 08:58:47.156497 20363 raft_consensus.cc:2238] T 5ccc0d42d4a148d7819366cea0b754a7 P cf55383bef6649eb9e78cb2d2512dedc [term 1 LEADER]: Raft consensus shutting down.
I20250109 08:58:47.157168 20363 raft_consensus.cc:2267] T 5ccc0d42d4a148d7819366cea0b754a7 P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:47.159024 20363 tablet_replica.cc:331] T 3208a99957554f6db341654856980c8a P cf55383bef6649eb9e78cb2d2512dedc: stopping tablet replica
I20250109 08:58:47.159497 20363 raft_consensus.cc:2238] T 3208a99957554f6db341654856980c8a P cf55383bef6649eb9e78cb2d2512dedc [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:47.159852 20363 raft_consensus.cc:2267] T 3208a99957554f6db341654856980c8a P cf55383bef6649eb9e78cb2d2512dedc [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:47.161671 20363 tablet_replica.cc:331] T 8c5e3d0155d140c4b32f162d3173f5c9 P cf55383bef6649eb9e78cb2d2512dedc: stopping tablet replica
I20250109 08:58:47.162328 20363 raft_consensus.cc:2238] T 8c5e3d0155d140c4b32f162d3173f5c9 P cf55383bef6649eb9e78cb2d2512dedc [term 1 LEADER]: Raft consensus shutting down.
I20250109 08:58:47.163230 20363 raft_consensus.cc:2267] T 8c5e3d0155d140c4b32f162d3173f5c9 P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Raft consensus is shut down!
W20250109 08:58:47.164131 23494 consensus_peers.cc:487] T 94f2be307f704c75bceb213e52307cfb P 05a74e1eb2474b3d8b3a2da1ef127e01 -> Peer cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551): Couldn't send request to peer cf55383bef6649eb9e78cb2d2512dedc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:47.165222 20363 tablet_replica.cc:331] T 11aa76a3a30643708fececed660a5250 P cf55383bef6649eb9e78cb2d2512dedc: stopping tablet replica
I20250109 08:58:47.165774 20363 raft_consensus.cc:2238] T 11aa76a3a30643708fececed660a5250 P cf55383bef6649eb9e78cb2d2512dedc [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:47.166160 20363 raft_consensus.cc:2267] T 11aa76a3a30643708fececed660a5250 P cf55383bef6649eb9e78cb2d2512dedc [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:47.168185 20363 tablet_replica.cc:331] T 86d4074509744900abe94402467a7d85 P cf55383bef6649eb9e78cb2d2512dedc: stopping tablet replica
I20250109 08:58:47.168808 20363 raft_consensus.cc:2238] T 86d4074509744900abe94402467a7d85 P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:47.169310 20363 raft_consensus.cc:2267] T 86d4074509744900abe94402467a7d85 P cf55383bef6649eb9e78cb2d2512dedc [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:47.171617 20363 tablet_replica.cc:331] T 2e3d34c0d09843d2be54e47f3ccc16d8 P cf55383bef6649eb9e78cb2d2512dedc: stopping tablet replica
I20250109 08:58:47.172187 20363 raft_consensus.cc:2238] T 2e3d34c0d09843d2be54e47f3ccc16d8 P cf55383bef6649eb9e78cb2d2512dedc [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:47.172685 20363 raft_consensus.cc:2267] T 2e3d34c0d09843d2be54e47f3ccc16d8 P cf55383bef6649eb9e78cb2d2512dedc [term 2 FOLLOWER]: Raft consensus is shut down!
W20250109 08:58:47.174768 23494 consensus_peers.cc:487] T 77597d68423749e7b87f9f0db7a1ff3b P 05a74e1eb2474b3d8b3a2da1ef127e01 -> Peer cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551): Couldn't send request to peer cf55383bef6649eb9e78cb2d2512dedc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20250109 08:58:47.191766 23494 consensus_peers.cc:487] T cc9886333c0343798226d59758a1ea5e P 05a74e1eb2474b3d8b3a2da1ef127e01 -> Peer cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551): Couldn't send request to peer cf55383bef6649eb9e78cb2d2512dedc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20250109 08:58:47.198668 23494 consensus_peers.cc:487] T ecc833ad1afa4079a75e41b1a807c923 P 05a74e1eb2474b3d8b3a2da1ef127e01 -> Peer cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551): Couldn't send request to peer cf55383bef6649eb9e78cb2d2512dedc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:47.237031 20363 tablet_server.cc:195] TabletServer@127.19.226.195:0 shutdown complete.
W20250109 08:58:47.243111 23494 consensus_peers.cc:487] T 24ce131f7bb143bea1581d5b0c2a7109 P 05a74e1eb2474b3d8b3a2da1ef127e01 -> Peer cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551): Couldn't send request to peer cf55383bef6649eb9e78cb2d2512dedc. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.195:35551: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20250109 08:58:47.281837 23494 consensus_peers.cc:487] T 1ac4317290b24209a76d2e19e68ee45b P 05a74e1eb2474b3d8b3a2da1ef127e01 -> Peer cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551): Couldn't send request to peer cf55383bef6649eb9e78cb2d2512dedc. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.195:35551: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20250109 08:58:47.301950 23494 consensus_peers.cc:487] T 82ea7dc042dd4d05b260d508f4422c3c P 05a74e1eb2474b3d8b3a2da1ef127e01 -> Peer cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551): Couldn't send request to peer cf55383bef6649eb9e78cb2d2512dedc. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.195:35551: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20250109 08:58:47.306533 23494 consensus_peers.cc:487] T 9b245cb3666c421c884cf20161a22831 P 05a74e1eb2474b3d8b3a2da1ef127e01 -> Peer cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551): Couldn't send request to peer cf55383bef6649eb9e78cb2d2512dedc. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.195:35551: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:47.306638 20363 tablet_server.cc:178] TabletServer@127.19.226.196:0 shutting down...
I20250109 08:58:47.342669 20363 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250109 08:58:47.343502 20363 tablet_replica.cc:331] T 1ac4317290b24209a76d2e19e68ee45b P 05a74e1eb2474b3d8b3a2da1ef127e01: stopping tablet replica
I20250109 08:58:47.344177 20363 raft_consensus.cc:2238] T 1ac4317290b24209a76d2e19e68ee45b P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 4 LEADER]: Raft consensus shutting down.
I20250109 08:58:47.345526 20363 raft_consensus.cc:2267] T 1ac4317290b24209a76d2e19e68ee45b P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 4 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:47.347890 20363 tablet_replica.cc:331] T 77597d68423749e7b87f9f0db7a1ff3b P 05a74e1eb2474b3d8b3a2da1ef127e01: stopping tablet replica
I20250109 08:58:47.348758 20363 raft_consensus.cc:2238] T 77597d68423749e7b87f9f0db7a1ff3b P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 LEADER]: Raft consensus shutting down.
W20250109 08:58:47.348968 23494 consensus_peers.cc:487] T 0dab6c23ce444dda9427901a98f07b15 P 05a74e1eb2474b3d8b3a2da1ef127e01 -> Peer cf55383bef6649eb9e78cb2d2512dedc (127.19.226.195:35551): Couldn't send request to peer cf55383bef6649eb9e78cb2d2512dedc. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.195:35551: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250109 08:58:47.349713 20363 raft_consensus.cc:2267] T 77597d68423749e7b87f9f0db7a1ff3b P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:47.351697 20363 tablet_replica.cc:331] T a7d2ab61d238471c9f287ac1d5db4ed0 P 05a74e1eb2474b3d8b3a2da1ef127e01: stopping tablet replica
I20250109 08:58:47.352274 20363 raft_consensus.cc:2238] T a7d2ab61d238471c9f287ac1d5db4ed0 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:47.352710 20363 raft_consensus.cc:2267] T a7d2ab61d238471c9f287ac1d5db4ed0 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:47.354436 20363 tablet_replica.cc:331] T 1f9c6525267145d6bb73dde32ed9b7d4 P 05a74e1eb2474b3d8b3a2da1ef127e01: stopping tablet replica
I20250109 08:58:47.355095 20363 raft_consensus.cc:2238] T 1f9c6525267145d6bb73dde32ed9b7d4 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:47.355489 20363 raft_consensus.cc:2267] T 1f9c6525267145d6bb73dde32ed9b7d4 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:47.357293 20363 tablet_replica.cc:331] T 950e07e9cc794b8ab9ec5bd97d7e8e59 P 05a74e1eb2474b3d8b3a2da1ef127e01: stopping tablet replica
I20250109 08:58:47.357808 20363 raft_consensus.cc:2238] T 950e07e9cc794b8ab9ec5bd97d7e8e59 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:47.358176 20363 raft_consensus.cc:2267] T 950e07e9cc794b8ab9ec5bd97d7e8e59 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:47.359831 20363 tablet_replica.cc:331] T 060daed1d3334c6188413dd381b82bf9 P 05a74e1eb2474b3d8b3a2da1ef127e01: stopping tablet replica
I20250109 08:58:47.360359 20363 raft_consensus.cc:2238] T 060daed1d3334c6188413dd381b82bf9 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 3 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:47.360751 20363 raft_consensus.cc:2267] T 060daed1d3334c6188413dd381b82bf9 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 3 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:47.362402 20363 tablet_replica.cc:331] T 11aa76a3a30643708fececed660a5250 P 05a74e1eb2474b3d8b3a2da1ef127e01: stopping tablet replica
I20250109 08:58:47.362903 20363 raft_consensus.cc:2238] T 11aa76a3a30643708fececed660a5250 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 LEADER]: Raft consensus shutting down.
I20250109 08:58:47.363587 20363 raft_consensus.cc:2267] T 11aa76a3a30643708fececed660a5250 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:47.365461 20363 tablet_replica.cc:331] T 750dd1e1055f4570b0aecdda3ebe8772 P 05a74e1eb2474b3d8b3a2da1ef127e01: stopping tablet replica
I20250109 08:58:47.365962 20363 raft_consensus.cc:2238] T 750dd1e1055f4570b0aecdda3ebe8772 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:47.366343 20363 raft_consensus.cc:2267] T 750dd1e1055f4570b0aecdda3ebe8772 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:47.368073 20363 tablet_replica.cc:331] T ecc833ad1afa4079a75e41b1a807c923 P 05a74e1eb2474b3d8b3a2da1ef127e01: stopping tablet replica
I20250109 08:58:47.368641 20363 raft_consensus.cc:2238] T ecc833ad1afa4079a75e41b1a807c923 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 LEADER]: Raft consensus shutting down.
I20250109 08:58:47.369350 20363 raft_consensus.cc:2267] T ecc833ad1afa4079a75e41b1a807c923 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:47.371002 20363 tablet_replica.cc:331] T 444507f7812441ceaeebc44bc8f06293 P 05a74e1eb2474b3d8b3a2da1ef127e01: stopping tablet replica
I20250109 08:58:47.371515 20363 raft_consensus.cc:2238] T 444507f7812441ceaeebc44bc8f06293 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 LEADER]: Raft consensus shutting down.
I20250109 08:58:47.373138 20363 raft_consensus.cc:2267] T 444507f7812441ceaeebc44bc8f06293 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:47.374773 20363 tablet_replica.cc:331] T 5c37c58abaa54656b31c72160b4dec9e P 05a74e1eb2474b3d8b3a2da1ef127e01: stopping tablet replica
I20250109 08:58:47.375280 20363 raft_consensus.cc:2238] T 5c37c58abaa54656b31c72160b4dec9e P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:47.375638 20363 raft_consensus.cc:2267] T 5c37c58abaa54656b31c72160b4dec9e P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:47.377308 20363 tablet_replica.cc:331] T 0dab6c23ce444dda9427901a98f07b15 P 05a74e1eb2474b3d8b3a2da1ef127e01: stopping tablet replica
I20250109 08:58:47.377820 20363 raft_consensus.cc:2238] T 0dab6c23ce444dda9427901a98f07b15 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 3 LEADER]: Raft consensus shutting down.
I20250109 08:58:47.378544 20363 raft_consensus.cc:2267] T 0dab6c23ce444dda9427901a98f07b15 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 3 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:47.380234 20363 tablet_replica.cc:331] T 5f0d500a91b54851bda52e5ccf741fd6 P 05a74e1eb2474b3d8b3a2da1ef127e01: stopping tablet replica
I20250109 08:58:47.380772 20363 raft_consensus.cc:2238] T 5f0d500a91b54851bda52e5ccf741fd6 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:47.381121 20363 raft_consensus.cc:2267] T 5f0d500a91b54851bda52e5ccf741fd6 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:47.382833 20363 tablet_replica.cc:331] T ee703ea575234c659da04f21ef3cdd70 P 05a74e1eb2474b3d8b3a2da1ef127e01: stopping tablet replica
I20250109 08:58:47.383359 20363 raft_consensus.cc:2238] T ee703ea575234c659da04f21ef3cdd70 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:47.383733 20363 raft_consensus.cc:2267] T ee703ea575234c659da04f21ef3cdd70 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:47.386193 20363 tablet_replica.cc:331] T 86b82e19759c4fd0b471a0dbff839aa5 P 05a74e1eb2474b3d8b3a2da1ef127e01: stopping tablet replica
I20250109 08:58:47.386881 20363 raft_consensus.cc:2238] T 86b82e19759c4fd0b471a0dbff839aa5 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:47.387331 20363 raft_consensus.cc:2267] T 86b82e19759c4fd0b471a0dbff839aa5 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:47.389364 20363 tablet_replica.cc:331] T a686bf4d947c4b638049e6b88c59ae09 P 05a74e1eb2474b3d8b3a2da1ef127e01: stopping tablet replica
I20250109 08:58:47.389880 20363 raft_consensus.cc:2238] T a686bf4d947c4b638049e6b88c59ae09 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:47.390250 20363 raft_consensus.cc:2267] T a686bf4d947c4b638049e6b88c59ae09 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:47.391875 20363 tablet_replica.cc:331] T 4df64aae69ca41dc91286cae1603000f P 05a74e1eb2474b3d8b3a2da1ef127e01: stopping tablet replica
I20250109 08:58:47.392421 20363 raft_consensus.cc:2238] T 4df64aae69ca41dc91286cae1603000f P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:47.392807 20363 raft_consensus.cc:2267] T 4df64aae69ca41dc91286cae1603000f P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:47.394466 20363 tablet_replica.cc:331] T 5831288451724d4ca1114ec5e5f01b78 P 05a74e1eb2474b3d8b3a2da1ef127e01: stopping tablet replica
I20250109 08:58:47.394965 20363 raft_consensus.cc:2238] T 5831288451724d4ca1114ec5e5f01b78 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:47.395334 20363 raft_consensus.cc:2267] T 5831288451724d4ca1114ec5e5f01b78 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:47.397022 20363 tablet_replica.cc:331] T ea2b0762940e44dbbf69b9a163ace846 P 05a74e1eb2474b3d8b3a2da1ef127e01: stopping tablet replica
I20250109 08:58:47.397542 20363 raft_consensus.cc:2238] T ea2b0762940e44dbbf69b9a163ace846 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 3 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:47.397886 20363 raft_consensus.cc:2267] T ea2b0762940e44dbbf69b9a163ace846 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 3 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:47.399510 20363 tablet_replica.cc:331] T 24ce131f7bb143bea1581d5b0c2a7109 P 05a74e1eb2474b3d8b3a2da1ef127e01: stopping tablet replica
I20250109 08:58:47.399978 20363 raft_consensus.cc:2238] T 24ce131f7bb143bea1581d5b0c2a7109 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 LEADER]: Raft consensus shutting down.
I20250109 08:58:47.400717 20363 raft_consensus.cc:2267] T 24ce131f7bb143bea1581d5b0c2a7109 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:47.402348 20363 tablet_replica.cc:331] T b572a6a3dfc04ea199bc703ddad8af67 P 05a74e1eb2474b3d8b3a2da1ef127e01: stopping tablet replica
I20250109 08:58:47.402841 20363 raft_consensus.cc:2238] T b572a6a3dfc04ea199bc703ddad8af67 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:47.403204 20363 raft_consensus.cc:2267] T b572a6a3dfc04ea199bc703ddad8af67 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:47.404976 20363 tablet_replica.cc:331] T 8c5e3d0155d140c4b32f162d3173f5c9 P 05a74e1eb2474b3d8b3a2da1ef127e01: stopping tablet replica
I20250109 08:58:47.405638 20363 raft_consensus.cc:2238] T 8c5e3d0155d140c4b32f162d3173f5c9 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:47.406090 20363 raft_consensus.cc:2267] T 8c5e3d0155d140c4b32f162d3173f5c9 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:47.408535 20363 tablet_replica.cc:331] T 1c5b5beea4a840eb9677097cacee3408 P 05a74e1eb2474b3d8b3a2da1ef127e01: stopping tablet replica
I20250109 08:58:47.409130 20363 raft_consensus.cc:2238] T 1c5b5beea4a840eb9677097cacee3408 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:47.409622 20363 raft_consensus.cc:2267] T 1c5b5beea4a840eb9677097cacee3408 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:47.411195 20363 tablet_replica.cc:331] T 08214ee9c4c14bda92bdc6fa8231c313 P 05a74e1eb2474b3d8b3a2da1ef127e01: stopping tablet replica
I20250109 08:58:47.411657 20363 raft_consensus.cc:2238] T 08214ee9c4c14bda92bdc6fa8231c313 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:47.411983 20363 raft_consensus.cc:2267] T 08214ee9c4c14bda92bdc6fa8231c313 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:47.414779 20363 tablet_replica.cc:331] T bd6fcd838b0249cabd5f1f95e5538ab6 P 05a74e1eb2474b3d8b3a2da1ef127e01: stopping tablet replica
I20250109 08:58:47.415284 20363 raft_consensus.cc:2238] T bd6fcd838b0249cabd5f1f95e5538ab6 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:47.415656 20363 raft_consensus.cc:2267] T bd6fcd838b0249cabd5f1f95e5538ab6 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:47.417310 20363 tablet_replica.cc:331] T 940653b6d55c46439827697b9aadf16b P 05a74e1eb2474b3d8b3a2da1ef127e01: stopping tablet replica
I20250109 08:58:47.417802 20363 raft_consensus.cc:2238] T 940653b6d55c46439827697b9aadf16b P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 3 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:47.418177 20363 raft_consensus.cc:2267] T 940653b6d55c46439827697b9aadf16b P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 3 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:47.419824 20363 tablet_replica.cc:331] T fc1038255dd64cb5ba2425037d22f1f7 P 05a74e1eb2474b3d8b3a2da1ef127e01: stopping tablet replica
I20250109 08:58:47.420305 20363 raft_consensus.cc:2238] T fc1038255dd64cb5ba2425037d22f1f7 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 LEADER]: Raft consensus shutting down.
I20250109 08:58:47.421006 20363 raft_consensus.cc:2267] T fc1038255dd64cb5ba2425037d22f1f7 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:47.422686 20363 tablet_replica.cc:331] T 9b245cb3666c421c884cf20161a22831 P 05a74e1eb2474b3d8b3a2da1ef127e01: stopping tablet replica
I20250109 08:58:47.423218 20363 raft_consensus.cc:2238] T 9b245cb3666c421c884cf20161a22831 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 LEADER]: Raft consensus shutting down.
I20250109 08:58:47.424049 20363 raft_consensus.cc:2267] T 9b245cb3666c421c884cf20161a22831 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:47.426259 20363 tablet_replica.cc:331] T 94f2be307f704c75bceb213e52307cfb P 05a74e1eb2474b3d8b3a2da1ef127e01: stopping tablet replica
I20250109 08:58:47.426779 20363 raft_consensus.cc:2238] T 94f2be307f704c75bceb213e52307cfb P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 LEADER]: Raft consensus shutting down.
I20250109 08:58:47.427521 20363 raft_consensus.cc:2267] T 94f2be307f704c75bceb213e52307cfb P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:47.429172 20363 tablet_replica.cc:331] T e5251927614642639bd46ecd5373d2a4 P 05a74e1eb2474b3d8b3a2da1ef127e01: stopping tablet replica
I20250109 08:58:47.429656 20363 raft_consensus.cc:2238] T e5251927614642639bd46ecd5373d2a4 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:47.429997 20363 raft_consensus.cc:2267] T e5251927614642639bd46ecd5373d2a4 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:47.431545 20363 tablet_replica.cc:331] T 41e7f0cc38c4414ab88c92c1cdeb3984 P 05a74e1eb2474b3d8b3a2da1ef127e01: stopping tablet replica
I20250109 08:58:47.432010 20363 raft_consensus.cc:2238] T 41e7f0cc38c4414ab88c92c1cdeb3984 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 LEADER]: Raft consensus shutting down.
I20250109 08:58:47.432716 20363 raft_consensus.cc:2267] T 41e7f0cc38c4414ab88c92c1cdeb3984 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:47.434388 20363 tablet_replica.cc:331] T 970a82e754db47b6b1a19551d4ba43d8 P 05a74e1eb2474b3d8b3a2da1ef127e01: stopping tablet replica
I20250109 08:58:47.434835 20363 raft_consensus.cc:2238] T 970a82e754db47b6b1a19551d4ba43d8 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 3 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:47.435209 20363 raft_consensus.cc:2267] T 970a82e754db47b6b1a19551d4ba43d8 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 3 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:47.436848 20363 tablet_replica.cc:331] T d5fa255ab5f24750a0afa6d81ec869f2 P 05a74e1eb2474b3d8b3a2da1ef127e01: stopping tablet replica
I20250109 08:58:47.437325 20363 raft_consensus.cc:2238] T d5fa255ab5f24750a0afa6d81ec869f2 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:47.437798 20363 raft_consensus.cc:2267] T d5fa255ab5f24750a0afa6d81ec869f2 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:47.441771 20363 tablet_replica.cc:331] T 82ea7dc042dd4d05b260d508f4422c3c P 05a74e1eb2474b3d8b3a2da1ef127e01: stopping tablet replica
I20250109 08:58:47.442252 20363 raft_consensus.cc:2238] T 82ea7dc042dd4d05b260d508f4422c3c P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 LEADER]: Raft consensus shutting down.
I20250109 08:58:47.442963 20363 raft_consensus.cc:2267] T 82ea7dc042dd4d05b260d508f4422c3c P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:47.444754 20363 tablet_replica.cc:331] T cc9886333c0343798226d59758a1ea5e P 05a74e1eb2474b3d8b3a2da1ef127e01: stopping tablet replica
I20250109 08:58:47.445209 20363 raft_consensus.cc:2238] T cc9886333c0343798226d59758a1ea5e P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 3 LEADER]: Raft consensus shutting down.
I20250109 08:58:47.445847 20363 raft_consensus.cc:2267] T cc9886333c0343798226d59758a1ea5e P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 3 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:47.447573 20363 tablet_replica.cc:331] T 8de6b2fb09bc46ce9e4033af06dffd04 P 05a74e1eb2474b3d8b3a2da1ef127e01: stopping tablet replica
I20250109 08:58:47.448029 20363 raft_consensus.cc:2238] T 8de6b2fb09bc46ce9e4033af06dffd04 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:47.448419 20363 raft_consensus.cc:2267] T 8de6b2fb09bc46ce9e4033af06dffd04 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:47.450011 20363 tablet_replica.cc:331] T 34caa5988a734b65a2eb512747d833b7 P 05a74e1eb2474b3d8b3a2da1ef127e01: stopping tablet replica
I20250109 08:58:47.450484 20363 raft_consensus.cc:2238] T 34caa5988a734b65a2eb512747d833b7 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:47.450827 20363 raft_consensus.cc:2267] T 34caa5988a734b65a2eb512747d833b7 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:47.452477 20363 tablet_replica.cc:331] T 29828d9befb845d5bd49a80e018502e3 P 05a74e1eb2474b3d8b3a2da1ef127e01: stopping tablet replica
I20250109 08:58:47.452904 20363 raft_consensus.cc:2238] T 29828d9befb845d5bd49a80e018502e3 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 LEADER]: Raft consensus shutting down.
I20250109 08:58:47.453553 20363 raft_consensus.cc:2267] T 29828d9befb845d5bd49a80e018502e3 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:47.455092 20363 tablet_replica.cc:331] T 9b0bbf2e4a424fb5ac13a3b55627b048 P 05a74e1eb2474b3d8b3a2da1ef127e01: stopping tablet replica
I20250109 08:58:47.455554 20363 raft_consensus.cc:2238] T 9b0bbf2e4a424fb5ac13a3b55627b048 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:47.455878 20363 raft_consensus.cc:2267] T 9b0bbf2e4a424fb5ac13a3b55627b048 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:47.457608 20363 tablet_replica.cc:331] T 5ccc0d42d4a148d7819366cea0b754a7 P 05a74e1eb2474b3d8b3a2da1ef127e01: stopping tablet replica
I20250109 08:58:47.458050 20363 raft_consensus.cc:2238] T 5ccc0d42d4a148d7819366cea0b754a7 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:47.458385 20363 raft_consensus.cc:2267] T 5ccc0d42d4a148d7819366cea0b754a7 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:47.460000 20363 tablet_replica.cc:331] T d87d3b4755344b82b0aec7ffd1ce785c P 05a74e1eb2474b3d8b3a2da1ef127e01: stopping tablet replica
I20250109 08:58:47.460491 20363 raft_consensus.cc:2238] T d87d3b4755344b82b0aec7ffd1ce785c P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 LEADER]: Raft consensus shutting down.
I20250109 08:58:47.461133 20363 raft_consensus.cc:2267] T d87d3b4755344b82b0aec7ffd1ce785c P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:47.462894 20363 tablet_replica.cc:331] T bfb140bb3a8b44bd8b4e1fcf0946162b P 05a74e1eb2474b3d8b3a2da1ef127e01: stopping tablet replica
I20250109 08:58:47.463371 20363 raft_consensus.cc:2238] T bfb140bb3a8b44bd8b4e1fcf0946162b P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:47.463721 20363 raft_consensus.cc:2267] T bfb140bb3a8b44bd8b4e1fcf0946162b P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:47.465409 20363 tablet_replica.cc:331] T 2ab70546dda143b5b5db15c3875aa348 P 05a74e1eb2474b3d8b3a2da1ef127e01: stopping tablet replica
I20250109 08:58:47.465832 20363 raft_consensus.cc:2238] T 2ab70546dda143b5b5db15c3875aa348 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 3 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:47.466184 20363 raft_consensus.cc:2267] T 2ab70546dda143b5b5db15c3875aa348 P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 3 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:47.467792 20363 tablet_replica.cc:331] T ed79047393d2425f8ecff31ea08b9cad P 05a74e1eb2474b3d8b3a2da1ef127e01: stopping tablet replica
I20250109 08:58:47.468183 20363 raft_consensus.cc:2238] T ed79047393d2425f8ecff31ea08b9cad P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:58:47.468547 20363 raft_consensus.cc:2267] T ed79047393d2425f8ecff31ea08b9cad P 05a74e1eb2474b3d8b3a2da1ef127e01 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:47.563562 20363 tablet_server.cc:195] TabletServer@127.19.226.196:0 shutdown complete.
I20250109 08:58:47.619987 20363 master.cc:537] Master@127.19.226.254:46425 shutting down...
I20250109 08:58:47.642290 20363 raft_consensus.cc:2238] T 00000000000000000000000000000000 P 7dbdd4b28a9d481eb01d950816966c31 [term 1 LEADER]: Raft consensus shutting down.
I20250109 08:58:47.642802 20363 raft_consensus.cc:2267] T 00000000000000000000000000000000 P 7dbdd4b28a9d481eb01d950816966c31 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:58:47.643083 20363 tablet_replica.cc:331] T 00000000000000000000000000000000 P 7dbdd4b28a9d481eb01d950816966c31: stopping tablet replica
I20250109 08:58:47.663688 20363 master.cc:559] Master@127.19.226.254:46425 shutdown complete.
[ OK ] LeaderRebalancerTest.RestartTserver (29833 ms)
[ RUN ] LeaderRebalancerTest.TestMaintenanceMode
/home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/master/auto_leader_rebalancer-test.cc:471: Skipped
test is skipped; set KUDU_ALLOW_SLOW_TESTS=1 to run
[ SKIPPED ] LeaderRebalancerTest.TestMaintenanceMode (6 ms)
[----------] 5 tests from LeaderRebalancerTest (140048 ms total)
[----------] 2 tests from FilterSoftDeletedTableTest
[ RUN ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0
I20250109 08:58:47.748008 20363 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.19.226.254:45577
I20250109 08:58:47.749094 20363 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250109 08:58:47.754832 24493 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250109 08:58:47.754932 24494 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250109 08:58:47.755858 20363 server_base.cc:1034] running on GCE node
W20250109 08:58:47.756103 24496 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250109 08:58:47.757026 20363 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250109 08:58:47.757225 20363 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250109 08:58:47.757386 20363 hybrid_clock.cc:648] HybridClock initialized: now 1736413127757367 us; error 0 us; skew 500 ppm
I20250109 08:58:47.757910 20363 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250109 08:58:47.765740 20363 webserver.cc:458] Webserver started at http://127.19.226.254:32895/ using document root <none> and password file <none>
I20250109 08:58:47.766294 20363 fs_manager.cc:362] Metadata directory not provided
I20250109 08:58:47.766469 20363 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250109 08:58:47.766721 20363 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250109 08:58:47.767863 20363 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1736412987538469-20363-0/minicluster-data/master-0-root/instance:
uuid: "8066e53f72404343aa3a9f0ff5455781"
format_stamp: "Formatted at 2025-01-09 08:58:47 on dist-test-slave-g8ds"
I20250109 08:58:47.772356 20363 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20250109 08:58:47.775705 24501 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250109 08:58:47.776468 20363 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20250109 08:58:47.776739 20363 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1736412987538469-20363-0/minicluster-data/master-0-root
uuid: "8066e53f72404343aa3a9f0ff5455781"
format_stamp: "Formatted at 2025-01-09 08:58:47 on dist-test-slave-g8ds"
I20250109 08:58:47.777004 20363 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1736412987538469-20363-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1736412987538469-20363-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1736412987538469-20363-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250109 08:58:47.816980 20363 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250109 08:58:47.818588 20363 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250109 08:58:47.862744 20363 rpc_server.cc:307] RPC server started. Bound to: 127.19.226.254:45577
I20250109 08:58:47.862847 24552 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.226.254:45577 every 8 connection(s)
I20250109 08:58:47.866494 24553 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:47.876086 24553 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8066e53f72404343aa3a9f0ff5455781: Bootstrap starting.
I20250109 08:58:47.880765 24553 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8066e53f72404343aa3a9f0ff5455781: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:47.884651 24553 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8066e53f72404343aa3a9f0ff5455781: No bootstrap required, opened a new log
I20250109 08:58:47.886595 24553 raft_consensus.cc:357] T 00000000000000000000000000000000 P 8066e53f72404343aa3a9f0ff5455781 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8066e53f72404343aa3a9f0ff5455781" member_type: VOTER }
I20250109 08:58:47.886955 24553 raft_consensus.cc:383] T 00000000000000000000000000000000 P 8066e53f72404343aa3a9f0ff5455781 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:47.887135 24553 raft_consensus.cc:738] T 00000000000000000000000000000000 P 8066e53f72404343aa3a9f0ff5455781 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8066e53f72404343aa3a9f0ff5455781, State: Initialized, Role: FOLLOWER
I20250109 08:58:47.887617 24553 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8066e53f72404343aa3a9f0ff5455781 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8066e53f72404343aa3a9f0ff5455781" member_type: VOTER }
I20250109 08:58:47.888020 24553 raft_consensus.cc:397] T 00000000000000000000000000000000 P 8066e53f72404343aa3a9f0ff5455781 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250109 08:58:47.888208 24553 raft_consensus.cc:491] T 00000000000000000000000000000000 P 8066e53f72404343aa3a9f0ff5455781 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250109 08:58:47.888445 24553 raft_consensus.cc:3054] T 00000000000000000000000000000000 P 8066e53f72404343aa3a9f0ff5455781 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:47.892457 24553 raft_consensus.cc:513] T 00000000000000000000000000000000 P 8066e53f72404343aa3a9f0ff5455781 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8066e53f72404343aa3a9f0ff5455781" member_type: VOTER }
I20250109 08:58:47.892952 24553 leader_election.cc:304] T 00000000000000000000000000000000 P 8066e53f72404343aa3a9f0ff5455781 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8066e53f72404343aa3a9f0ff5455781; no voters:
I20250109 08:58:47.893967 24553 leader_election.cc:290] T 00000000000000000000000000000000 P 8066e53f72404343aa3a9f0ff5455781 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250109 08:58:47.894342 24556 raft_consensus.cc:2798] T 00000000000000000000000000000000 P 8066e53f72404343aa3a9f0ff5455781 [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:58:47.895633 24556 raft_consensus.cc:695] T 00000000000000000000000000000000 P 8066e53f72404343aa3a9f0ff5455781 [term 1 LEADER]: Becoming Leader. State: Replica: 8066e53f72404343aa3a9f0ff5455781, State: Running, Role: LEADER
I20250109 08:58:47.896258 24556 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8066e53f72404343aa3a9f0ff5455781 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8066e53f72404343aa3a9f0ff5455781" member_type: VOTER }
I20250109 08:58:47.896775 24553 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8066e53f72404343aa3a9f0ff5455781 [sys.catalog]: configured and running, proceeding with master startup.
I20250109 08:58:47.900153 24557 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8066e53f72404343aa3a9f0ff5455781 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8066e53f72404343aa3a9f0ff5455781" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8066e53f72404343aa3a9f0ff5455781" member_type: VOTER } }
I20250109 08:58:47.900805 24557 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8066e53f72404343aa3a9f0ff5455781 [sys.catalog]: This master's current role is: LEADER
I20250109 08:58:47.901856 24558 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8066e53f72404343aa3a9f0ff5455781 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8066e53f72404343aa3a9f0ff5455781. Latest consensus state: current_term: 1 leader_uuid: "8066e53f72404343aa3a9f0ff5455781" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8066e53f72404343aa3a9f0ff5455781" member_type: VOTER } }
I20250109 08:58:47.902397 24558 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8066e53f72404343aa3a9f0ff5455781 [sys.catalog]: This master's current role is: LEADER
I20250109 08:58:47.903268 24562 catalog_manager.cc:1476] Loading table and tablet metadata into memory...
I20250109 08:58:47.910241 24562 catalog_manager.cc:1485] Initializing Kudu cluster ID...
I20250109 08:58:47.913012 20363 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20250109 08:58:47.918917 24562 catalog_manager.cc:1348] Generated new cluster ID: 4686371751ae4051bc59bd3299319f8b
I20250109 08:58:47.919219 24562 catalog_manager.cc:1496] Initializing Kudu internal certificate authority...
I20250109 08:58:47.933411 24562 catalog_manager.cc:1371] Generated new certificate authority record
I20250109 08:58:47.934690 24562 catalog_manager.cc:1505] Loading token signing keys...
I20250109 08:58:47.950731 24562 catalog_manager.cc:5899] T 00000000000000000000000000000000 P 8066e53f72404343aa3a9f0ff5455781: Generated new TSK 0
I20250109 08:58:47.951352 24562 catalog_manager.cc:1515] Initializing in-progress tserver states...
I20250109 08:58:47.980142 20363 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250109 08:58:47.986081 24574 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250109 08:58:47.986784 24575 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250109 08:58:47.988664 24577 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250109 08:58:47.989329 20363 server_base.cc:1034] running on GCE node
I20250109 08:58:47.990224 20363 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250109 08:58:47.990407 20363 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250109 08:58:47.990561 20363 hybrid_clock.cc:648] HybridClock initialized: now 1736413127990546 us; error 0 us; skew 500 ppm
I20250109 08:58:47.991055 20363 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250109 08:58:47.993360 20363 webserver.cc:458] Webserver started at http://127.19.226.193:44707/ using document root <none> and password file <none>
I20250109 08:58:47.993829 20363 fs_manager.cc:362] Metadata directory not provided
I20250109 08:58:47.994010 20363 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250109 08:58:47.994243 20363 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250109 08:58:47.995272 20363 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1736412987538469-20363-0/minicluster-data/ts-0-root/instance:
uuid: "08c70c9a6a064d40bfa68fd105d86d86"
format_stamp: "Formatted at 2025-01-09 08:58:47 on dist-test-slave-g8ds"
I20250109 08:58:47.999958 20363 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20250109 08:58:48.003212 24582 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250109 08:58:48.003921 20363 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20250109 08:58:48.004196 20363 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1736412987538469-20363-0/minicluster-data/ts-0-root
uuid: "08c70c9a6a064d40bfa68fd105d86d86"
format_stamp: "Formatted at 2025-01-09 08:58:47 on dist-test-slave-g8ds"
I20250109 08:58:48.004530 20363 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1736412987538469-20363-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1736412987538469-20363-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1736412987538469-20363-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250109 08:58:48.025946 20363 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250109 08:58:48.027108 20363 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250109 08:58:48.028571 20363 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250109 08:58:48.030704 20363 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250109 08:58:48.030885 20363 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250109 08:58:48.031098 20363 ts_tablet_manager.cc:610] Registered 0 tablets
I20250109 08:58:48.031241 20363 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250109 08:58:48.068984 20363 rpc_server.cc:307] RPC server started. Bound to: 127.19.226.193:43567
I20250109 08:58:48.069082 24644 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.226.193:43567 every 8 connection(s)
I20250109 08:58:48.073612 20363 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250109 08:58:48.080814 24649 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250109 08:58:48.082628 24650 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250109 08:58:48.083729 24652 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250109 08:58:48.084859 20363 server_base.cc:1034] running on GCE node
I20250109 08:58:48.085943 20363 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20250109 08:58:48.085959 24645 heartbeater.cc:346] Connected to a master server at 127.19.226.254:45577
W20250109 08:58:48.086195 20363 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250109 08:58:48.086416 20363 hybrid_clock.cc:648] HybridClock initialized: now 1736413128086399 us; error 0 us; skew 500 ppm
I20250109 08:58:48.086441 24645 heartbeater.cc:463] Registering TS with master...
I20250109 08:58:48.086977 20363 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250109 08:58:48.087225 24645 heartbeater.cc:510] Master 127.19.226.254:45577 requested a full tablet report, sending...
I20250109 08:58:48.089362 24518 ts_manager.cc:194] Registered new tserver with Master: 08c70c9a6a064d40bfa68fd105d86d86 (127.19.226.193:43567)
I20250109 08:58:48.089613 20363 webserver.cc:458] Webserver started at http://127.19.226.194:43801/ using document root <none> and password file <none>
I20250109 08:58:48.090188 20363 fs_manager.cc:362] Metadata directory not provided
I20250109 08:58:48.090363 20363 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250109 08:58:48.090607 20363 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250109 08:58:48.091148 24518 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37854
I20250109 08:58:48.091679 20363 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1736412987538469-20363-0/minicluster-data/ts-1-root/instance:
uuid: "f366ff6c37a24eaaa6abcafab1467912"
format_stamp: "Formatted at 2025-01-09 08:58:48 on dist-test-slave-g8ds"
I20250109 08:58:48.096078 20363 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20250109 08:58:48.099159 24657 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250109 08:58:48.099843 20363 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20250109 08:58:48.100106 20363 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1736412987538469-20363-0/minicluster-data/ts-1-root
uuid: "f366ff6c37a24eaaa6abcafab1467912"
format_stamp: "Formatted at 2025-01-09 08:58:48 on dist-test-slave-g8ds"
I20250109 08:58:48.100412 20363 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1736412987538469-20363-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1736412987538469-20363-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1736412987538469-20363-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250109 08:58:48.127977 20363 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250109 08:58:48.129194 20363 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250109 08:58:48.130625 20363 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250109 08:58:48.132822 20363 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250109 08:58:48.133006 20363 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250109 08:58:48.133219 20363 ts_tablet_manager.cc:610] Registered 0 tablets
I20250109 08:58:48.133364 20363 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250109 08:58:48.170423 20363 rpc_server.cc:307] RPC server started. Bound to: 127.19.226.194:43355
I20250109 08:58:48.170545 24719 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.226.194:43355 every 8 connection(s)
I20250109 08:58:48.174832 20363 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250109 08:58:48.181753 24723 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250109 08:58:48.182503 24724 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250109 08:58:48.185067 24726 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250109 08:58:48.185798 24720 heartbeater.cc:346] Connected to a master server at 127.19.226.254:45577
I20250109 08:58:48.186173 24720 heartbeater.cc:463] Registering TS with master...
I20250109 08:58:48.186350 20363 server_base.cc:1034] running on GCE node
I20250109 08:58:48.187041 24720 heartbeater.cc:510] Master 127.19.226.254:45577 requested a full tablet report, sending...
I20250109 08:58:48.187381 20363 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250109 08:58:48.187667 20363 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250109 08:58:48.187872 20363 hybrid_clock.cc:648] HybridClock initialized: now 1736413128187854 us; error 0 us; skew 500 ppm
I20250109 08:58:48.188568 20363 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250109 08:58:48.189189 24518 ts_manager.cc:194] Registered new tserver with Master: f366ff6c37a24eaaa6abcafab1467912 (127.19.226.194:43355)
I20250109 08:58:48.190994 24518 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37868
I20250109 08:58:48.191305 20363 webserver.cc:458] Webserver started at http://127.19.226.195:33135/ using document root <none> and password file <none>
I20250109 08:58:48.191956 20363 fs_manager.cc:362] Metadata directory not provided
I20250109 08:58:48.192183 20363 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250109 08:58:48.192480 20363 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250109 08:58:48.193591 20363 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1736412987538469-20363-0/minicluster-data/ts-2-root/instance:
uuid: "1a9b5a4afd63410b813b41edbfe69e28"
format_stamp: "Formatted at 2025-01-09 08:58:48 on dist-test-slave-g8ds"
I20250109 08:58:48.197813 20363 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20250109 08:58:48.200868 24731 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250109 08:58:48.201690 20363 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20250109 08:58:48.201944 20363 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1736412987538469-20363-0/minicluster-data/ts-2-root
uuid: "1a9b5a4afd63410b813b41edbfe69e28"
format_stamp: "Formatted at 2025-01-09 08:58:48 on dist-test-slave-g8ds"
I20250109 08:58:48.202221 20363 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1736412987538469-20363-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1736412987538469-20363-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1736412987538469-20363-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250109 08:58:48.216640 20363 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250109 08:58:48.217777 20363 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250109 08:58:48.219141 20363 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250109 08:58:48.221349 20363 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250109 08:58:48.221521 20363 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250109 08:58:48.221741 20363 ts_tablet_manager.cc:610] Registered 0 tablets
I20250109 08:58:48.221879 20363 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250109 08:58:48.258710 20363 rpc_server.cc:307] RPC server started. Bound to: 127.19.226.195:42529
I20250109 08:58:48.258821 24793 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.226.195:42529 every 8 connection(s)
I20250109 08:58:48.271144 24794 heartbeater.cc:346] Connected to a master server at 127.19.226.254:45577
I20250109 08:58:48.271589 24794 heartbeater.cc:463] Registering TS with master...
I20250109 08:58:48.272240 24794 heartbeater.cc:510] Master 127.19.226.254:45577 requested a full tablet report, sending...
I20250109 08:58:48.273991 24518 ts_manager.cc:194] Registered new tserver with Master: 1a9b5a4afd63410b813b41edbfe69e28 (127.19.226.195:42529)
I20250109 08:58:48.274713 20363 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.013225184s
I20250109 08:58:48.275702 24518 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37872
I20250109 08:58:48.277451 20363 test_util.cc:274] Using random seed: 800177583
I20250109 08:58:48.298683 24518 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37876:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250109 08:58:48.301395 24518 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250109 08:58:48.373281 24685 tablet_service.cc:1467] Processing CreateTablet for tablet a930173474344b2c93cc8ac295a74729 (DEFAULT_TABLE table=test-workload [id=7f5a3e5c2e314e83ba662a913b7b3f17]), partition=RANGE (key) PARTITION VALUES < 238609294
I20250109 08:58:48.374706 24685 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a930173474344b2c93cc8ac295a74729. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:48.375999 24684 tablet_service.cc:1467] Processing CreateTablet for tablet 970b2f8b130d431cb25d79d33d5f0e5c (DEFAULT_TABLE table=test-workload [id=7f5a3e5c2e314e83ba662a913b7b3f17]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20250109 08:58:48.377468 24759 tablet_service.cc:1467] Processing CreateTablet for tablet a930173474344b2c93cc8ac295a74729 (DEFAULT_TABLE table=test-workload [id=7f5a3e5c2e314e83ba662a913b7b3f17]), partition=RANGE (key) PARTITION VALUES < 238609294
I20250109 08:58:48.377404 24684 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 970b2f8b130d431cb25d79d33d5f0e5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:48.378080 24678 tablet_service.cc:1467] Processing CreateTablet for tablet d8f9f0e755da4a7186c2c9316918ca73 (DEFAULT_TABLE table=test-workload [id=7f5a3e5c2e314e83ba662a913b7b3f17]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20250109 08:58:48.378790 24759 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a930173474344b2c93cc8ac295a74729. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:48.379417 24678 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8f9f0e755da4a7186c2c9316918ca73. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:48.380512 24683 tablet_service.cc:1467] Processing CreateTablet for tablet 0d92161e8e3f4a9c88b0ca5fa5555a2c (DEFAULT_TABLE table=test-workload [id=7f5a3e5c2e314e83ba662a913b7b3f17]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20250109 08:58:48.381845 24683 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d92161e8e3f4a9c88b0ca5fa5555a2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:48.383996 24682 tablet_service.cc:1467] Processing CreateTablet for tablet c1c586b8d3a64430bf1fe82b27fef388 (DEFAULT_TABLE table=test-workload [id=7f5a3e5c2e314e83ba662a913b7b3f17]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20250109 08:58:48.385387 24682 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1c586b8d3a64430bf1fe82b27fef388. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:48.385113 24758 tablet_service.cc:1467] Processing CreateTablet for tablet 970b2f8b130d431cb25d79d33d5f0e5c (DEFAULT_TABLE table=test-workload [id=7f5a3e5c2e314e83ba662a913b7b3f17]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20250109 08:58:48.386453 24758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 970b2f8b130d431cb25d79d33d5f0e5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:48.389405 24681 tablet_service.cc:1467] Processing CreateTablet for tablet 98383febcb2145208b05b96598aed748 (DEFAULT_TABLE table=test-workload [id=7f5a3e5c2e314e83ba662a913b7b3f17]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20250109 08:58:48.390357 24756 tablet_service.cc:1467] Processing CreateTablet for tablet c1c586b8d3a64430bf1fe82b27fef388 (DEFAULT_TABLE table=test-workload [id=7f5a3e5c2e314e83ba662a913b7b3f17]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20250109 08:58:48.391651 24756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1c586b8d3a64430bf1fe82b27fef388. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:48.391863 24680 tablet_service.cc:1467] Processing CreateTablet for tablet 630fe12978194fb9ae9f60597b3ce1c1 (DEFAULT_TABLE table=test-workload [id=7f5a3e5c2e314e83ba662a913b7b3f17]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20250109 08:58:48.393946 24679 tablet_service.cc:1467] Processing CreateTablet for tablet 6e2f4cbbf3f744e586058426df0287c2 (DEFAULT_TABLE table=test-workload [id=7f5a3e5c2e314e83ba662a913b7b3f17]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20250109 08:58:48.395982 24755 tablet_service.cc:1467] Processing CreateTablet for tablet 98383febcb2145208b05b96598aed748 (DEFAULT_TABLE table=test-workload [id=7f5a3e5c2e314e83ba662a913b7b3f17]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20250109 08:58:48.397372 24755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98383febcb2145208b05b96598aed748. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:48.397253 24754 tablet_service.cc:1467] Processing CreateTablet for tablet 630fe12978194fb9ae9f60597b3ce1c1 (DEFAULT_TABLE table=test-workload [id=7f5a3e5c2e314e83ba662a913b7b3f17]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20250109 08:58:48.398521 24754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 630fe12978194fb9ae9f60597b3ce1c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:48.389400 24757 tablet_service.cc:1467] Processing CreateTablet for tablet 0d92161e8e3f4a9c88b0ca5fa5555a2c (DEFAULT_TABLE table=test-workload [id=7f5a3e5c2e314e83ba662a913b7b3f17]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20250109 08:58:48.400521 24809 tablet_bootstrap.cc:492] T a930173474344b2c93cc8ac295a74729 P f366ff6c37a24eaaa6abcafab1467912: Bootstrap starting.
I20250109 08:58:48.383991 24752 tablet_service.cc:1467] Processing CreateTablet for tablet d8f9f0e755da4a7186c2c9316918ca73 (DEFAULT_TABLE table=test-workload [id=7f5a3e5c2e314e83ba662a913b7b3f17]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20250109 08:58:48.401516 24751 tablet_service.cc:1467] Processing CreateTablet for tablet 4e2136a82d344ae6b0657803f1551906 (DEFAULT_TABLE table=test-workload [id=7f5a3e5c2e314e83ba662a913b7b3f17]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20250109 08:58:48.390758 24681 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98383febcb2145208b05b96598aed748. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:48.396112 24753 tablet_service.cc:1467] Processing CreateTablet for tablet 6e2f4cbbf3f744e586058426df0287c2 (DEFAULT_TABLE table=test-workload [id=7f5a3e5c2e314e83ba662a913b7b3f17]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20250109 08:58:48.406281 24680 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 630fe12978194fb9ae9f60597b3ce1c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:48.411595 24679 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6e2f4cbbf3f744e586058426df0287c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:48.400676 24757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d92161e8e3f4a9c88b0ca5fa5555a2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:48.422436 24810 tablet_bootstrap.cc:492] T 970b2f8b130d431cb25d79d33d5f0e5c P 1a9b5a4afd63410b813b41edbfe69e28: Bootstrap starting.
I20250109 08:58:48.426291 24677 tablet_service.cc:1467] Processing CreateTablet for tablet 4e2136a82d344ae6b0657803f1551906 (DEFAULT_TABLE table=test-workload [id=7f5a3e5c2e314e83ba662a913b7b3f17]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20250109 08:58:48.487661 24809 tablet_bootstrap.cc:654] T a930173474344b2c93cc8ac295a74729 P f366ff6c37a24eaaa6abcafab1467912: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:48.487917 24677 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e2136a82d344ae6b0657803f1551906. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:48.427287 24610 tablet_service.cc:1467] Processing CreateTablet for tablet a930173474344b2c93cc8ac295a74729 (DEFAULT_TABLE table=test-workload [id=7f5a3e5c2e314e83ba662a913b7b3f17]), partition=RANGE (key) PARTITION VALUES < 238609294
I20250109 08:58:48.490115 24610 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a930173474344b2c93cc8ac295a74729. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:48.491843 24606 tablet_service.cc:1467] Processing CreateTablet for tablet 98383febcb2145208b05b96598aed748 (DEFAULT_TABLE table=test-workload [id=7f5a3e5c2e314e83ba662a913b7b3f17]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20250109 08:58:48.493327 24606 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98383febcb2145208b05b96598aed748. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:48.429097 24609 tablet_service.cc:1467] Processing CreateTablet for tablet 970b2f8b130d431cb25d79d33d5f0e5c (DEFAULT_TABLE table=test-workload [id=7f5a3e5c2e314e83ba662a913b7b3f17]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20250109 08:58:48.495483 24609 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 970b2f8b130d431cb25d79d33d5f0e5c. 1 dirs total, 0 dirs full, 0 dirs failed
W20250109 08:58:48.478801 24502 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.19.226.194:43355, user_credentials={real_user=slave}} blocked reactor thread for 50929.1us
I20250109 08:58:48.483953 24608 tablet_service.cc:1467] Processing CreateTablet for tablet 0d92161e8e3f4a9c88b0ca5fa5555a2c (DEFAULT_TABLE table=test-workload [id=7f5a3e5c2e314e83ba662a913b7b3f17]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20250109 08:58:48.498610 24608 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d92161e8e3f4a9c88b0ca5fa5555a2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:48.485743 24607 tablet_service.cc:1467] Processing CreateTablet for tablet c1c586b8d3a64430bf1fe82b27fef388 (DEFAULT_TABLE table=test-workload [id=7f5a3e5c2e314e83ba662a913b7b3f17]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20250109 08:58:48.500717 24607 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1c586b8d3a64430bf1fe82b27fef388. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:48.503672 24605 tablet_service.cc:1467] Processing CreateTablet for tablet 630fe12978194fb9ae9f60597b3ce1c1 (DEFAULT_TABLE table=test-workload [id=7f5a3e5c2e314e83ba662a913b7b3f17]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20250109 08:58:48.505395 24604 tablet_service.cc:1467] Processing CreateTablet for tablet 6e2f4cbbf3f744e586058426df0287c2 (DEFAULT_TABLE table=test-workload [id=7f5a3e5c2e314e83ba662a913b7b3f17]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20250109 08:58:48.506726 24604 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6e2f4cbbf3f744e586058426df0287c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:48.508610 24603 tablet_service.cc:1467] Processing CreateTablet for tablet d8f9f0e755da4a7186c2c9316918ca73 (DEFAULT_TABLE table=test-workload [id=7f5a3e5c2e314e83ba662a913b7b3f17]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20250109 08:58:48.509848 24603 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8f9f0e755da4a7186c2c9316918ca73. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:48.513386 24602 tablet_service.cc:1467] Processing CreateTablet for tablet 4e2136a82d344ae6b0657803f1551906 (DEFAULT_TABLE table=test-workload [id=7f5a3e5c2e314e83ba662a913b7b3f17]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20250109 08:58:48.424691 24752 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8f9f0e755da4a7186c2c9316918ca73. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:48.515288 24751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e2136a82d344ae6b0657803f1551906. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:48.527339 24810 tablet_bootstrap.cc:654] T 970b2f8b130d431cb25d79d33d5f0e5c P 1a9b5a4afd63410b813b41edbfe69e28: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:48.528720 24753 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6e2f4cbbf3f744e586058426df0287c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:48.528738 24602 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e2136a82d344ae6b0657803f1551906. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:48.532191 24605 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 630fe12978194fb9ae9f60597b3ce1c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:48.533756 24809 tablet_bootstrap.cc:492] T a930173474344b2c93cc8ac295a74729 P f366ff6c37a24eaaa6abcafab1467912: No bootstrap required, opened a new log
I20250109 08:58:48.534346 24809 ts_tablet_manager.cc:1397] T a930173474344b2c93cc8ac295a74729 P f366ff6c37a24eaaa6abcafab1467912: Time spent bootstrapping tablet: real 0.135s user 0.012s sys 0.008s
I20250109 08:58:48.537437 24809 raft_consensus.cc:357] T a930173474344b2c93cc8ac295a74729 P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } }
I20250109 08:58:48.544996 24809 raft_consensus.cc:383] T a930173474344b2c93cc8ac295a74729 P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:48.545396 24809 raft_consensus.cc:738] T a930173474344b2c93cc8ac295a74729 P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f366ff6c37a24eaaa6abcafab1467912, State: Initialized, Role: FOLLOWER
I20250109 08:58:48.546362 24809 consensus_queue.cc:260] T a930173474344b2c93cc8ac295a74729 P f366ff6c37a24eaaa6abcafab1467912 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } }
W20250109 08:58:48.486207 24505 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.19.226.195:42529, user_credentials={real_user=slave}} blocked reactor thread for 44244.7us
I20250109 08:58:48.558583 24720 heartbeater.cc:502] Master 127.19.226.254:45577 was elected leader, sending a full tablet report...
I20250109 08:58:48.559770 24809 ts_tablet_manager.cc:1428] T a930173474344b2c93cc8ac295a74729 P f366ff6c37a24eaaa6abcafab1467912: Time spent starting tablet: real 0.025s user 0.006s sys 0.000s
I20250109 08:58:48.560909 24809 tablet_bootstrap.cc:492] T d8f9f0e755da4a7186c2c9316918ca73 P f366ff6c37a24eaaa6abcafab1467912: Bootstrap starting.
I20250109 08:58:48.566077 24810 tablet_bootstrap.cc:492] T 970b2f8b130d431cb25d79d33d5f0e5c P 1a9b5a4afd63410b813b41edbfe69e28: No bootstrap required, opened a new log
I20250109 08:58:48.566637 24810 ts_tablet_manager.cc:1397] T 970b2f8b130d431cb25d79d33d5f0e5c P 1a9b5a4afd63410b813b41edbfe69e28: Time spent bootstrapping tablet: real 0.144s user 0.033s sys 0.045s
I20250109 08:58:48.566994 24809 tablet_bootstrap.cc:654] T d8f9f0e755da4a7186c2c9316918ca73 P f366ff6c37a24eaaa6abcafab1467912: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:48.569418 24810 raft_consensus.cc:357] T 970b2f8b130d431cb25d79d33d5f0e5c P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:48.570264 24810 raft_consensus.cc:383] T 970b2f8b130d431cb25d79d33d5f0e5c P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:48.570575 24810 raft_consensus.cc:738] T 970b2f8b130d431cb25d79d33d5f0e5c P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a9b5a4afd63410b813b41edbfe69e28, State: Initialized, Role: FOLLOWER
I20250109 08:58:48.571290 24810 consensus_queue.cc:260] T 970b2f8b130d431cb25d79d33d5f0e5c P 1a9b5a4afd63410b813b41edbfe69e28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:48.576253 24794 heartbeater.cc:502] Master 127.19.226.254:45577 was elected leader, sending a full tablet report...
I20250109 08:58:48.576428 24810 ts_tablet_manager.cc:1428] T 970b2f8b130d431cb25d79d33d5f0e5c P 1a9b5a4afd63410b813b41edbfe69e28: Time spent starting tablet: real 0.009s user 0.009s sys 0.000s
I20250109 08:58:48.580199 24809 tablet_bootstrap.cc:492] T d8f9f0e755da4a7186c2c9316918ca73 P f366ff6c37a24eaaa6abcafab1467912: No bootstrap required, opened a new log
I20250109 08:58:48.580818 24809 ts_tablet_manager.cc:1397] T d8f9f0e755da4a7186c2c9316918ca73 P f366ff6c37a24eaaa6abcafab1467912: Time spent bootstrapping tablet: real 0.020s user 0.012s sys 0.004s
I20250109 08:58:48.583603 24809 raft_consensus.cc:357] T d8f9f0e755da4a7186c2c9316918ca73 P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:48.584331 24809 raft_consensus.cc:383] T d8f9f0e755da4a7186c2c9316918ca73 P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:48.584671 24809 raft_consensus.cc:738] T d8f9f0e755da4a7186c2c9316918ca73 P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f366ff6c37a24eaaa6abcafab1467912, State: Initialized, Role: FOLLOWER
I20250109 08:58:48.585368 24809 consensus_queue.cc:260] T d8f9f0e755da4a7186c2c9316918ca73 P f366ff6c37a24eaaa6abcafab1467912 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:48.587575 24809 ts_tablet_manager.cc:1428] T d8f9f0e755da4a7186c2c9316918ca73 P f366ff6c37a24eaaa6abcafab1467912: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20250109 08:58:48.588495 24809 tablet_bootstrap.cc:492] T 0d92161e8e3f4a9c88b0ca5fa5555a2c P f366ff6c37a24eaaa6abcafab1467912: Bootstrap starting.
I20250109 08:58:48.594281 24809 tablet_bootstrap.cc:654] T 0d92161e8e3f4a9c88b0ca5fa5555a2c P f366ff6c37a24eaaa6abcafab1467912: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:48.607671 24815 raft_consensus.cc:491] T 970b2f8b130d431cb25d79d33d5f0e5c P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:48.608233 24815 raft_consensus.cc:513] T 970b2f8b130d431cb25d79d33d5f0e5c P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:48.611414 24815 leader_election.cc:290] T 970b2f8b130d431cb25d79d33d5f0e5c P 1a9b5a4afd63410b813b41edbfe69e28 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 08c70c9a6a064d40bfa68fd105d86d86 (127.19.226.193:43567), f366ff6c37a24eaaa6abcafab1467912 (127.19.226.194:43355)
I20250109 08:58:48.623680 24816 tablet_bootstrap.cc:492] T a930173474344b2c93cc8ac295a74729 P 08c70c9a6a064d40bfa68fd105d86d86: Bootstrap starting.
I20250109 08:58:48.625376 24695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "970b2f8b130d431cb25d79d33d5f0e5c" candidate_uuid: "1a9b5a4afd63410b813b41edbfe69e28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f366ff6c37a24eaaa6abcafab1467912" is_pre_election: true
W20250109 08:58:48.626993 24732 leader_election.cc:343] T 970b2f8b130d431cb25d79d33d5f0e5c P 1a9b5a4afd63410b813b41edbfe69e28 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f366ff6c37a24eaaa6abcafab1467912 (127.19.226.194:43355): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:48.625033 24810 tablet_bootstrap.cc:492] T 630fe12978194fb9ae9f60597b3ce1c1 P 1a9b5a4afd63410b813b41edbfe69e28: Bootstrap starting.
I20250109 08:58:48.631896 24809 tablet_bootstrap.cc:492] T 0d92161e8e3f4a9c88b0ca5fa5555a2c P f366ff6c37a24eaaa6abcafab1467912: No bootstrap required, opened a new log
I20250109 08:58:48.633920 24809 ts_tablet_manager.cc:1397] T 0d92161e8e3f4a9c88b0ca5fa5555a2c P f366ff6c37a24eaaa6abcafab1467912: Time spent bootstrapping tablet: real 0.046s user 0.008s sys 0.011s
I20250109 08:58:48.636713 24809 raft_consensus.cc:357] T 0d92161e8e3f4a9c88b0ca5fa5555a2c P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:48.637449 24809 raft_consensus.cc:383] T 0d92161e8e3f4a9c88b0ca5fa5555a2c P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:48.637755 24809 raft_consensus.cc:738] T 0d92161e8e3f4a9c88b0ca5fa5555a2c P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f366ff6c37a24eaaa6abcafab1467912, State: Initialized, Role: FOLLOWER
I20250109 08:58:48.638409 24809 consensus_queue.cc:260] T 0d92161e8e3f4a9c88b0ca5fa5555a2c P f366ff6c37a24eaaa6abcafab1467912 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:48.639360 24620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "970b2f8b130d431cb25d79d33d5f0e5c" candidate_uuid: "1a9b5a4afd63410b813b41edbfe69e28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08c70c9a6a064d40bfa68fd105d86d86" is_pre_election: true
I20250109 08:58:48.640643 24809 ts_tablet_manager.cc:1428] T 0d92161e8e3f4a9c88b0ca5fa5555a2c P f366ff6c37a24eaaa6abcafab1467912: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20250109 08:58:48.641460 24816 tablet_bootstrap.cc:654] T a930173474344b2c93cc8ac295a74729 P 08c70c9a6a064d40bfa68fd105d86d86: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:48.641588 24809 tablet_bootstrap.cc:492] T 6e2f4cbbf3f744e586058426df0287c2 P f366ff6c37a24eaaa6abcafab1467912: Bootstrap starting.
I20250109 08:58:48.643558 24810 tablet_bootstrap.cc:654] T 630fe12978194fb9ae9f60597b3ce1c1 P 1a9b5a4afd63410b813b41edbfe69e28: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:48.647584 24809 tablet_bootstrap.cc:654] T 6e2f4cbbf3f744e586058426df0287c2 P f366ff6c37a24eaaa6abcafab1467912: Neither blocks nor log segments found. Creating new log.
W20250109 08:58:48.647922 24735 leader_election.cc:343] T 970b2f8b130d431cb25d79d33d5f0e5c P 1a9b5a4afd63410b813b41edbfe69e28 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 08c70c9a6a064d40bfa68fd105d86d86 (127.19.226.193:43567): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:48.648481 24735 leader_election.cc:304] T 970b2f8b130d431cb25d79d33d5f0e5c P 1a9b5a4afd63410b813b41edbfe69e28 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1a9b5a4afd63410b813b41edbfe69e28; no voters: 08c70c9a6a064d40bfa68fd105d86d86, f366ff6c37a24eaaa6abcafab1467912
I20250109 08:58:48.649266 24815 raft_consensus.cc:2743] T 970b2f8b130d431cb25d79d33d5f0e5c P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:58:48.652806 24816 tablet_bootstrap.cc:492] T a930173474344b2c93cc8ac295a74729 P 08c70c9a6a064d40bfa68fd105d86d86: No bootstrap required, opened a new log
I20250109 08:58:48.653019 24809 tablet_bootstrap.cc:492] T 6e2f4cbbf3f744e586058426df0287c2 P f366ff6c37a24eaaa6abcafab1467912: No bootstrap required, opened a new log
I20250109 08:58:48.653340 24810 tablet_bootstrap.cc:492] T 630fe12978194fb9ae9f60597b3ce1c1 P 1a9b5a4afd63410b813b41edbfe69e28: No bootstrap required, opened a new log
I20250109 08:58:48.653473 24809 ts_tablet_manager.cc:1397] T 6e2f4cbbf3f744e586058426df0287c2 P f366ff6c37a24eaaa6abcafab1467912: Time spent bootstrapping tablet: real 0.012s user 0.011s sys 0.000s
I20250109 08:58:48.653352 24816 ts_tablet_manager.cc:1397] T a930173474344b2c93cc8ac295a74729 P 08c70c9a6a064d40bfa68fd105d86d86: Time spent bootstrapping tablet: real 0.048s user 0.011s sys 0.007s
I20250109 08:58:48.654054 24810 ts_tablet_manager.cc:1397] T 630fe12978194fb9ae9f60597b3ce1c1 P 1a9b5a4afd63410b813b41edbfe69e28: Time spent bootstrapping tablet: real 0.029s user 0.012s sys 0.005s
I20250109 08:58:48.656090 24810 raft_consensus.cc:357] T 630fe12978194fb9ae9f60597b3ce1c1 P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:48.656747 24810 raft_consensus.cc:383] T 630fe12978194fb9ae9f60597b3ce1c1 P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:48.656570 24809 raft_consensus.cc:357] T 6e2f4cbbf3f744e586058426df0287c2 P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:48.657207 24810 raft_consensus.cc:738] T 630fe12978194fb9ae9f60597b3ce1c1 P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a9b5a4afd63410b813b41edbfe69e28, State: Initialized, Role: FOLLOWER
I20250109 08:58:48.656775 24816 raft_consensus.cc:357] T a930173474344b2c93cc8ac295a74729 P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } }
I20250109 08:58:48.657419 24809 raft_consensus.cc:383] T 6e2f4cbbf3f744e586058426df0287c2 P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:48.657709 24816 raft_consensus.cc:383] T a930173474344b2c93cc8ac295a74729 P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:48.657738 24809 raft_consensus.cc:738] T 6e2f4cbbf3f744e586058426df0287c2 P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f366ff6c37a24eaaa6abcafab1467912, State: Initialized, Role: FOLLOWER
I20250109 08:58:48.658128 24816 raft_consensus.cc:738] T a930173474344b2c93cc8ac295a74729 P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08c70c9a6a064d40bfa68fd105d86d86, State: Initialized, Role: FOLLOWER
I20250109 08:58:48.657964 24810 consensus_queue.cc:260] T 630fe12978194fb9ae9f60597b3ce1c1 P 1a9b5a4afd63410b813b41edbfe69e28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:48.658620 24809 consensus_queue.cc:260] T 6e2f4cbbf3f744e586058426df0287c2 P f366ff6c37a24eaaa6abcafab1467912 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:48.658970 24816 consensus_queue.cc:260] T a930173474344b2c93cc8ac295a74729 P 08c70c9a6a064d40bfa68fd105d86d86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } }
I20250109 08:58:48.661099 24809 ts_tablet_manager.cc:1428] T 6e2f4cbbf3f744e586058426df0287c2 P f366ff6c37a24eaaa6abcafab1467912: Time spent starting tablet: real 0.007s user 0.005s sys 0.002s
I20250109 08:58:48.662248 24809 tablet_bootstrap.cc:492] T c1c586b8d3a64430bf1fe82b27fef388 P f366ff6c37a24eaaa6abcafab1467912: Bootstrap starting.
I20250109 08:58:48.662482 24810 ts_tablet_manager.cc:1428] T 630fe12978194fb9ae9f60597b3ce1c1 P 1a9b5a4afd63410b813b41edbfe69e28: Time spent starting tablet: real 0.008s user 0.004s sys 0.002s
I20250109 08:58:48.663414 24810 tablet_bootstrap.cc:492] T a930173474344b2c93cc8ac295a74729 P 1a9b5a4afd63410b813b41edbfe69e28: Bootstrap starting.
I20250109 08:58:48.668700 24809 tablet_bootstrap.cc:654] T c1c586b8d3a64430bf1fe82b27fef388 P f366ff6c37a24eaaa6abcafab1467912: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:48.669142 24810 tablet_bootstrap.cc:654] T a930173474344b2c93cc8ac295a74729 P 1a9b5a4afd63410b813b41edbfe69e28: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:48.682226 24810 tablet_bootstrap.cc:492] T a930173474344b2c93cc8ac295a74729 P 1a9b5a4afd63410b813b41edbfe69e28: No bootstrap required, opened a new log
I20250109 08:58:48.682700 24810 ts_tablet_manager.cc:1397] T a930173474344b2c93cc8ac295a74729 P 1a9b5a4afd63410b813b41edbfe69e28: Time spent bootstrapping tablet: real 0.019s user 0.016s sys 0.001s
I20250109 08:58:48.683295 24816 ts_tablet_manager.cc:1428] T a930173474344b2c93cc8ac295a74729 P 08c70c9a6a064d40bfa68fd105d86d86: Time spent starting tablet: real 0.029s user 0.010s sys 0.013s
I20250109 08:58:48.683387 24809 tablet_bootstrap.cc:492] T c1c586b8d3a64430bf1fe82b27fef388 P f366ff6c37a24eaaa6abcafab1467912: No bootstrap required, opened a new log
I20250109 08:58:48.683296 24645 heartbeater.cc:502] Master 127.19.226.254:45577 was elected leader, sending a full tablet report...
I20250109 08:58:48.684149 24809 ts_tablet_manager.cc:1397] T c1c586b8d3a64430bf1fe82b27fef388 P f366ff6c37a24eaaa6abcafab1467912: Time spent bootstrapping tablet: real 0.022s user 0.009s sys 0.012s
I20250109 08:58:48.684480 24816 tablet_bootstrap.cc:492] T 6e2f4cbbf3f744e586058426df0287c2 P 08c70c9a6a064d40bfa68fd105d86d86: Bootstrap starting.
I20250109 08:58:48.685463 24810 raft_consensus.cc:357] T a930173474344b2c93cc8ac295a74729 P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } }
I20250109 08:58:48.686244 24810 raft_consensus.cc:383] T a930173474344b2c93cc8ac295a74729 P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:48.686529 24810 raft_consensus.cc:738] T a930173474344b2c93cc8ac295a74729 P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a9b5a4afd63410b813b41edbfe69e28, State: Initialized, Role: FOLLOWER
I20250109 08:58:48.686951 24809 raft_consensus.cc:357] T c1c586b8d3a64430bf1fe82b27fef388 P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:48.687880 24809 raft_consensus.cc:383] T c1c586b8d3a64430bf1fe82b27fef388 P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:48.688328 24809 raft_consensus.cc:738] T c1c586b8d3a64430bf1fe82b27fef388 P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f366ff6c37a24eaaa6abcafab1467912, State: Initialized, Role: FOLLOWER
I20250109 08:58:48.689386 24809 consensus_queue.cc:260] T c1c586b8d3a64430bf1fe82b27fef388 P f366ff6c37a24eaaa6abcafab1467912 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:48.690385 24810 consensus_queue.cc:260] T a930173474344b2c93cc8ac295a74729 P 1a9b5a4afd63410b813b41edbfe69e28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } }
I20250109 08:58:48.691191 24816 tablet_bootstrap.cc:654] T 6e2f4cbbf3f744e586058426df0287c2 P 08c70c9a6a064d40bfa68fd105d86d86: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:48.692971 24810 ts_tablet_manager.cc:1428] T a930173474344b2c93cc8ac295a74729 P 1a9b5a4afd63410b813b41edbfe69e28: Time spent starting tablet: real 0.010s user 0.006s sys 0.001s
I20250109 08:58:48.694116 24810 tablet_bootstrap.cc:492] T 0d92161e8e3f4a9c88b0ca5fa5555a2c P 1a9b5a4afd63410b813b41edbfe69e28: Bootstrap starting.
I20250109 08:58:48.697132 24809 ts_tablet_manager.cc:1428] T c1c586b8d3a64430bf1fe82b27fef388 P f366ff6c37a24eaaa6abcafab1467912: Time spent starting tablet: real 0.013s user 0.007s sys 0.000s
I20250109 08:58:48.698433 24809 tablet_bootstrap.cc:492] T 98383febcb2145208b05b96598aed748 P f366ff6c37a24eaaa6abcafab1467912: Bootstrap starting.
I20250109 08:58:48.701254 24810 tablet_bootstrap.cc:654] T 0d92161e8e3f4a9c88b0ca5fa5555a2c P 1a9b5a4afd63410b813b41edbfe69e28: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:48.703166 24816 tablet_bootstrap.cc:492] T 6e2f4cbbf3f744e586058426df0287c2 P 08c70c9a6a064d40bfa68fd105d86d86: No bootstrap required, opened a new log
I20250109 08:58:48.703716 24816 ts_tablet_manager.cc:1397] T 6e2f4cbbf3f744e586058426df0287c2 P 08c70c9a6a064d40bfa68fd105d86d86: Time spent bootstrapping tablet: real 0.019s user 0.012s sys 0.004s
I20250109 08:58:48.704932 24809 tablet_bootstrap.cc:654] T 98383febcb2145208b05b96598aed748 P f366ff6c37a24eaaa6abcafab1467912: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:48.706856 24816 raft_consensus.cc:357] T 6e2f4cbbf3f744e586058426df0287c2 P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:48.707772 24816 raft_consensus.cc:383] T 6e2f4cbbf3f744e586058426df0287c2 P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:48.708138 24816 raft_consensus.cc:738] T 6e2f4cbbf3f744e586058426df0287c2 P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08c70c9a6a064d40bfa68fd105d86d86, State: Initialized, Role: FOLLOWER
I20250109 08:58:48.708981 24816 consensus_queue.cc:260] T 6e2f4cbbf3f744e586058426df0287c2 P 08c70c9a6a064d40bfa68fd105d86d86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:48.711568 24816 ts_tablet_manager.cc:1428] T 6e2f4cbbf3f744e586058426df0287c2 P 08c70c9a6a064d40bfa68fd105d86d86: Time spent starting tablet: real 0.008s user 0.008s sys 0.000s
I20250109 08:58:48.712558 24816 tablet_bootstrap.cc:492] T 4e2136a82d344ae6b0657803f1551906 P 08c70c9a6a064d40bfa68fd105d86d86: Bootstrap starting.
I20250109 08:58:48.718639 24813 raft_consensus.cc:491] T c1c586b8d3a64430bf1fe82b27fef388 P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:48.718973 24816 tablet_bootstrap.cc:654] T 4e2136a82d344ae6b0657803f1551906 P 08c70c9a6a064d40bfa68fd105d86d86: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:48.719172 24813 raft_consensus.cc:513] T c1c586b8d3a64430bf1fe82b27fef388 P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:48.721817 24813 leader_election.cc:290] T c1c586b8d3a64430bf1fe82b27fef388 P f366ff6c37a24eaaa6abcafab1467912 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 08c70c9a6a064d40bfa68fd105d86d86 (127.19.226.193:43567), 1a9b5a4afd63410b813b41edbfe69e28 (127.19.226.195:42529)
I20250109 08:58:48.722975 24810 tablet_bootstrap.cc:492] T 0d92161e8e3f4a9c88b0ca5fa5555a2c P 1a9b5a4afd63410b813b41edbfe69e28: No bootstrap required, opened a new log
I20250109 08:58:48.723527 24810 ts_tablet_manager.cc:1397] T 0d92161e8e3f4a9c88b0ca5fa5555a2c P 1a9b5a4afd63410b813b41edbfe69e28: Time spent bootstrapping tablet: real 0.030s user 0.015s sys 0.010s
I20250109 08:58:48.726366 24810 raft_consensus.cc:357] T 0d92161e8e3f4a9c88b0ca5fa5555a2c P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:48.727145 24810 raft_consensus.cc:383] T 0d92161e8e3f4a9c88b0ca5fa5555a2c P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:48.727556 24810 raft_consensus.cc:738] T 0d92161e8e3f4a9c88b0ca5fa5555a2c P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a9b5a4afd63410b813b41edbfe69e28, State: Initialized, Role: FOLLOWER
I20250109 08:58:48.730890 24810 consensus_queue.cc:260] T 0d92161e8e3f4a9c88b0ca5fa5555a2c P 1a9b5a4afd63410b813b41edbfe69e28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:48.732867 24809 tablet_bootstrap.cc:492] T 98383febcb2145208b05b96598aed748 P f366ff6c37a24eaaa6abcafab1467912: No bootstrap required, opened a new log
I20250109 08:58:48.733381 24809 ts_tablet_manager.cc:1397] T 98383febcb2145208b05b96598aed748 P f366ff6c37a24eaaa6abcafab1467912: Time spent bootstrapping tablet: real 0.035s user 0.015s sys 0.011s
I20250109 08:58:48.735102 24810 ts_tablet_manager.cc:1428] T 0d92161e8e3f4a9c88b0ca5fa5555a2c P 1a9b5a4afd63410b813b41edbfe69e28: Time spent starting tablet: real 0.011s user 0.006s sys 0.002s
I20250109 08:58:48.736001 24809 raft_consensus.cc:357] T 98383febcb2145208b05b96598aed748 P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:48.736207 24810 tablet_bootstrap.cc:492] T c1c586b8d3a64430bf1fe82b27fef388 P 1a9b5a4afd63410b813b41edbfe69e28: Bootstrap starting.
I20250109 08:58:48.736850 24809 raft_consensus.cc:383] T 98383febcb2145208b05b96598aed748 P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:48.737648 24809 raft_consensus.cc:738] T 98383febcb2145208b05b96598aed748 P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f366ff6c37a24eaaa6abcafab1467912, State: Initialized, Role: FOLLOWER
I20250109 08:58:48.738448 24809 consensus_queue.cc:260] T 98383febcb2145208b05b96598aed748 P f366ff6c37a24eaaa6abcafab1467912 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:48.741391 24816 tablet_bootstrap.cc:492] T 4e2136a82d344ae6b0657803f1551906 P 08c70c9a6a064d40bfa68fd105d86d86: No bootstrap required, opened a new log
I20250109 08:58:48.742269 24816 ts_tablet_manager.cc:1397] T 4e2136a82d344ae6b0657803f1551906 P 08c70c9a6a064d40bfa68fd105d86d86: Time spent bootstrapping tablet: real 0.030s user 0.011s sys 0.002s
I20250109 08:58:48.744802 24810 tablet_bootstrap.cc:654] T c1c586b8d3a64430bf1fe82b27fef388 P 1a9b5a4afd63410b813b41edbfe69e28: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:48.746599 24809 ts_tablet_manager.cc:1428] T 98383febcb2145208b05b96598aed748 P f366ff6c37a24eaaa6abcafab1467912: Time spent starting tablet: real 0.013s user 0.007s sys 0.000s
I20250109 08:58:48.747617 24809 tablet_bootstrap.cc:492] T 630fe12978194fb9ae9f60597b3ce1c1 P f366ff6c37a24eaaa6abcafab1467912: Bootstrap starting.
I20250109 08:58:48.751256 24822 raft_consensus.cc:491] T a930173474344b2c93cc8ac295a74729 P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:48.751760 24822 raft_consensus.cc:513] T a930173474344b2c93cc8ac295a74729 P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } }
I20250109 08:58:48.753610 24769 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1c586b8d3a64430bf1fe82b27fef388" candidate_uuid: "f366ff6c37a24eaaa6abcafab1467912" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a9b5a4afd63410b813b41edbfe69e28" is_pre_election: true
I20250109 08:58:48.754652 24822 leader_election.cc:290] T a930173474344b2c93cc8ac295a74729 P 08c70c9a6a064d40bfa68fd105d86d86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1a9b5a4afd63410b813b41edbfe69e28 (127.19.226.195:42529), f366ff6c37a24eaaa6abcafab1467912 (127.19.226.194:43355)
W20250109 08:58:48.755093 24661 leader_election.cc:343] T c1c586b8d3a64430bf1fe82b27fef388 P f366ff6c37a24eaaa6abcafab1467912 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1a9b5a4afd63410b813b41edbfe69e28 (127.19.226.195:42529): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:48.759761 24816 raft_consensus.cc:357] T 4e2136a82d344ae6b0657803f1551906 P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:48.760608 24816 raft_consensus.cc:383] T 4e2136a82d344ae6b0657803f1551906 P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:48.760902 24816 raft_consensus.cc:738] T 4e2136a82d344ae6b0657803f1551906 P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08c70c9a6a064d40bfa68fd105d86d86, State: Initialized, Role: FOLLOWER
I20250109 08:58:48.761694 24809 tablet_bootstrap.cc:654] T 630fe12978194fb9ae9f60597b3ce1c1 P f366ff6c37a24eaaa6abcafab1467912: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:48.761533 24816 consensus_queue.cc:260] T 4e2136a82d344ae6b0657803f1551906 P 08c70c9a6a064d40bfa68fd105d86d86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:48.770401 24816 ts_tablet_manager.cc:1428] T 4e2136a82d344ae6b0657803f1551906 P 08c70c9a6a064d40bfa68fd105d86d86: Time spent starting tablet: real 0.028s user 0.005s sys 0.000s
I20250109 08:58:48.771085 24620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1c586b8d3a64430bf1fe82b27fef388" candidate_uuid: "f366ff6c37a24eaaa6abcafab1467912" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08c70c9a6a064d40bfa68fd105d86d86" is_pre_election: true
I20250109 08:58:48.771351 24816 tablet_bootstrap.cc:492] T 0d92161e8e3f4a9c88b0ca5fa5555a2c P 08c70c9a6a064d40bfa68fd105d86d86: Bootstrap starting.
W20250109 08:58:48.772608 24661 leader_election.cc:343] T c1c586b8d3a64430bf1fe82b27fef388 P f366ff6c37a24eaaa6abcafab1467912 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 08c70c9a6a064d40bfa68fd105d86d86 (127.19.226.193:43567): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:48.773095 24661 leader_election.cc:304] T c1c586b8d3a64430bf1fe82b27fef388 P f366ff6c37a24eaaa6abcafab1467912 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f366ff6c37a24eaaa6abcafab1467912; no voters: 08c70c9a6a064d40bfa68fd105d86d86, 1a9b5a4afd63410b813b41edbfe69e28
I20250109 08:58:48.774273 24813 raft_consensus.cc:2743] T c1c586b8d3a64430bf1fe82b27fef388 P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:58:48.778147 24816 tablet_bootstrap.cc:654] T 0d92161e8e3f4a9c88b0ca5fa5555a2c P 08c70c9a6a064d40bfa68fd105d86d86: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:48.779224 24809 tablet_bootstrap.cc:492] T 630fe12978194fb9ae9f60597b3ce1c1 P f366ff6c37a24eaaa6abcafab1467912: No bootstrap required, opened a new log
I20250109 08:58:48.779744 24809 ts_tablet_manager.cc:1397] T 630fe12978194fb9ae9f60597b3ce1c1 P f366ff6c37a24eaaa6abcafab1467912: Time spent bootstrapping tablet: real 0.032s user 0.011s sys 0.000s
I20250109 08:58:48.782600 24809 raft_consensus.cc:357] T 630fe12978194fb9ae9f60597b3ce1c1 P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:48.783309 24809 raft_consensus.cc:383] T 630fe12978194fb9ae9f60597b3ce1c1 P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:48.783602 24809 raft_consensus.cc:738] T 630fe12978194fb9ae9f60597b3ce1c1 P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f366ff6c37a24eaaa6abcafab1467912, State: Initialized, Role: FOLLOWER
I20250109 08:58:48.785853 24816 tablet_bootstrap.cc:492] T 0d92161e8e3f4a9c88b0ca5fa5555a2c P 08c70c9a6a064d40bfa68fd105d86d86: No bootstrap required, opened a new log
I20250109 08:58:48.786188 24810 tablet_bootstrap.cc:492] T c1c586b8d3a64430bf1fe82b27fef388 P 1a9b5a4afd63410b813b41edbfe69e28: No bootstrap required, opened a new log
I20250109 08:58:48.784277 24809 consensus_queue.cc:260] T 630fe12978194fb9ae9f60597b3ce1c1 P f366ff6c37a24eaaa6abcafab1467912 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:48.786314 24816 ts_tablet_manager.cc:1397] T 0d92161e8e3f4a9c88b0ca5fa5555a2c P 08c70c9a6a064d40bfa68fd105d86d86: Time spent bootstrapping tablet: real 0.015s user 0.012s sys 0.000s
I20250109 08:58:48.786693 24810 ts_tablet_manager.cc:1397] T c1c586b8d3a64430bf1fe82b27fef388 P 1a9b5a4afd63410b813b41edbfe69e28: Time spent bootstrapping tablet: real 0.051s user 0.023s sys 0.016s
I20250109 08:58:48.789453 24810 raft_consensus.cc:357] T c1c586b8d3a64430bf1fe82b27fef388 P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:48.790122 24810 raft_consensus.cc:383] T c1c586b8d3a64430bf1fe82b27fef388 P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:48.790390 24810 raft_consensus.cc:738] T c1c586b8d3a64430bf1fe82b27fef388 P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a9b5a4afd63410b813b41edbfe69e28, State: Initialized, Role: FOLLOWER
I20250109 08:58:48.790693 24816 raft_consensus.cc:357] T 0d92161e8e3f4a9c88b0ca5fa5555a2c P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:48.791494 24816 raft_consensus.cc:383] T 0d92161e8e3f4a9c88b0ca5fa5555a2c P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:48.791954 24816 raft_consensus.cc:738] T 0d92161e8e3f4a9c88b0ca5fa5555a2c P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08c70c9a6a064d40bfa68fd105d86d86, State: Initialized, Role: FOLLOWER
I20250109 08:58:48.791502 24810 consensus_queue.cc:260] T c1c586b8d3a64430bf1fe82b27fef388 P 1a9b5a4afd63410b813b41edbfe69e28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:48.792716 24816 consensus_queue.cc:260] T 0d92161e8e3f4a9c88b0ca5fa5555a2c P 08c70c9a6a064d40bfa68fd105d86d86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:48.793387 24695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a930173474344b2c93cc8ac295a74729" candidate_uuid: "08c70c9a6a064d40bfa68fd105d86d86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f366ff6c37a24eaaa6abcafab1467912" is_pre_election: true
I20250109 08:58:48.794044 24695 raft_consensus.cc:2463] T a930173474344b2c93cc8ac295a74729 P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 08c70c9a6a064d40bfa68fd105d86d86 in term 0.
I20250109 08:58:48.794904 24809 ts_tablet_manager.cc:1428] T 630fe12978194fb9ae9f60597b3ce1c1 P f366ff6c37a24eaaa6abcafab1467912: Time spent starting tablet: real 0.015s user 0.005s sys 0.004s
I20250109 08:58:48.795634 24583 leader_election.cc:304] T a930173474344b2c93cc8ac295a74729 P 08c70c9a6a064d40bfa68fd105d86d86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08c70c9a6a064d40bfa68fd105d86d86, f366ff6c37a24eaaa6abcafab1467912; no voters:
I20250109 08:58:48.795974 24809 tablet_bootstrap.cc:492] T 970b2f8b130d431cb25d79d33d5f0e5c P f366ff6c37a24eaaa6abcafab1467912: Bootstrap starting.
I20250109 08:58:48.797521 24822 raft_consensus.cc:2798] T a930173474344b2c93cc8ac295a74729 P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:58:48.797912 24822 raft_consensus.cc:491] T a930173474344b2c93cc8ac295a74729 P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:58:48.798271 24822 raft_consensus.cc:3054] T a930173474344b2c93cc8ac295a74729 P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:48.801651 24810 ts_tablet_manager.cc:1428] T c1c586b8d3a64430bf1fe82b27fef388 P 1a9b5a4afd63410b813b41edbfe69e28: Time spent starting tablet: real 0.015s user 0.006s sys 0.000s
I20250109 08:58:48.802273 24809 tablet_bootstrap.cc:654] T 970b2f8b130d431cb25d79d33d5f0e5c P f366ff6c37a24eaaa6abcafab1467912: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:48.805047 24810 tablet_bootstrap.cc:492] T 98383febcb2145208b05b96598aed748 P 1a9b5a4afd63410b813b41edbfe69e28: Bootstrap starting.
I20250109 08:58:48.809968 24815 raft_consensus.cc:491] T a930173474344b2c93cc8ac295a74729 P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:48.810467 24815 raft_consensus.cc:513] T a930173474344b2c93cc8ac295a74729 P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } }
I20250109 08:58:48.811334 24810 tablet_bootstrap.cc:654] T 98383febcb2145208b05b96598aed748 P 1a9b5a4afd63410b813b41edbfe69e28: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:48.812002 24822 raft_consensus.cc:513] T a930173474344b2c93cc8ac295a74729 P 08c70c9a6a064d40bfa68fd105d86d86 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } }
I20250109 08:58:48.813467 24695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a930173474344b2c93cc8ac295a74729" candidate_uuid: "1a9b5a4afd63410b813b41edbfe69e28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f366ff6c37a24eaaa6abcafab1467912" is_pre_election: true
I20250109 08:58:48.814116 24695 raft_consensus.cc:2463] T a930173474344b2c93cc8ac295a74729 P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a9b5a4afd63410b813b41edbfe69e28 in term 0.
I20250109 08:58:48.815423 24816 ts_tablet_manager.cc:1428] T 0d92161e8e3f4a9c88b0ca5fa5555a2c P 08c70c9a6a064d40bfa68fd105d86d86: Time spent starting tablet: real 0.029s user 0.005s sys 0.000s
I20250109 08:58:48.815853 24620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a930173474344b2c93cc8ac295a74729" candidate_uuid: "1a9b5a4afd63410b813b41edbfe69e28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08c70c9a6a064d40bfa68fd105d86d86" is_pre_election: true
I20250109 08:58:48.816498 24822 leader_election.cc:290] T a930173474344b2c93cc8ac295a74729 P 08c70c9a6a064d40bfa68fd105d86d86 [CANDIDATE]: Term 1 election: Requested vote from peers 1a9b5a4afd63410b813b41edbfe69e28 (127.19.226.195:42529), f366ff6c37a24eaaa6abcafab1467912 (127.19.226.194:43355)
I20250109 08:58:48.816991 24732 leader_election.cc:304] T a930173474344b2c93cc8ac295a74729 P 1a9b5a4afd63410b813b41edbfe69e28 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a9b5a4afd63410b813b41edbfe69e28, f366ff6c37a24eaaa6abcafab1467912; no voters:
I20250109 08:58:48.818575 24620 raft_consensus.cc:2388] T a930173474344b2c93cc8ac295a74729 P 08c70c9a6a064d40bfa68fd105d86d86 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1a9b5a4afd63410b813b41edbfe69e28 in current term 1: Already voted for candidate 08c70c9a6a064d40bfa68fd105d86d86 in this term.
I20250109 08:58:48.821182 24695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a930173474344b2c93cc8ac295a74729" candidate_uuid: "08c70c9a6a064d40bfa68fd105d86d86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f366ff6c37a24eaaa6abcafab1467912"
I20250109 08:58:48.821844 24695 raft_consensus.cc:3054] T a930173474344b2c93cc8ac295a74729 P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:48.824349 24813 raft_consensus.cc:491] T 6e2f4cbbf3f744e586058426df0287c2 P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:48.824627 24816 tablet_bootstrap.cc:492] T 98383febcb2145208b05b96598aed748 P 08c70c9a6a064d40bfa68fd105d86d86: Bootstrap starting.
I20250109 08:58:48.825022 24813 raft_consensus.cc:513] T 6e2f4cbbf3f744e586058426df0287c2 P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:48.827178 24813 leader_election.cc:290] T 6e2f4cbbf3f744e586058426df0287c2 P f366ff6c37a24eaaa6abcafab1467912 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 08c70c9a6a064d40bfa68fd105d86d86 (127.19.226.193:43567), 1a9b5a4afd63410b813b41edbfe69e28 (127.19.226.195:42529)
I20250109 08:58:48.830287 24815 leader_election.cc:290] T a930173474344b2c93cc8ac295a74729 P 1a9b5a4afd63410b813b41edbfe69e28 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f366ff6c37a24eaaa6abcafab1467912 (127.19.226.194:43355), 08c70c9a6a064d40bfa68fd105d86d86 (127.19.226.193:43567)
I20250109 08:58:48.831519 24769 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e2f4cbbf3f744e586058426df0287c2" candidate_uuid: "f366ff6c37a24eaaa6abcafab1467912" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a9b5a4afd63410b813b41edbfe69e28" is_pre_election: true
I20250109 08:58:48.833225 24816 tablet_bootstrap.cc:654] T 98383febcb2145208b05b96598aed748 P 08c70c9a6a064d40bfa68fd105d86d86: Neither blocks nor log segments found. Creating new log.
W20250109 08:58:48.838097 24661 leader_election.cc:343] T 6e2f4cbbf3f744e586058426df0287c2 P f366ff6c37a24eaaa6abcafab1467912 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1a9b5a4afd63410b813b41edbfe69e28 (127.19.226.195:42529): Illegal state: must be running to vote when last-logged opid is not known
W20250109 08:58:48.918074 24661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.226.195:42529, user_credentials={real_user=slave}} blocked reactor thread for 80032.5us
I20250109 08:58:48.838615 24768 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a930173474344b2c93cc8ac295a74729" candidate_uuid: "08c70c9a6a064d40bfa68fd105d86d86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a9b5a4afd63410b813b41edbfe69e28"
I20250109 08:58:48.837836 24769 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a930173474344b2c93cc8ac295a74729" candidate_uuid: "08c70c9a6a064d40bfa68fd105d86d86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a9b5a4afd63410b813b41edbfe69e28" is_pre_election: true
I20250109 08:58:48.919426 24809 tablet_bootstrap.cc:492] T 970b2f8b130d431cb25d79d33d5f0e5c P f366ff6c37a24eaaa6abcafab1467912: No bootstrap required, opened a new log
I20250109 08:58:48.838097 24620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e2f4cbbf3f744e586058426df0287c2" candidate_uuid: "f366ff6c37a24eaaa6abcafab1467912" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08c70c9a6a064d40bfa68fd105d86d86" is_pre_election: true
W20250109 08:58:48.919312 24566 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20250109 08:58:48.920182 24809 ts_tablet_manager.cc:1397] T 970b2f8b130d431cb25d79d33d5f0e5c P f366ff6c37a24eaaa6abcafab1467912: Time spent bootstrapping tablet: real 0.124s user 0.015s sys 0.019s
I20250109 08:58:48.920423 24620 raft_consensus.cc:2463] T 6e2f4cbbf3f744e586058426df0287c2 P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f366ff6c37a24eaaa6abcafab1467912 in term 0.
I20250109 08:58:48.846374 24822 raft_consensus.cc:491] T 4e2136a82d344ae6b0657803f1551906 P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:48.897215 24813 raft_consensus.cc:491] T 0d92161e8e3f4a9c88b0ca5fa5555a2c P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:48.922740 24813 raft_consensus.cc:513] T 0d92161e8e3f4a9c88b0ca5fa5555a2c P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:48.923182 24816 tablet_bootstrap.cc:492] T 98383febcb2145208b05b96598aed748 P 08c70c9a6a064d40bfa68fd105d86d86: No bootstrap required, opened a new log
I20250109 08:58:48.923729 24816 ts_tablet_manager.cc:1397] T 98383febcb2145208b05b96598aed748 P 08c70c9a6a064d40bfa68fd105d86d86: Time spent bootstrapping tablet: real 0.099s user 0.012s sys 0.007s
I20250109 08:58:48.924002 24809 raft_consensus.cc:357] T 970b2f8b130d431cb25d79d33d5f0e5c P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:48.924790 24809 raft_consensus.cc:383] T 970b2f8b130d431cb25d79d33d5f0e5c P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:48.836782 24815 raft_consensus.cc:2798] T a930173474344b2c93cc8ac295a74729 P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:58:48.925462 24815 raft_consensus.cc:491] T a930173474344b2c93cc8ac295a74729 P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:58:48.925853 24815 raft_consensus.cc:3054] T a930173474344b2c93cc8ac295a74729 P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Advancing to term 1
W20250109 08:58:48.916766 24732 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.19.226.194:43355, user_credentials={real_user=slave}} blocked reactor thread for 99876.1us
I20250109 08:58:48.926484 24816 raft_consensus.cc:357] T 98383febcb2145208b05b96598aed748 P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:48.927270 24816 raft_consensus.cc:383] T 98383febcb2145208b05b96598aed748 P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:48.927579 24816 raft_consensus.cc:738] T 98383febcb2145208b05b96598aed748 P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08c70c9a6a064d40bfa68fd105d86d86, State: Initialized, Role: FOLLOWER
I20250109 08:58:48.921330 24822 raft_consensus.cc:513] T 4e2136a82d344ae6b0657803f1551906 P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:48.928293 24816 consensus_queue.cc:260] T 98383febcb2145208b05b96598aed748 P 08c70c9a6a064d40bfa68fd105d86d86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:48.929749 24822 leader_election.cc:290] T 4e2136a82d344ae6b0657803f1551906 P 08c70c9a6a064d40bfa68fd105d86d86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f366ff6c37a24eaaa6abcafab1467912 (127.19.226.194:43355), 1a9b5a4afd63410b813b41edbfe69e28 (127.19.226.195:42529)
I20250109 08:58:48.930292 24822 raft_consensus.cc:491] T 0d92161e8e3f4a9c88b0ca5fa5555a2c P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:48.927052 24810 tablet_bootstrap.cc:492] T 98383febcb2145208b05b96598aed748 P 1a9b5a4afd63410b813b41edbfe69e28: No bootstrap required, opened a new log
I20250109 08:58:48.930727 24822 raft_consensus.cc:513] T 0d92161e8e3f4a9c88b0ca5fa5555a2c P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:48.931272 24810 ts_tablet_manager.cc:1397] T 98383febcb2145208b05b96598aed748 P 1a9b5a4afd63410b813b41edbfe69e28: Time spent bootstrapping tablet: real 0.126s user 0.032s sys 0.035s
I20250109 08:58:48.931183 24694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e2136a82d344ae6b0657803f1551906" candidate_uuid: "08c70c9a6a064d40bfa68fd105d86d86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f366ff6c37a24eaaa6abcafab1467912" is_pre_election: true
I20250109 08:58:48.924939 24813 leader_election.cc:290] T 0d92161e8e3f4a9c88b0ca5fa5555a2c P f366ff6c37a24eaaa6abcafab1467912 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 08c70c9a6a064d40bfa68fd105d86d86 (127.19.226.193:43567), 1a9b5a4afd63410b813b41edbfe69e28 (127.19.226.195:42529)
I20250109 08:58:48.932705 24769 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e2136a82d344ae6b0657803f1551906" candidate_uuid: "08c70c9a6a064d40bfa68fd105d86d86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a9b5a4afd63410b813b41edbfe69e28" is_pre_election: true
I20250109 08:58:48.916102 24695 raft_consensus.cc:2463] T a930173474344b2c93cc8ac295a74729 P f366ff6c37a24eaaa6abcafab1467912 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 08c70c9a6a064d40bfa68fd105d86d86 in term 1.
I20250109 08:58:48.921974 24661 leader_election.cc:304] T 6e2f4cbbf3f744e586058426df0287c2 P f366ff6c37a24eaaa6abcafab1467912 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 08c70c9a6a064d40bfa68fd105d86d86, f366ff6c37a24eaaa6abcafab1467912; no voters: 1a9b5a4afd63410b813b41edbfe69e28
W20250109 08:58:48.934054 24586 leader_election.cc:343] T 4e2136a82d344ae6b0657803f1551906 P 08c70c9a6a064d40bfa68fd105d86d86 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1a9b5a4afd63410b813b41edbfe69e28 (127.19.226.195:42529): Illegal state: must be running to vote when last-logged opid is not known
W20250109 08:58:48.934881 24583 leader_election.cc:343] T 4e2136a82d344ae6b0657803f1551906 P 08c70c9a6a064d40bfa68fd105d86d86 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f366ff6c37a24eaaa6abcafab1467912 (127.19.226.194:43355): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:48.934412 24815 raft_consensus.cc:513] T a930173474344b2c93cc8ac295a74729 P 1a9b5a4afd63410b813b41edbfe69e28 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } }
I20250109 08:58:48.935113 24813 raft_consensus.cc:2798] T 6e2f4cbbf3f744e586058426df0287c2 P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:58:48.935343 24583 leader_election.cc:304] T 4e2136a82d344ae6b0657803f1551906 P 08c70c9a6a064d40bfa68fd105d86d86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 08c70c9a6a064d40bfa68fd105d86d86; no voters: 1a9b5a4afd63410b813b41edbfe69e28, f366ff6c37a24eaaa6abcafab1467912
I20250109 08:58:48.935587 24813 raft_consensus.cc:491] T 6e2f4cbbf3f744e586058426df0287c2 P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:58:48.935947 24813 raft_consensus.cc:3054] T 6e2f4cbbf3f744e586058426df0287c2 P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:48.936522 24583 leader_election.cc:304] T a930173474344b2c93cc8ac295a74729 P 08c70c9a6a064d40bfa68fd105d86d86 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08c70c9a6a064d40bfa68fd105d86d86, f366ff6c37a24eaaa6abcafab1467912; no voters:
I20250109 08:58:48.937058 24815 leader_election.cc:290] T a930173474344b2c93cc8ac295a74729 P 1a9b5a4afd63410b813b41edbfe69e28 [CANDIDATE]: Term 1 election: Requested vote from peers f366ff6c37a24eaaa6abcafab1467912 (127.19.226.194:43355), 08c70c9a6a064d40bfa68fd105d86d86 (127.19.226.193:43567)
I20250109 08:58:48.934343 24810 raft_consensus.cc:357] T 98383febcb2145208b05b96598aed748 P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:48.937994 24810 raft_consensus.cc:383] T 98383febcb2145208b05b96598aed748 P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:48.925060 24809 raft_consensus.cc:738] T 970b2f8b130d431cb25d79d33d5f0e5c P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f366ff6c37a24eaaa6abcafab1467912, State: Initialized, Role: FOLLOWER
I20250109 08:58:48.938293 24810 raft_consensus.cc:738] T 98383febcb2145208b05b96598aed748 P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a9b5a4afd63410b813b41edbfe69e28, State: Initialized, Role: FOLLOWER
I20250109 08:58:48.939162 24809 consensus_queue.cc:260] T 970b2f8b130d431cb25d79d33d5f0e5c P f366ff6c37a24eaaa6abcafab1467912 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:48.939704 24810 consensus_queue.cc:260] T 98383febcb2145208b05b96598aed748 P 1a9b5a4afd63410b813b41edbfe69e28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:48.942378 24619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d92161e8e3f4a9c88b0ca5fa5555a2c" candidate_uuid: "f366ff6c37a24eaaa6abcafab1467912" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08c70c9a6a064d40bfa68fd105d86d86" is_pre_election: true
I20250109 08:58:48.943185 24619 raft_consensus.cc:2463] T 0d92161e8e3f4a9c88b0ca5fa5555a2c P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f366ff6c37a24eaaa6abcafab1467912 in term 0.
I20250109 08:58:48.938200 24620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a930173474344b2c93cc8ac295a74729" candidate_uuid: "1a9b5a4afd63410b813b41edbfe69e28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08c70c9a6a064d40bfa68fd105d86d86"
I20250109 08:58:48.945663 24620 raft_consensus.cc:2388] T a930173474344b2c93cc8ac295a74729 P 08c70c9a6a064d40bfa68fd105d86d86 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 1a9b5a4afd63410b813b41edbfe69e28 in current term 1: Already voted for candidate 08c70c9a6a064d40bfa68fd105d86d86 in this term.
I20250109 08:58:48.945005 24816 ts_tablet_manager.cc:1428] T 98383febcb2145208b05b96598aed748 P 08c70c9a6a064d40bfa68fd105d86d86: Time spent starting tablet: real 0.021s user 0.007s sys 0.007s
I20250109 08:58:48.946966 24813 raft_consensus.cc:513] T 6e2f4cbbf3f744e586058426df0287c2 P f366ff6c37a24eaaa6abcafab1467912 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:48.947723 24816 tablet_bootstrap.cc:492] T d8f9f0e755da4a7186c2c9316918ca73 P 08c70c9a6a064d40bfa68fd105d86d86: Bootstrap starting.
I20250109 08:58:48.942396 24810 ts_tablet_manager.cc:1428] T 98383febcb2145208b05b96598aed748 P 1a9b5a4afd63410b813b41edbfe69e28: Time spent starting tablet: real 0.011s user 0.006s sys 0.000s
I20250109 08:58:48.948922 24768 raft_consensus.cc:2388] T a930173474344b2c93cc8ac295a74729 P 1a9b5a4afd63410b813b41edbfe69e28 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 08c70c9a6a064d40bfa68fd105d86d86 in current term 1: Already voted for candidate 1a9b5a4afd63410b813b41edbfe69e28 in this term.
I20250109 08:58:48.949708 24810 tablet_bootstrap.cc:492] T 6e2f4cbbf3f744e586058426df0287c2 P 1a9b5a4afd63410b813b41edbfe69e28: Bootstrap starting.
I20250109 08:58:48.952184 24841 raft_consensus.cc:2743] T 4e2136a82d344ae6b0657803f1551906 P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:58:48.952646 24695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a930173474344b2c93cc8ac295a74729" candidate_uuid: "1a9b5a4afd63410b813b41edbfe69e28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f366ff6c37a24eaaa6abcafab1467912"
I20250109 08:58:48.953462 24695 raft_consensus.cc:2388] T a930173474344b2c93cc8ac295a74729 P f366ff6c37a24eaaa6abcafab1467912 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 1a9b5a4afd63410b813b41edbfe69e28 in current term 1: Already voted for candidate 08c70c9a6a064d40bfa68fd105d86d86 in this term.
I20250109 08:58:48.954638 24732 leader_election.cc:304] T a930173474344b2c93cc8ac295a74729 P 1a9b5a4afd63410b813b41edbfe69e28 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1a9b5a4afd63410b813b41edbfe69e28; no voters: 08c70c9a6a064d40bfa68fd105d86d86, f366ff6c37a24eaaa6abcafab1467912
I20250109 08:58:48.956789 24816 tablet_bootstrap.cc:654] T d8f9f0e755da4a7186c2c9316918ca73 P 08c70c9a6a064d40bfa68fd105d86d86: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:48.958065 24769 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d92161e8e3f4a9c88b0ca5fa5555a2c" candidate_uuid: "08c70c9a6a064d40bfa68fd105d86d86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a9b5a4afd63410b813b41edbfe69e28" is_pre_election: true
I20250109 08:58:48.958767 24769 raft_consensus.cc:2463] T 0d92161e8e3f4a9c88b0ca5fa5555a2c P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 08c70c9a6a064d40bfa68fd105d86d86 in term 0.
I20250109 08:58:48.959839 24586 leader_election.cc:304] T 0d92161e8e3f4a9c88b0ca5fa5555a2c P 08c70c9a6a064d40bfa68fd105d86d86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08c70c9a6a064d40bfa68fd105d86d86, 1a9b5a4afd63410b813b41edbfe69e28; no voters:
I20250109 08:58:48.952869 24841 raft_consensus.cc:2798] T a930173474344b2c93cc8ac295a74729 P 08c70c9a6a064d40bfa68fd105d86d86 [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:58:48.953188 24661 leader_election.cc:304] T 0d92161e8e3f4a9c88b0ca5fa5555a2c P f366ff6c37a24eaaa6abcafab1467912 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08c70c9a6a064d40bfa68fd105d86d86, f366ff6c37a24eaaa6abcafab1467912; no voters:
I20250109 08:58:48.960785 24841 raft_consensus.cc:695] T a930173474344b2c93cc8ac295a74729 P 08c70c9a6a064d40bfa68fd105d86d86 [term 1 LEADER]: Becoming Leader. State: Replica: 08c70c9a6a064d40bfa68fd105d86d86, State: Running, Role: LEADER
I20250109 08:58:48.961303 24843 raft_consensus.cc:2798] T 0d92161e8e3f4a9c88b0ca5fa5555a2c P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:58:48.958215 24822 leader_election.cc:290] T 0d92161e8e3f4a9c88b0ca5fa5555a2c P 08c70c9a6a064d40bfa68fd105d86d86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f366ff6c37a24eaaa6abcafab1467912 (127.19.226.194:43355), 1a9b5a4afd63410b813b41edbfe69e28 (127.19.226.195:42529)
I20250109 08:58:48.961670 24843 raft_consensus.cc:491] T 0d92161e8e3f4a9c88b0ca5fa5555a2c P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:58:48.961994 24822 raft_consensus.cc:2798] T 0d92161e8e3f4a9c88b0ca5fa5555a2c P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:58:48.962144 24815 raft_consensus.cc:2743] T a930173474344b2c93cc8ac295a74729 P 1a9b5a4afd63410b813b41edbfe69e28 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20250109 08:58:48.961598 24841 consensus_queue.cc:237] T a930173474344b2c93cc8ac295a74729 P 08c70c9a6a064d40bfa68fd105d86d86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } }
I20250109 08:58:48.963306 24813 leader_election.cc:290] T 6e2f4cbbf3f744e586058426df0287c2 P f366ff6c37a24eaaa6abcafab1467912 [CANDIDATE]: Term 1 election: Requested vote from peers 08c70c9a6a064d40bfa68fd105d86d86 (127.19.226.193:43567), 1a9b5a4afd63410b813b41edbfe69e28 (127.19.226.195:42529)
I20250109 08:58:48.962113 24843 raft_consensus.cc:3054] T 0d92161e8e3f4a9c88b0ca5fa5555a2c P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:48.964103 24620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e2f4cbbf3f744e586058426df0287c2" candidate_uuid: "f366ff6c37a24eaaa6abcafab1467912" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08c70c9a6a064d40bfa68fd105d86d86"
I20250109 08:58:48.964808 24620 raft_consensus.cc:3054] T 6e2f4cbbf3f744e586058426df0287c2 P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:48.962395 24822 raft_consensus.cc:491] T 0d92161e8e3f4a9c88b0ca5fa5555a2c P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:58:48.966109 24822 raft_consensus.cc:3054] T 0d92161e8e3f4a9c88b0ca5fa5555a2c P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:48.964725 24809 ts_tablet_manager.cc:1428] T 970b2f8b130d431cb25d79d33d5f0e5c P f366ff6c37a24eaaa6abcafab1467912: Time spent starting tablet: real 0.044s user 0.006s sys 0.002s
I20250109 08:58:48.966943 24768 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d92161e8e3f4a9c88b0ca5fa5555a2c" candidate_uuid: "f366ff6c37a24eaaa6abcafab1467912" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a9b5a4afd63410b813b41edbfe69e28" is_pre_election: true
I20250109 08:58:48.967623 24768 raft_consensus.cc:2463] T 0d92161e8e3f4a9c88b0ca5fa5555a2c P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f366ff6c37a24eaaa6abcafab1467912 in term 0.
I20250109 08:58:48.969959 24810 tablet_bootstrap.cc:654] T 6e2f4cbbf3f744e586058426df0287c2 P 1a9b5a4afd63410b813b41edbfe69e28: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:48.972028 24769 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e2f4cbbf3f744e586058426df0287c2" candidate_uuid: "f366ff6c37a24eaaa6abcafab1467912" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a9b5a4afd63410b813b41edbfe69e28"
I20250109 08:58:48.974277 24809 tablet_bootstrap.cc:492] T 4e2136a82d344ae6b0657803f1551906 P f366ff6c37a24eaaa6abcafab1467912: Bootstrap starting.
W20250109 08:58:48.978260 24661 leader_election.cc:343] T 6e2f4cbbf3f744e586058426df0287c2 P f366ff6c37a24eaaa6abcafab1467912 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1a9b5a4afd63410b813b41edbfe69e28 (127.19.226.195:42529): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:48.979532 24695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d92161e8e3f4a9c88b0ca5fa5555a2c" candidate_uuid: "08c70c9a6a064d40bfa68fd105d86d86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f366ff6c37a24eaaa6abcafab1467912" is_pre_election: true
I20250109 08:58:48.982234 24822 raft_consensus.cc:513] T 0d92161e8e3f4a9c88b0ca5fa5555a2c P 08c70c9a6a064d40bfa68fd105d86d86 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:48.984326 24822 leader_election.cc:290] T 0d92161e8e3f4a9c88b0ca5fa5555a2c P 08c70c9a6a064d40bfa68fd105d86d86 [CANDIDATE]: Term 1 election: Requested vote from peers f366ff6c37a24eaaa6abcafab1467912 (127.19.226.194:43355), 1a9b5a4afd63410b813b41edbfe69e28 (127.19.226.195:42529)
I20250109 08:58:48.985001 24694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d92161e8e3f4a9c88b0ca5fa5555a2c" candidate_uuid: "08c70c9a6a064d40bfa68fd105d86d86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f366ff6c37a24eaaa6abcafab1467912"
I20250109 08:58:48.986650 24769 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d92161e8e3f4a9c88b0ca5fa5555a2c" candidate_uuid: "08c70c9a6a064d40bfa68fd105d86d86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a9b5a4afd63410b813b41edbfe69e28"
I20250109 08:58:48.987270 24769 raft_consensus.cc:3054] T 0d92161e8e3f4a9c88b0ca5fa5555a2c P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:48.987362 24620 raft_consensus.cc:2463] T 6e2f4cbbf3f744e586058426df0287c2 P 08c70c9a6a064d40bfa68fd105d86d86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f366ff6c37a24eaaa6abcafab1467912 in term 1.
I20250109 08:58:48.988641 24661 leader_election.cc:304] T 6e2f4cbbf3f744e586058426df0287c2 P f366ff6c37a24eaaa6abcafab1467912 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 08c70c9a6a064d40bfa68fd105d86d86, f366ff6c37a24eaaa6abcafab1467912; no voters: 1a9b5a4afd63410b813b41edbfe69e28
I20250109 08:58:48.988914 24809 tablet_bootstrap.cc:654] T 4e2136a82d344ae6b0657803f1551906 P f366ff6c37a24eaaa6abcafab1467912: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:48.989894 24813 raft_consensus.cc:2798] T 6e2f4cbbf3f744e586058426df0287c2 P f366ff6c37a24eaaa6abcafab1467912 [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:58:48.991261 24816 tablet_bootstrap.cc:492] T d8f9f0e755da4a7186c2c9316918ca73 P 08c70c9a6a064d40bfa68fd105d86d86: No bootstrap required, opened a new log
I20250109 08:58:48.991801 24816 ts_tablet_manager.cc:1397] T d8f9f0e755da4a7186c2c9316918ca73 P 08c70c9a6a064d40bfa68fd105d86d86: Time spent bootstrapping tablet: real 0.044s user 0.010s sys 0.004s
I20250109 08:58:48.994701 24769 raft_consensus.cc:2463] T 0d92161e8e3f4a9c88b0ca5fa5555a2c P 1a9b5a4afd63410b813b41edbfe69e28 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 08c70c9a6a064d40bfa68fd105d86d86 in term 1.
I20250109 08:58:48.989069 24518 catalog_manager.cc:5526] T a930173474344b2c93cc8ac295a74729 P 08c70c9a6a064d40bfa68fd105d86d86 reported cstate change: term changed from 0 to 1, leader changed from <none> to 08c70c9a6a064d40bfa68fd105d86d86 (127.19.226.193). New cstate: current_term: 1 leader_uuid: "08c70c9a6a064d40bfa68fd105d86d86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } health_report { overall_health: HEALTHY } } }
I20250109 08:58:48.995896 24586 leader_election.cc:304] T 0d92161e8e3f4a9c88b0ca5fa5555a2c P 08c70c9a6a064d40bfa68fd105d86d86 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 08c70c9a6a064d40bfa68fd105d86d86, 1a9b5a4afd63410b813b41edbfe69e28; no voters: f366ff6c37a24eaaa6abcafab1467912
I20250109 08:58:49.004320 24810 tablet_bootstrap.cc:492] T 6e2f4cbbf3f744e586058426df0287c2 P 1a9b5a4afd63410b813b41edbfe69e28: No bootstrap required, opened a new log
I20250109 08:58:49.005007 24810 ts_tablet_manager.cc:1397] T 6e2f4cbbf3f744e586058426df0287c2 P 1a9b5a4afd63410b813b41edbfe69e28: Time spent bootstrapping tablet: real 0.055s user 0.010s sys 0.001s
I20250109 08:58:49.004350 24822 raft_consensus.cc:2798] T 0d92161e8e3f4a9c88b0ca5fa5555a2c P 08c70c9a6a064d40bfa68fd105d86d86 [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:58:49.007201 24822 raft_consensus.cc:695] T 0d92161e8e3f4a9c88b0ca5fa5555a2c P 08c70c9a6a064d40bfa68fd105d86d86 [term 1 LEADER]: Becoming Leader. State: Replica: 08c70c9a6a064d40bfa68fd105d86d86, State: Running, Role: LEADER
I20250109 08:58:49.008139 24809 tablet_bootstrap.cc:492] T 4e2136a82d344ae6b0657803f1551906 P f366ff6c37a24eaaa6abcafab1467912: No bootstrap required, opened a new log
I20250109 08:58:49.008832 24809 ts_tablet_manager.cc:1397] T 4e2136a82d344ae6b0657803f1551906 P f366ff6c37a24eaaa6abcafab1467912: Time spent bootstrapping tablet: real 0.035s user 0.013s sys 0.004s
I20250109 08:58:49.008164 24822 consensus_queue.cc:237] T 0d92161e8e3f4a9c88b0ca5fa5555a2c P 08c70c9a6a064d40bfa68fd105d86d86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:49.008220 24849 raft_consensus.cc:491] T d8f9f0e755da4a7186c2c9316918ca73 P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:49.009038 24843 raft_consensus.cc:513] T 0d92161e8e3f4a9c88b0ca5fa5555a2c P f366ff6c37a24eaaa6abcafab1467912 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:49.009670 24849 raft_consensus.cc:513] T d8f9f0e755da4a7186c2c9316918ca73 P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:49.010294 24695 raft_consensus.cc:2388] T 0d92161e8e3f4a9c88b0ca5fa5555a2c P f366ff6c37a24eaaa6abcafab1467912 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 08c70c9a6a064d40bfa68fd105d86d86 in current term 1: Already voted for candidate f366ff6c37a24eaaa6abcafab1467912 in this term.
I20250109 08:58:49.011188 24843 leader_election.cc:290] T 0d92161e8e3f4a9c88b0ca5fa5555a2c P f366ff6c37a24eaaa6abcafab1467912 [CANDIDATE]: Term 1 election: Requested vote from peers 08c70c9a6a064d40bfa68fd105d86d86 (127.19.226.193:43567), 1a9b5a4afd63410b813b41edbfe69e28 (127.19.226.195:42529)
I20250109 08:58:49.012647 24769 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d92161e8e3f4a9c88b0ca5fa5555a2c" candidate_uuid: "f366ff6c37a24eaaa6abcafab1467912" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a9b5a4afd63410b813b41edbfe69e28"
I20250109 08:58:49.013588 24769 raft_consensus.cc:2388] T 0d92161e8e3f4a9c88b0ca5fa5555a2c P 1a9b5a4afd63410b813b41edbfe69e28 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate f366ff6c37a24eaaa6abcafab1467912 in current term 1: Already voted for candidate 08c70c9a6a064d40bfa68fd105d86d86 in this term.
I20250109 08:58:49.007778 24810 raft_consensus.cc:357] T 6e2f4cbbf3f744e586058426df0287c2 P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:49.014122 24810 raft_consensus.cc:383] T 6e2f4cbbf3f744e586058426df0287c2 P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:49.013870 24809 raft_consensus.cc:357] T 4e2136a82d344ae6b0657803f1551906 P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:49.014417 24810 raft_consensus.cc:738] T 6e2f4cbbf3f744e586058426df0287c2 P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a9b5a4afd63410b813b41edbfe69e28, State: Initialized, Role: FOLLOWER
W20250109 08:58:49.014469 24795 tablet.cc:2367] T 98383febcb2145208b05b96598aed748 P 1a9b5a4afd63410b813b41edbfe69e28: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20250109 08:58:49.012647 24620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d92161e8e3f4a9c88b0ca5fa5555a2c" candidate_uuid: "f366ff6c37a24eaaa6abcafab1467912" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08c70c9a6a064d40bfa68fd105d86d86"
I20250109 08:58:49.015359 24816 raft_consensus.cc:357] T d8f9f0e755da4a7186c2c9316918ca73 P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:49.016407 24809 raft_consensus.cc:383] T 4e2136a82d344ae6b0657803f1551906 P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:49.016213 24661 leader_election.cc:304] T 0d92161e8e3f4a9c88b0ca5fa5555a2c P f366ff6c37a24eaaa6abcafab1467912 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f366ff6c37a24eaaa6abcafab1467912; no voters: 08c70c9a6a064d40bfa68fd105d86d86, 1a9b5a4afd63410b813b41edbfe69e28
I20250109 08:58:49.016964 24810 consensus_queue.cc:260] T 6e2f4cbbf3f744e586058426df0287c2 P 1a9b5a4afd63410b813b41edbfe69e28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:49.017607 24813 raft_consensus.cc:695] T 6e2f4cbbf3f744e586058426df0287c2 P f366ff6c37a24eaaa6abcafab1467912 [term 1 LEADER]: Becoming Leader. State: Replica: f366ff6c37a24eaaa6abcafab1467912, State: Running, Role: LEADER
I20250109 08:58:49.018119 24847 raft_consensus.cc:2743] T 0d92161e8e3f4a9c88b0ca5fa5555a2c P f366ff6c37a24eaaa6abcafab1467912 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20250109 08:58:49.018625 24813 consensus_queue.cc:237] T 6e2f4cbbf3f744e586058426df0287c2 P f366ff6c37a24eaaa6abcafab1467912 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:49.016136 24816 raft_consensus.cc:383] T d8f9f0e755da4a7186c2c9316918ca73 P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:49.019984 24816 raft_consensus.cc:738] T d8f9f0e755da4a7186c2c9316918ca73 P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08c70c9a6a064d40bfa68fd105d86d86, State: Initialized, Role: FOLLOWER
I20250109 08:58:49.020675 24816 consensus_queue.cc:260] T d8f9f0e755da4a7186c2c9316918ca73 P 08c70c9a6a064d40bfa68fd105d86d86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:49.023025 24816 ts_tablet_manager.cc:1428] T d8f9f0e755da4a7186c2c9316918ca73 P 08c70c9a6a064d40bfa68fd105d86d86: Time spent starting tablet: real 0.031s user 0.005s sys 0.000s
I20250109 08:58:49.016741 24809 raft_consensus.cc:738] T 4e2136a82d344ae6b0657803f1551906 P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f366ff6c37a24eaaa6abcafab1467912, State: Initialized, Role: FOLLOWER
I20250109 08:58:49.024067 24620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8f9f0e755da4a7186c2c9316918ca73" candidate_uuid: "f366ff6c37a24eaaa6abcafab1467912" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08c70c9a6a064d40bfa68fd105d86d86" is_pre_election: true
I20250109 08:58:49.024924 24620 raft_consensus.cc:2463] T d8f9f0e755da4a7186c2c9316918ca73 P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f366ff6c37a24eaaa6abcafab1467912 in term 0.
I20250109 08:58:49.025074 24809 consensus_queue.cc:260] T 4e2136a82d344ae6b0657803f1551906 P f366ff6c37a24eaaa6abcafab1467912 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:49.026166 24661 leader_election.cc:304] T d8f9f0e755da4a7186c2c9316918ca73 P f366ff6c37a24eaaa6abcafab1467912 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08c70c9a6a064d40bfa68fd105d86d86, f366ff6c37a24eaaa6abcafab1467912; no voters:
I20250109 08:58:49.024020 24816 tablet_bootstrap.cc:492] T c1c586b8d3a64430bf1fe82b27fef388 P 08c70c9a6a064d40bfa68fd105d86d86: Bootstrap starting.
I20250109 08:58:49.027004 24843 raft_consensus.cc:2798] T d8f9f0e755da4a7186c2c9316918ca73 P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:58:49.028496 24769 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8f9f0e755da4a7186c2c9316918ca73" candidate_uuid: "f366ff6c37a24eaaa6abcafab1467912" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a9b5a4afd63410b813b41edbfe69e28" is_pre_election: true
I20250109 08:58:49.028992 24843 raft_consensus.cc:491] T d8f9f0e755da4a7186c2c9316918ca73 P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:58:49.029381 24843 raft_consensus.cc:3054] T d8f9f0e755da4a7186c2c9316918ca73 P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:49.027196 24809 ts_tablet_manager.cc:1428] T 4e2136a82d344ae6b0657803f1551906 P f366ff6c37a24eaaa6abcafab1467912: Time spent starting tablet: real 0.018s user 0.001s sys 0.003s
W20250109 08:58:49.030081 24661 leader_election.cc:343] T d8f9f0e755da4a7186c2c9316918ca73 P f366ff6c37a24eaaa6abcafab1467912 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1a9b5a4afd63410b813b41edbfe69e28 (127.19.226.195:42529): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:49.033329 24849 leader_election.cc:290] T d8f9f0e755da4a7186c2c9316918ca73 P f366ff6c37a24eaaa6abcafab1467912 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 08c70c9a6a064d40bfa68fd105d86d86 (127.19.226.193:43567), 1a9b5a4afd63410b813b41edbfe69e28 (127.19.226.195:42529)
I20250109 08:58:49.026757 24810 ts_tablet_manager.cc:1428] T 6e2f4cbbf3f744e586058426df0287c2 P 1a9b5a4afd63410b813b41edbfe69e28: Time spent starting tablet: real 0.021s user 0.007s sys 0.000s
I20250109 08:58:49.036487 24517 catalog_manager.cc:5526] T 6e2f4cbbf3f744e586058426df0287c2 P f366ff6c37a24eaaa6abcafab1467912 reported cstate change: term changed from 0 to 1, leader changed from <none> to f366ff6c37a24eaaa6abcafab1467912 (127.19.226.194). New cstate: current_term: 1 leader_uuid: "f366ff6c37a24eaaa6abcafab1467912" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:49.040474 24815 raft_consensus.cc:491] T 630fe12978194fb9ae9f60597b3ce1c1 P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:49.041014 24815 raft_consensus.cc:513] T 630fe12978194fb9ae9f60597b3ce1c1 P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:49.043004 24815 leader_election.cc:290] T 630fe12978194fb9ae9f60597b3ce1c1 P 1a9b5a4afd63410b813b41edbfe69e28 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 08c70c9a6a064d40bfa68fd105d86d86 (127.19.226.193:43567), f366ff6c37a24eaaa6abcafab1467912 (127.19.226.194:43355)
I20250109 08:58:49.043565 24810 tablet_bootstrap.cc:492] T 4e2136a82d344ae6b0657803f1551906 P 1a9b5a4afd63410b813b41edbfe69e28: Bootstrap starting.
I20250109 08:58:49.045075 24849 raft_consensus.cc:491] T 98383febcb2145208b05b96598aed748 P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:49.045485 24816 tablet_bootstrap.cc:654] T c1c586b8d3a64430bf1fe82b27fef388 P 08c70c9a6a064d40bfa68fd105d86d86: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:49.045560 24849 raft_consensus.cc:513] T 98383febcb2145208b05b96598aed748 P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:49.047659 24849 leader_election.cc:290] T 98383febcb2145208b05b96598aed748 P f366ff6c37a24eaaa6abcafab1467912 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 08c70c9a6a064d40bfa68fd105d86d86 (127.19.226.193:43567), 1a9b5a4afd63410b813b41edbfe69e28 (127.19.226.195:42529)
I20250109 08:58:49.048446 24695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "630fe12978194fb9ae9f60597b3ce1c1" candidate_uuid: "1a9b5a4afd63410b813b41edbfe69e28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f366ff6c37a24eaaa6abcafab1467912" is_pre_election: true
I20250109 08:58:49.049278 24695 raft_consensus.cc:2463] T 630fe12978194fb9ae9f60597b3ce1c1 P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a9b5a4afd63410b813b41edbfe69e28 in term 0.
I20250109 08:58:49.050674 24620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "630fe12978194fb9ae9f60597b3ce1c1" candidate_uuid: "1a9b5a4afd63410b813b41edbfe69e28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08c70c9a6a064d40bfa68fd105d86d86" is_pre_election: true
I20250109 08:58:49.050742 24843 raft_consensus.cc:513] T d8f9f0e755da4a7186c2c9316918ca73 P f366ff6c37a24eaaa6abcafab1467912 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:49.052611 24843 leader_election.cc:290] T d8f9f0e755da4a7186c2c9316918ca73 P f366ff6c37a24eaaa6abcafab1467912 [CANDIDATE]: Term 1 election: Requested vote from peers 08c70c9a6a064d40bfa68fd105d86d86 (127.19.226.193:43567), 1a9b5a4afd63410b813b41edbfe69e28 (127.19.226.195:42529)
I20250109 08:58:49.053126 24769 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98383febcb2145208b05b96598aed748" candidate_uuid: "f366ff6c37a24eaaa6abcafab1467912" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a9b5a4afd63410b813b41edbfe69e28" is_pre_election: true
I20250109 08:58:49.053844 24769 raft_consensus.cc:2463] T 98383febcb2145208b05b96598aed748 P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f366ff6c37a24eaaa6abcafab1467912 in term 0.
I20250109 08:58:49.054474 24810 tablet_bootstrap.cc:654] T 4e2136a82d344ae6b0657803f1551906 P 1a9b5a4afd63410b813b41edbfe69e28: Neither blocks nor log segments found. Creating new log.
W20250109 08:58:49.054924 24735 leader_election.cc:343] T 630fe12978194fb9ae9f60597b3ce1c1 P 1a9b5a4afd63410b813b41edbfe69e28 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 08c70c9a6a064d40bfa68fd105d86d86 (127.19.226.193:43567): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:49.055838 24620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98383febcb2145208b05b96598aed748" candidate_uuid: "f366ff6c37a24eaaa6abcafab1467912" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08c70c9a6a064d40bfa68fd105d86d86" is_pre_election: true
I20250109 08:58:49.056527 24620 raft_consensus.cc:2463] T 98383febcb2145208b05b96598aed748 P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f366ff6c37a24eaaa6abcafab1467912 in term 0.
I20250109 08:58:49.056704 24661 leader_election.cc:304] T 98383febcb2145208b05b96598aed748 P f366ff6c37a24eaaa6abcafab1467912 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a9b5a4afd63410b813b41edbfe69e28, f366ff6c37a24eaaa6abcafab1467912; no voters:
I20250109 08:58:49.057624 24843 raft_consensus.cc:2798] T 98383febcb2145208b05b96598aed748 P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:58:49.057756 24769 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8f9f0e755da4a7186c2c9316918ca73" candidate_uuid: "f366ff6c37a24eaaa6abcafab1467912" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a9b5a4afd63410b813b41edbfe69e28"
I20250109 08:58:49.058010 24843 raft_consensus.cc:491] T 98383febcb2145208b05b96598aed748 P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:58:49.058358 24843 raft_consensus.cc:3054] T 98383febcb2145208b05b96598aed748 P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Advancing to term 1
W20250109 08:58:49.059250 24661 leader_election.cc:343] T d8f9f0e755da4a7186c2c9316918ca73 P f366ff6c37a24eaaa6abcafab1467912 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1a9b5a4afd63410b813b41edbfe69e28 (127.19.226.195:42529): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:49.060133 24619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8f9f0e755da4a7186c2c9316918ca73" candidate_uuid: "f366ff6c37a24eaaa6abcafab1467912" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08c70c9a6a064d40bfa68fd105d86d86"
I20250109 08:58:49.060813 24619 raft_consensus.cc:3054] T d8f9f0e755da4a7186c2c9316918ca73 P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:49.063866 24843 raft_consensus.cc:513] T 98383febcb2145208b05b96598aed748 P f366ff6c37a24eaaa6abcafab1467912 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:49.055773 24732 leader_election.cc:304] T 630fe12978194fb9ae9f60597b3ce1c1 P 1a9b5a4afd63410b813b41edbfe69e28 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1a9b5a4afd63410b813b41edbfe69e28, f366ff6c37a24eaaa6abcafab1467912; no voters: 08c70c9a6a064d40bfa68fd105d86d86
I20250109 08:58:49.065496 24815 raft_consensus.cc:2798] T 630fe12978194fb9ae9f60597b3ce1c1 P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:58:49.066012 24815 raft_consensus.cc:491] T 630fe12978194fb9ae9f60597b3ce1c1 P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:58:49.066373 24815 raft_consensus.cc:3054] T 630fe12978194fb9ae9f60597b3ce1c1 P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:49.066543 24620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98383febcb2145208b05b96598aed748" candidate_uuid: "f366ff6c37a24eaaa6abcafab1467912" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08c70c9a6a064d40bfa68fd105d86d86"
I20250109 08:58:49.067134 24843 leader_election.cc:290] T 98383febcb2145208b05b96598aed748 P f366ff6c37a24eaaa6abcafab1467912 [CANDIDATE]: Term 1 election: Requested vote from peers 08c70c9a6a064d40bfa68fd105d86d86 (127.19.226.193:43567), 1a9b5a4afd63410b813b41edbfe69e28 (127.19.226.195:42529)
I20250109 08:58:49.066596 24810 tablet_bootstrap.cc:492] T 4e2136a82d344ae6b0657803f1551906 P 1a9b5a4afd63410b813b41edbfe69e28: No bootstrap required, opened a new log
I20250109 08:58:49.068183 24810 ts_tablet_manager.cc:1397] T 4e2136a82d344ae6b0657803f1551906 P 1a9b5a4afd63410b813b41edbfe69e28: Time spent bootstrapping tablet: real 0.025s user 0.010s sys 0.001s
I20250109 08:58:49.068329 24769 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98383febcb2145208b05b96598aed748" candidate_uuid: "f366ff6c37a24eaaa6abcafab1467912" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a9b5a4afd63410b813b41edbfe69e28"
I20250109 08:58:49.068943 24769 raft_consensus.cc:3054] T 98383febcb2145208b05b96598aed748 P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:49.070782 24619 raft_consensus.cc:2463] T d8f9f0e755da4a7186c2c9316918ca73 P 08c70c9a6a064d40bfa68fd105d86d86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f366ff6c37a24eaaa6abcafab1467912 in term 1.
I20250109 08:58:49.070914 24810 raft_consensus.cc:357] T 4e2136a82d344ae6b0657803f1551906 P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:49.071758 24810 raft_consensus.cc:383] T 4e2136a82d344ae6b0657803f1551906 P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:49.072105 24810 raft_consensus.cc:738] T 4e2136a82d344ae6b0657803f1551906 P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a9b5a4afd63410b813b41edbfe69e28, State: Initialized, Role: FOLLOWER
I20250109 08:58:49.067171 24620 raft_consensus.cc:3054] T 98383febcb2145208b05b96598aed748 P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:49.073020 24810 consensus_queue.cc:260] T 4e2136a82d344ae6b0657803f1551906 P 1a9b5a4afd63410b813b41edbfe69e28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:49.072029 24661 leader_election.cc:304] T d8f9f0e755da4a7186c2c9316918ca73 P f366ff6c37a24eaaa6abcafab1467912 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 08c70c9a6a064d40bfa68fd105d86d86, f366ff6c37a24eaaa6abcafab1467912; no voters: 1a9b5a4afd63410b813b41edbfe69e28
I20250109 08:58:49.075187 24810 ts_tablet_manager.cc:1428] T 4e2136a82d344ae6b0657803f1551906 P 1a9b5a4afd63410b813b41edbfe69e28: Time spent starting tablet: real 0.007s user 0.003s sys 0.003s
I20250109 08:58:49.075261 24843 raft_consensus.cc:2798] T d8f9f0e755da4a7186c2c9316918ca73 P f366ff6c37a24eaaa6abcafab1467912 [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:58:49.075788 24843 raft_consensus.cc:695] T d8f9f0e755da4a7186c2c9316918ca73 P f366ff6c37a24eaaa6abcafab1467912 [term 1 LEADER]: Becoming Leader. State: Replica: f366ff6c37a24eaaa6abcafab1467912, State: Running, Role: LEADER
I20250109 08:58:49.076283 24810 tablet_bootstrap.cc:492] T d8f9f0e755da4a7186c2c9316918ca73 P 1a9b5a4afd63410b813b41edbfe69e28: Bootstrap starting.
I20250109 08:58:49.076630 24843 consensus_queue.cc:237] T d8f9f0e755da4a7186c2c9316918ca73 P f366ff6c37a24eaaa6abcafab1467912 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:49.078634 24815 raft_consensus.cc:513] T 630fe12978194fb9ae9f60597b3ce1c1 P 1a9b5a4afd63410b813b41edbfe69e28 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:49.082250 24810 tablet_bootstrap.cc:654] T d8f9f0e755da4a7186c2c9316918ca73 P 1a9b5a4afd63410b813b41edbfe69e28: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:49.082329 24619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "630fe12978194fb9ae9f60597b3ce1c1" candidate_uuid: "1a9b5a4afd63410b813b41edbfe69e28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08c70c9a6a064d40bfa68fd105d86d86"
I20250109 08:58:49.056742 24816 tablet_bootstrap.cc:492] T c1c586b8d3a64430bf1fe82b27fef388 P 08c70c9a6a064d40bfa68fd105d86d86: No bootstrap required, opened a new log
I20250109 08:58:49.083977 24816 ts_tablet_manager.cc:1397] T c1c586b8d3a64430bf1fe82b27fef388 P 08c70c9a6a064d40bfa68fd105d86d86: Time spent bootstrapping tablet: real 0.060s user 0.015s sys 0.003s
I20250109 08:58:49.086827 24816 raft_consensus.cc:357] T c1c586b8d3a64430bf1fe82b27fef388 P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:49.087565 24816 raft_consensus.cc:383] T c1c586b8d3a64430bf1fe82b27fef388 P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:49.087843 24816 raft_consensus.cc:738] T c1c586b8d3a64430bf1fe82b27fef388 P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08c70c9a6a064d40bfa68fd105d86d86, State: Initialized, Role: FOLLOWER
I20250109 08:58:49.088426 24620 raft_consensus.cc:2463] T 98383febcb2145208b05b96598aed748 P 08c70c9a6a064d40bfa68fd105d86d86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f366ff6c37a24eaaa6abcafab1467912 in term 1.
I20250109 08:58:49.089658 24661 leader_election.cc:304] T 98383febcb2145208b05b96598aed748 P f366ff6c37a24eaaa6abcafab1467912 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08c70c9a6a064d40bfa68fd105d86d86, f366ff6c37a24eaaa6abcafab1467912; no voters:
I20250109 08:58:49.090504 24849 raft_consensus.cc:2798] T 98383febcb2145208b05b96598aed748 P f366ff6c37a24eaaa6abcafab1467912 [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:58:49.090975 24849 raft_consensus.cc:695] T 98383febcb2145208b05b96598aed748 P f366ff6c37a24eaaa6abcafab1467912 [term 1 LEADER]: Becoming Leader. State: Replica: f366ff6c37a24eaaa6abcafab1467912, State: Running, Role: LEADER
I20250109 08:58:49.082695 24769 raft_consensus.cc:2463] T 98383febcb2145208b05b96598aed748 P 1a9b5a4afd63410b813b41edbfe69e28 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f366ff6c37a24eaaa6abcafab1467912 in term 1.
I20250109 08:58:49.091804 24849 consensus_queue.cc:237] T 98383febcb2145208b05b96598aed748 P f366ff6c37a24eaaa6abcafab1467912 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
W20250109 08:58:49.095705 24735 leader_election.cc:343] T 630fe12978194fb9ae9f60597b3ce1c1 P 1a9b5a4afd63410b813b41edbfe69e28 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 08c70c9a6a064d40bfa68fd105d86d86 (127.19.226.193:43567): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:49.098186 24695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "630fe12978194fb9ae9f60597b3ce1c1" candidate_uuid: "1a9b5a4afd63410b813b41edbfe69e28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f366ff6c37a24eaaa6abcafab1467912"
I20250109 08:58:49.098872 24695 raft_consensus.cc:3054] T 630fe12978194fb9ae9f60597b3ce1c1 P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:49.105901 24838 raft_consensus.cc:491] T c1c586b8d3a64430bf1fe82b27fef388 P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:49.106482 24695 raft_consensus.cc:2463] T 630fe12978194fb9ae9f60597b3ce1c1 P f366ff6c37a24eaaa6abcafab1467912 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a9b5a4afd63410b813b41edbfe69e28 in term 1.
I20250109 08:58:49.104701 24518 catalog_manager.cc:5526] T 0d92161e8e3f4a9c88b0ca5fa5555a2c P 08c70c9a6a064d40bfa68fd105d86d86 reported cstate change: term changed from 0 to 1, leader changed from <none> to 08c70c9a6a064d40bfa68fd105d86d86 (127.19.226.193). New cstate: current_term: 1 leader_uuid: "08c70c9a6a064d40bfa68fd105d86d86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:49.106559 24838 raft_consensus.cc:513] T c1c586b8d3a64430bf1fe82b27fef388 P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:49.107761 24732 leader_election.cc:304] T 630fe12978194fb9ae9f60597b3ce1c1 P 1a9b5a4afd63410b813b41edbfe69e28 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1a9b5a4afd63410b813b41edbfe69e28, f366ff6c37a24eaaa6abcafab1467912; no voters: 08c70c9a6a064d40bfa68fd105d86d86
I20250109 08:58:49.109050 24838 leader_election.cc:290] T c1c586b8d3a64430bf1fe82b27fef388 P 1a9b5a4afd63410b813b41edbfe69e28 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 08c70c9a6a064d40bfa68fd105d86d86 (127.19.226.193:43567), f366ff6c37a24eaaa6abcafab1467912 (127.19.226.194:43355)
I20250109 08:58:49.109722 24838 raft_consensus.cc:2798] T 630fe12978194fb9ae9f60597b3ce1c1 P 1a9b5a4afd63410b813b41edbfe69e28 [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:58:49.110170 24838 raft_consensus.cc:695] T 630fe12978194fb9ae9f60597b3ce1c1 P 1a9b5a4afd63410b813b41edbfe69e28 [term 1 LEADER]: Becoming Leader. State: Replica: 1a9b5a4afd63410b813b41edbfe69e28, State: Running, Role: LEADER
I20250109 08:58:49.111074 24838 consensus_queue.cc:237] T 630fe12978194fb9ae9f60597b3ce1c1 P 1a9b5a4afd63410b813b41edbfe69e28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:49.116518 24694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1c586b8d3a64430bf1fe82b27fef388" candidate_uuid: "1a9b5a4afd63410b813b41edbfe69e28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f366ff6c37a24eaaa6abcafab1467912" is_pre_election: true
I20250109 08:58:49.116981 24619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1c586b8d3a64430bf1fe82b27fef388" candidate_uuid: "1a9b5a4afd63410b813b41edbfe69e28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08c70c9a6a064d40bfa68fd105d86d86" is_pre_election: true
I20250109 08:58:49.117302 24694 raft_consensus.cc:2463] T c1c586b8d3a64430bf1fe82b27fef388 P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a9b5a4afd63410b813b41edbfe69e28 in term 0.
I20250109 08:58:49.118634 24732 leader_election.cc:304] T c1c586b8d3a64430bf1fe82b27fef388 P 1a9b5a4afd63410b813b41edbfe69e28 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a9b5a4afd63410b813b41edbfe69e28, f366ff6c37a24eaaa6abcafab1467912; no voters:
I20250109 08:58:49.119433 24838 raft_consensus.cc:2798] T c1c586b8d3a64430bf1fe82b27fef388 P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:58:49.119864 24838 raft_consensus.cc:491] T c1c586b8d3a64430bf1fe82b27fef388 P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:58:49.120186 24838 raft_consensus.cc:3054] T c1c586b8d3a64430bf1fe82b27fef388 P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:49.088636 24816 consensus_queue.cc:260] T c1c586b8d3a64430bf1fe82b27fef388 P 08c70c9a6a064d40bfa68fd105d86d86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:49.121409 24815 leader_election.cc:290] T 630fe12978194fb9ae9f60597b3ce1c1 P 1a9b5a4afd63410b813b41edbfe69e28 [CANDIDATE]: Term 1 election: Requested vote from peers 08c70c9a6a064d40bfa68fd105d86d86 (127.19.226.193:43567), f366ff6c37a24eaaa6abcafab1467912 (127.19.226.194:43355)
I20250109 08:58:49.121603 24619 raft_consensus.cc:2463] T c1c586b8d3a64430bf1fe82b27fef388 P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a9b5a4afd63410b813b41edbfe69e28 in term 0.
I20250109 08:58:49.123921 24816 ts_tablet_manager.cc:1428] T c1c586b8d3a64430bf1fe82b27fef388 P 08c70c9a6a064d40bfa68fd105d86d86: Time spent starting tablet: real 0.040s user 0.003s sys 0.003s
I20250109 08:58:49.125095 24816 tablet_bootstrap.cc:492] T 970b2f8b130d431cb25d79d33d5f0e5c P 08c70c9a6a064d40bfa68fd105d86d86: Bootstrap starting.
I20250109 08:58:49.126082 24517 catalog_manager.cc:5526] T 630fe12978194fb9ae9f60597b3ce1c1 P 1a9b5a4afd63410b813b41edbfe69e28 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1a9b5a4afd63410b813b41edbfe69e28 (127.19.226.195). New cstate: current_term: 1 leader_uuid: "1a9b5a4afd63410b813b41edbfe69e28" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } health_report { overall_health: HEALTHY } } }
I20250109 08:58:49.127388 24838 raft_consensus.cc:513] T c1c586b8d3a64430bf1fe82b27fef388 P 1a9b5a4afd63410b813b41edbfe69e28 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:49.129532 24838 leader_election.cc:290] T c1c586b8d3a64430bf1fe82b27fef388 P 1a9b5a4afd63410b813b41edbfe69e28 [CANDIDATE]: Term 1 election: Requested vote from peers 08c70c9a6a064d40bfa68fd105d86d86 (127.19.226.193:43567), f366ff6c37a24eaaa6abcafab1467912 (127.19.226.194:43355)
I20250109 08:58:49.131139 24816 tablet_bootstrap.cc:654] T 970b2f8b130d431cb25d79d33d5f0e5c P 08c70c9a6a064d40bfa68fd105d86d86: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:49.131000 24695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1c586b8d3a64430bf1fe82b27fef388" candidate_uuid: "1a9b5a4afd63410b813b41edbfe69e28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f366ff6c37a24eaaa6abcafab1467912"
I20250109 08:58:49.131700 24695 raft_consensus.cc:3054] T c1c586b8d3a64430bf1fe82b27fef388 P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:49.133618 24619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1c586b8d3a64430bf1fe82b27fef388" candidate_uuid: "1a9b5a4afd63410b813b41edbfe69e28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08c70c9a6a064d40bfa68fd105d86d86"
I20250109 08:58:49.133224 24516 catalog_manager.cc:5526] T d8f9f0e755da4a7186c2c9316918ca73 P f366ff6c37a24eaaa6abcafab1467912 reported cstate change: term changed from 0 to 1, leader changed from <none> to f366ff6c37a24eaaa6abcafab1467912 (127.19.226.194). New cstate: current_term: 1 leader_uuid: "f366ff6c37a24eaaa6abcafab1467912" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:49.134238 24619 raft_consensus.cc:3054] T c1c586b8d3a64430bf1fe82b27fef388 P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:49.135980 24816 tablet_bootstrap.cc:492] T 970b2f8b130d431cb25d79d33d5f0e5c P 08c70c9a6a064d40bfa68fd105d86d86: No bootstrap required, opened a new log
I20250109 08:58:49.136548 24816 ts_tablet_manager.cc:1397] T 970b2f8b130d431cb25d79d33d5f0e5c P 08c70c9a6a064d40bfa68fd105d86d86: Time spent bootstrapping tablet: real 0.012s user 0.010s sys 0.000s
I20250109 08:58:49.139103 24816 raft_consensus.cc:357] T 970b2f8b130d431cb25d79d33d5f0e5c P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:49.139858 24816 raft_consensus.cc:383] T 970b2f8b130d431cb25d79d33d5f0e5c P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:49.140157 24816 raft_consensus.cc:738] T 970b2f8b130d431cb25d79d33d5f0e5c P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08c70c9a6a064d40bfa68fd105d86d86, State: Initialized, Role: FOLLOWER
I20250109 08:58:49.139671 24695 raft_consensus.cc:2463] T c1c586b8d3a64430bf1fe82b27fef388 P f366ff6c37a24eaaa6abcafab1467912 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a9b5a4afd63410b813b41edbfe69e28 in term 1.
I20250109 08:58:49.141050 24816 consensus_queue.cc:260] T 970b2f8b130d431cb25d79d33d5f0e5c P 08c70c9a6a064d40bfa68fd105d86d86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:49.141865 24732 leader_election.cc:304] T c1c586b8d3a64430bf1fe82b27fef388 P 1a9b5a4afd63410b813b41edbfe69e28 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a9b5a4afd63410b813b41edbfe69e28, f366ff6c37a24eaaa6abcafab1467912; no voters:
I20250109 08:58:49.142983 24838 raft_consensus.cc:2798] T c1c586b8d3a64430bf1fe82b27fef388 P 1a9b5a4afd63410b813b41edbfe69e28 [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:58:49.143476 24816 ts_tablet_manager.cc:1428] T 970b2f8b130d431cb25d79d33d5f0e5c P 08c70c9a6a064d40bfa68fd105d86d86: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20250109 08:58:49.143750 24810 tablet_bootstrap.cc:492] T d8f9f0e755da4a7186c2c9316918ca73 P 1a9b5a4afd63410b813b41edbfe69e28: No bootstrap required, opened a new log
I20250109 08:58:49.144212 24810 ts_tablet_manager.cc:1397] T d8f9f0e755da4a7186c2c9316918ca73 P 1a9b5a4afd63410b813b41edbfe69e28: Time spent bootstrapping tablet: real 0.068s user 0.019s sys 0.019s
I20250109 08:58:49.144647 24816 tablet_bootstrap.cc:492] T 630fe12978194fb9ae9f60597b3ce1c1 P 08c70c9a6a064d40bfa68fd105d86d86: Bootstrap starting.
I20250109 08:58:49.146697 24810 raft_consensus.cc:357] T d8f9f0e755da4a7186c2c9316918ca73 P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:49.147383 24810 raft_consensus.cc:383] T d8f9f0e755da4a7186c2c9316918ca73 P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:49.147650 24810 raft_consensus.cc:738] T d8f9f0e755da4a7186c2c9316918ca73 P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a9b5a4afd63410b813b41edbfe69e28, State: Initialized, Role: FOLLOWER
I20250109 08:58:49.148268 24810 consensus_queue.cc:260] T d8f9f0e755da4a7186c2c9316918ca73 P 1a9b5a4afd63410b813b41edbfe69e28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:49.150184 24816 tablet_bootstrap.cc:654] T 630fe12978194fb9ae9f60597b3ce1c1 P 08c70c9a6a064d40bfa68fd105d86d86: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:49.150269 24810 ts_tablet_manager.cc:1428] T d8f9f0e755da4a7186c2c9316918ca73 P 1a9b5a4afd63410b813b41edbfe69e28: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20250109 08:58:49.143430 24838 raft_consensus.cc:695] T c1c586b8d3a64430bf1fe82b27fef388 P 1a9b5a4afd63410b813b41edbfe69e28 [term 1 LEADER]: Becoming Leader. State: Replica: 1a9b5a4afd63410b813b41edbfe69e28, State: Running, Role: LEADER
I20250109 08:58:49.154208 24838 consensus_queue.cc:237] T c1c586b8d3a64430bf1fe82b27fef388 P 1a9b5a4afd63410b813b41edbfe69e28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:49.140926 24619 raft_consensus.cc:2463] T c1c586b8d3a64430bf1fe82b27fef388 P 08c70c9a6a064d40bfa68fd105d86d86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a9b5a4afd63410b813b41edbfe69e28 in term 1.
I20250109 08:58:49.160799 24517 catalog_manager.cc:5526] T 98383febcb2145208b05b96598aed748 P f366ff6c37a24eaaa6abcafab1467912 reported cstate change: term changed from 0 to 1, leader changed from <none> to f366ff6c37a24eaaa6abcafab1467912 (127.19.226.194). New cstate: current_term: 1 leader_uuid: "f366ff6c37a24eaaa6abcafab1467912" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:49.164054 24816 tablet_bootstrap.cc:492] T 630fe12978194fb9ae9f60597b3ce1c1 P 08c70c9a6a064d40bfa68fd105d86d86: No bootstrap required, opened a new log
I20250109 08:58:49.164631 24816 ts_tablet_manager.cc:1397] T 630fe12978194fb9ae9f60597b3ce1c1 P 08c70c9a6a064d40bfa68fd105d86d86: Time spent bootstrapping tablet: real 0.020s user 0.015s sys 0.003s
I20250109 08:58:49.166241 24516 catalog_manager.cc:5526] T c1c586b8d3a64430bf1fe82b27fef388 P 1a9b5a4afd63410b813b41edbfe69e28 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1a9b5a4afd63410b813b41edbfe69e28 (127.19.226.195). New cstate: current_term: 1 leader_uuid: "1a9b5a4afd63410b813b41edbfe69e28" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } health_report { overall_health: HEALTHY } } }
I20250109 08:58:49.166940 24816 raft_consensus.cc:357] T 630fe12978194fb9ae9f60597b3ce1c1 P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:49.167637 24816 raft_consensus.cc:383] T 630fe12978194fb9ae9f60597b3ce1c1 P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:49.167868 24816 raft_consensus.cc:738] T 630fe12978194fb9ae9f60597b3ce1c1 P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08c70c9a6a064d40bfa68fd105d86d86, State: Initialized, Role: FOLLOWER
I20250109 08:58:49.168434 24816 consensus_queue.cc:260] T 630fe12978194fb9ae9f60597b3ce1c1 P 08c70c9a6a064d40bfa68fd105d86d86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:49.170122 24816 ts_tablet_manager.cc:1428] T 630fe12978194fb9ae9f60597b3ce1c1 P 08c70c9a6a064d40bfa68fd105d86d86: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20250109 08:58:49.194552 24849 raft_consensus.cc:491] T 970b2f8b130d431cb25d79d33d5f0e5c P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:49.195123 24849 raft_consensus.cc:513] T 970b2f8b130d431cb25d79d33d5f0e5c P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:49.196709 24849 leader_election.cc:290] T 970b2f8b130d431cb25d79d33d5f0e5c P f366ff6c37a24eaaa6abcafab1467912 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 08c70c9a6a064d40bfa68fd105d86d86 (127.19.226.193:43567), 1a9b5a4afd63410b813b41edbfe69e28 (127.19.226.195:42529)
I20250109 08:58:49.197571 24619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "970b2f8b130d431cb25d79d33d5f0e5c" candidate_uuid: "f366ff6c37a24eaaa6abcafab1467912" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08c70c9a6a064d40bfa68fd105d86d86" is_pre_election: true
I20250109 08:58:49.197726 24769 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "970b2f8b130d431cb25d79d33d5f0e5c" candidate_uuid: "f366ff6c37a24eaaa6abcafab1467912" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a9b5a4afd63410b813b41edbfe69e28" is_pre_election: true
I20250109 08:58:49.198144 24619 raft_consensus.cc:2463] T 970b2f8b130d431cb25d79d33d5f0e5c P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f366ff6c37a24eaaa6abcafab1467912 in term 0.
I20250109 08:58:49.198359 24769 raft_consensus.cc:2463] T 970b2f8b130d431cb25d79d33d5f0e5c P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f366ff6c37a24eaaa6abcafab1467912 in term 0.
I20250109 08:58:49.199075 24661 leader_election.cc:304] T 970b2f8b130d431cb25d79d33d5f0e5c P f366ff6c37a24eaaa6abcafab1467912 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08c70c9a6a064d40bfa68fd105d86d86, f366ff6c37a24eaaa6abcafab1467912; no voters:
I20250109 08:58:49.199740 24849 raft_consensus.cc:2798] T 970b2f8b130d431cb25d79d33d5f0e5c P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:58:49.200083 24849 raft_consensus.cc:491] T 970b2f8b130d431cb25d79d33d5f0e5c P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:58:49.200405 24849 raft_consensus.cc:3054] T 970b2f8b130d431cb25d79d33d5f0e5c P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:49.205330 24849 raft_consensus.cc:513] T 970b2f8b130d431cb25d79d33d5f0e5c P f366ff6c37a24eaaa6abcafab1467912 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:49.207108 24849 leader_election.cc:290] T 970b2f8b130d431cb25d79d33d5f0e5c P f366ff6c37a24eaaa6abcafab1467912 [CANDIDATE]: Term 1 election: Requested vote from peers 08c70c9a6a064d40bfa68fd105d86d86 (127.19.226.193:43567), 1a9b5a4afd63410b813b41edbfe69e28 (127.19.226.195:42529)
I20250109 08:58:49.207875 24619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "970b2f8b130d431cb25d79d33d5f0e5c" candidate_uuid: "f366ff6c37a24eaaa6abcafab1467912" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08c70c9a6a064d40bfa68fd105d86d86"
I20250109 08:58:49.208149 24769 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "970b2f8b130d431cb25d79d33d5f0e5c" candidate_uuid: "f366ff6c37a24eaaa6abcafab1467912" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a9b5a4afd63410b813b41edbfe69e28"
I20250109 08:58:49.208480 24619 raft_consensus.cc:3054] T 970b2f8b130d431cb25d79d33d5f0e5c P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:49.208768 24769 raft_consensus.cc:3054] T 970b2f8b130d431cb25d79d33d5f0e5c P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:49.213047 24619 raft_consensus.cc:2463] T 970b2f8b130d431cb25d79d33d5f0e5c P 08c70c9a6a064d40bfa68fd105d86d86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f366ff6c37a24eaaa6abcafab1467912 in term 1.
I20250109 08:58:49.213114 24769 raft_consensus.cc:2463] T 970b2f8b130d431cb25d79d33d5f0e5c P 1a9b5a4afd63410b813b41edbfe69e28 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f366ff6c37a24eaaa6abcafab1467912 in term 1.
I20250109 08:58:49.214074 24661 leader_election.cc:304] T 970b2f8b130d431cb25d79d33d5f0e5c P f366ff6c37a24eaaa6abcafab1467912 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08c70c9a6a064d40bfa68fd105d86d86, f366ff6c37a24eaaa6abcafab1467912; no voters:
I20250109 08:58:49.214699 24849 raft_consensus.cc:2798] T 970b2f8b130d431cb25d79d33d5f0e5c P f366ff6c37a24eaaa6abcafab1467912 [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:58:49.215054 24849 raft_consensus.cc:695] T 970b2f8b130d431cb25d79d33d5f0e5c P f366ff6c37a24eaaa6abcafab1467912 [term 1 LEADER]: Becoming Leader. State: Replica: f366ff6c37a24eaaa6abcafab1467912, State: Running, Role: LEADER
I20250109 08:58:49.215708 24849 consensus_queue.cc:237] T 970b2f8b130d431cb25d79d33d5f0e5c P f366ff6c37a24eaaa6abcafab1467912 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:49.222857 24517 catalog_manager.cc:5526] T 970b2f8b130d431cb25d79d33d5f0e5c P f366ff6c37a24eaaa6abcafab1467912 reported cstate change: term changed from 0 to 1, leader changed from <none> to f366ff6c37a24eaaa6abcafab1467912 (127.19.226.194). New cstate: current_term: 1 leader_uuid: "f366ff6c37a24eaaa6abcafab1467912" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:49.269392 24838 raft_consensus.cc:491] T 4e2136a82d344ae6b0657803f1551906 P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:49.269922 24838 raft_consensus.cc:513] T 4e2136a82d344ae6b0657803f1551906 P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:49.271616 24838 leader_election.cc:290] T 4e2136a82d344ae6b0657803f1551906 P 1a9b5a4afd63410b813b41edbfe69e28 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 08c70c9a6a064d40bfa68fd105d86d86 (127.19.226.193:43567), f366ff6c37a24eaaa6abcafab1467912 (127.19.226.194:43355)
I20250109 08:58:49.272476 24619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e2136a82d344ae6b0657803f1551906" candidate_uuid: "1a9b5a4afd63410b813b41edbfe69e28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08c70c9a6a064d40bfa68fd105d86d86" is_pre_election: true
I20250109 08:58:49.272543 24695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e2136a82d344ae6b0657803f1551906" candidate_uuid: "1a9b5a4afd63410b813b41edbfe69e28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f366ff6c37a24eaaa6abcafab1467912" is_pre_election: true
I20250109 08:58:49.273087 24619 raft_consensus.cc:2463] T 4e2136a82d344ae6b0657803f1551906 P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a9b5a4afd63410b813b41edbfe69e28 in term 0.
I20250109 08:58:49.273162 24695 raft_consensus.cc:2463] T 4e2136a82d344ae6b0657803f1551906 P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a9b5a4afd63410b813b41edbfe69e28 in term 0.
I20250109 08:58:49.274037 24735 leader_election.cc:304] T 4e2136a82d344ae6b0657803f1551906 P 1a9b5a4afd63410b813b41edbfe69e28 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08c70c9a6a064d40bfa68fd105d86d86, 1a9b5a4afd63410b813b41edbfe69e28; no voters:
I20250109 08:58:49.274677 24838 raft_consensus.cc:2798] T 4e2136a82d344ae6b0657803f1551906 P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:58:49.275013 24838 raft_consensus.cc:491] T 4e2136a82d344ae6b0657803f1551906 P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:58:49.275260 24838 raft_consensus.cc:3054] T 4e2136a82d344ae6b0657803f1551906 P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:49.279937 24838 raft_consensus.cc:513] T 4e2136a82d344ae6b0657803f1551906 P 1a9b5a4afd63410b813b41edbfe69e28 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:49.281495 24838 leader_election.cc:290] T 4e2136a82d344ae6b0657803f1551906 P 1a9b5a4afd63410b813b41edbfe69e28 [CANDIDATE]: Term 1 election: Requested vote from peers 08c70c9a6a064d40bfa68fd105d86d86 (127.19.226.193:43567), f366ff6c37a24eaaa6abcafab1467912 (127.19.226.194:43355)
I20250109 08:58:49.282208 24619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e2136a82d344ae6b0657803f1551906" candidate_uuid: "1a9b5a4afd63410b813b41edbfe69e28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08c70c9a6a064d40bfa68fd105d86d86"
I20250109 08:58:49.282402 24695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e2136a82d344ae6b0657803f1551906" candidate_uuid: "1a9b5a4afd63410b813b41edbfe69e28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f366ff6c37a24eaaa6abcafab1467912"
I20250109 08:58:49.282760 24619 raft_consensus.cc:3054] T 4e2136a82d344ae6b0657803f1551906 P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:49.282934 24695 raft_consensus.cc:3054] T 4e2136a82d344ae6b0657803f1551906 P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:49.287650 24619 raft_consensus.cc:2463] T 4e2136a82d344ae6b0657803f1551906 P 08c70c9a6a064d40bfa68fd105d86d86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a9b5a4afd63410b813b41edbfe69e28 in term 1.
I20250109 08:58:49.287649 24695 raft_consensus.cc:2463] T 4e2136a82d344ae6b0657803f1551906 P f366ff6c37a24eaaa6abcafab1467912 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a9b5a4afd63410b813b41edbfe69e28 in term 1.
I20250109 08:58:49.288638 24735 leader_election.cc:304] T 4e2136a82d344ae6b0657803f1551906 P 1a9b5a4afd63410b813b41edbfe69e28 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08c70c9a6a064d40bfa68fd105d86d86, 1a9b5a4afd63410b813b41edbfe69e28; no voters:
I20250109 08:58:49.289258 24838 raft_consensus.cc:2798] T 4e2136a82d344ae6b0657803f1551906 P 1a9b5a4afd63410b813b41edbfe69e28 [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:58:49.289670 24838 raft_consensus.cc:695] T 4e2136a82d344ae6b0657803f1551906 P 1a9b5a4afd63410b813b41edbfe69e28 [term 1 LEADER]: Becoming Leader. State: Replica: 1a9b5a4afd63410b813b41edbfe69e28, State: Running, Role: LEADER
I20250109 08:58:49.290396 24838 consensus_queue.cc:237] T 4e2136a82d344ae6b0657803f1551906 P 1a9b5a4afd63410b813b41edbfe69e28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:49.297070 24517 catalog_manager.cc:5526] T 4e2136a82d344ae6b0657803f1551906 P 1a9b5a4afd63410b813b41edbfe69e28 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1a9b5a4afd63410b813b41edbfe69e28 (127.19.226.195). New cstate: current_term: 1 leader_uuid: "1a9b5a4afd63410b813b41edbfe69e28" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } health_report { overall_health: HEALTHY } } }
I20250109 08:58:49.308933 24838 raft_consensus.cc:491] T 6e2f4cbbf3f744e586058426df0287c2 P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:49.309410 24838 raft_consensus.cc:513] T 6e2f4cbbf3f744e586058426df0287c2 P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:49.310922 24838 leader_election.cc:290] T 6e2f4cbbf3f744e586058426df0287c2 P 1a9b5a4afd63410b813b41edbfe69e28 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 08c70c9a6a064d40bfa68fd105d86d86 (127.19.226.193:43567), f366ff6c37a24eaaa6abcafab1467912 (127.19.226.194:43355)
I20250109 08:58:49.311815 24619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e2f4cbbf3f744e586058426df0287c2" candidate_uuid: "1a9b5a4afd63410b813b41edbfe69e28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08c70c9a6a064d40bfa68fd105d86d86" is_pre_election: true
I20250109 08:58:49.311970 24695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e2f4cbbf3f744e586058426df0287c2" candidate_uuid: "1a9b5a4afd63410b813b41edbfe69e28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f366ff6c37a24eaaa6abcafab1467912" is_pre_election: true
I20250109 08:58:49.312582 24619 raft_consensus.cc:2388] T 6e2f4cbbf3f744e586058426df0287c2 P 08c70c9a6a064d40bfa68fd105d86d86 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1a9b5a4afd63410b813b41edbfe69e28 in current term 1: Already voted for candidate f366ff6c37a24eaaa6abcafab1467912 in this term.
I20250109 08:58:49.313400 24735 leader_election.cc:304] T 6e2f4cbbf3f744e586058426df0287c2 P 1a9b5a4afd63410b813b41edbfe69e28 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1a9b5a4afd63410b813b41edbfe69e28; no voters: 08c70c9a6a064d40bfa68fd105d86d86, f366ff6c37a24eaaa6abcafab1467912
I20250109 08:58:49.314024 24838 raft_consensus.cc:3054] T 6e2f4cbbf3f744e586058426df0287c2 P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:49.318753 24838 raft_consensus.cc:2743] T 6e2f4cbbf3f744e586058426df0287c2 P 1a9b5a4afd63410b813b41edbfe69e28 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:58:49.335736 24822 raft_consensus.cc:491] T 630fe12978194fb9ae9f60597b3ce1c1 P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:49.336493 24822 raft_consensus.cc:513] T 630fe12978194fb9ae9f60597b3ce1c1 P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:49.338119 24822 leader_election.cc:290] T 630fe12978194fb9ae9f60597b3ce1c1 P 08c70c9a6a064d40bfa68fd105d86d86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f366ff6c37a24eaaa6abcafab1467912 (127.19.226.194:43355), 1a9b5a4afd63410b813b41edbfe69e28 (127.19.226.195:42529)
I20250109 08:58:49.339033 24695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "630fe12978194fb9ae9f60597b3ce1c1" candidate_uuid: "08c70c9a6a064d40bfa68fd105d86d86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f366ff6c37a24eaaa6abcafab1467912" is_pre_election: true
I20250109 08:58:49.339079 24769 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "630fe12978194fb9ae9f60597b3ce1c1" candidate_uuid: "08c70c9a6a064d40bfa68fd105d86d86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a9b5a4afd63410b813b41edbfe69e28" is_pre_election: true
I20250109 08:58:49.339743 24695 raft_consensus.cc:2388] T 630fe12978194fb9ae9f60597b3ce1c1 P f366ff6c37a24eaaa6abcafab1467912 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 08c70c9a6a064d40bfa68fd105d86d86 in current term 1: Already voted for candidate 1a9b5a4afd63410b813b41edbfe69e28 in this term.
I20250109 08:58:49.340653 24583 leader_election.cc:304] T 630fe12978194fb9ae9f60597b3ce1c1 P 08c70c9a6a064d40bfa68fd105d86d86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 08c70c9a6a064d40bfa68fd105d86d86; no voters: 1a9b5a4afd63410b813b41edbfe69e28, f366ff6c37a24eaaa6abcafab1467912
I20250109 08:58:49.341272 24822 raft_consensus.cc:3054] T 630fe12978194fb9ae9f60597b3ce1c1 P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:49.345939 24822 raft_consensus.cc:2743] T 630fe12978194fb9ae9f60597b3ce1c1 P 08c70c9a6a064d40bfa68fd105d86d86 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:58:49.413599 24838 raft_consensus.cc:491] T d8f9f0e755da4a7186c2c9316918ca73 P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:49.415443 24838 raft_consensus.cc:513] T d8f9f0e755da4a7186c2c9316918ca73 P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:49.419392 24838 leader_election.cc:290] T d8f9f0e755da4a7186c2c9316918ca73 P 1a9b5a4afd63410b813b41edbfe69e28 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 08c70c9a6a064d40bfa68fd105d86d86 (127.19.226.193:43567), f366ff6c37a24eaaa6abcafab1467912 (127.19.226.194:43355)
I20250109 08:58:49.420684 24619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8f9f0e755da4a7186c2c9316918ca73" candidate_uuid: "1a9b5a4afd63410b813b41edbfe69e28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08c70c9a6a064d40bfa68fd105d86d86" is_pre_election: true
I20250109 08:58:49.421186 24695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8f9f0e755da4a7186c2c9316918ca73" candidate_uuid: "1a9b5a4afd63410b813b41edbfe69e28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f366ff6c37a24eaaa6abcafab1467912" is_pre_election: true
I20250109 08:58:49.422495 24619 raft_consensus.cc:2388] T d8f9f0e755da4a7186c2c9316918ca73 P 08c70c9a6a064d40bfa68fd105d86d86 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1a9b5a4afd63410b813b41edbfe69e28 in current term 1: Already voted for candidate f366ff6c37a24eaaa6abcafab1467912 in this term.
I20250109 08:58:49.424540 24735 leader_election.cc:304] T d8f9f0e755da4a7186c2c9316918ca73 P 1a9b5a4afd63410b813b41edbfe69e28 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1a9b5a4afd63410b813b41edbfe69e28; no voters: 08c70c9a6a064d40bfa68fd105d86d86, f366ff6c37a24eaaa6abcafab1467912
I20250109 08:58:49.425484 24838 raft_consensus.cc:3054] T d8f9f0e755da4a7186c2c9316918ca73 P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:49.426448 24822 consensus_queue.cc:1035] T 0d92161e8e3f4a9c88b0ca5fa5555a2c P 08c70c9a6a064d40bfa68fd105d86d86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250109 08:58:49.440949 24838 raft_consensus.cc:2743] T d8f9f0e755da4a7186c2c9316918ca73 P 1a9b5a4afd63410b813b41edbfe69e28 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:58:49.442652 24822 consensus_queue.cc:1035] T 0d92161e8e3f4a9c88b0ca5fa5555a2c P 08c70c9a6a064d40bfa68fd105d86d86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250109 08:58:49.464263 24694 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "970b2f8b130d431cb25d79d33d5f0e5c"
dest_uuid: "f366ff6c37a24eaaa6abcafab1467912"
mode: GRACEFUL
new_leader_uuid: "08c70c9a6a064d40bfa68fd105d86d86"
from {username='slave'} at 127.0.0.1:39922
I20250109 08:58:49.465170 24694 raft_consensus.cc:604] T 970b2f8b130d431cb25d79d33d5f0e5c P f366ff6c37a24eaaa6abcafab1467912 [term 1 LEADER]: Received request to transfer leadership to TS 08c70c9a6a064d40bfa68fd105d86d86
I20250109 08:58:49.472770 24849 consensus_queue.cc:1035] T 6e2f4cbbf3f744e586058426df0287c2 P f366ff6c37a24eaaa6abcafab1467912 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250109 08:58:49.499830 24849 consensus_queue.cc:1035] T 6e2f4cbbf3f744e586058426df0287c2 P f366ff6c37a24eaaa6abcafab1467912 [LEADER]: Connected to new peer: Peer: permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:58:49.506624 24769 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "c1c586b8d3a64430bf1fe82b27fef388"
dest_uuid: "1a9b5a4afd63410b813b41edbfe69e28"
mode: GRACEFUL
new_leader_uuid: "08c70c9a6a064d40bfa68fd105d86d86"
from {username='slave'} at 127.0.0.1:37366
I20250109 08:58:49.507373 24769 raft_consensus.cc:604] T c1c586b8d3a64430bf1fe82b27fef388 P 1a9b5a4afd63410b813b41edbfe69e28 [term 1 LEADER]: Received request to transfer leadership to TS 08c70c9a6a064d40bfa68fd105d86d86
I20250109 08:58:49.508893 24843 consensus_queue.cc:1035] T 98383febcb2145208b05b96598aed748 P f366ff6c37a24eaaa6abcafab1467912 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250109 08:58:49.516661 24838 consensus_queue.cc:1035] T 630fe12978194fb9ae9f60597b3ce1c1 P 1a9b5a4afd63410b813b41edbfe69e28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:58:49.521448 24822 consensus_queue.cc:1035] T a930173474344b2c93cc8ac295a74729 P 08c70c9a6a064d40bfa68fd105d86d86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:58:49.532294 24767 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "630fe12978194fb9ae9f60597b3ce1c1"
dest_uuid: "1a9b5a4afd63410b813b41edbfe69e28"
mode: GRACEFUL
new_leader_uuid: "f366ff6c37a24eaaa6abcafab1467912"
from {username='slave'} at 127.0.0.1:37366
I20250109 08:58:49.532997 24767 raft_consensus.cc:604] T 630fe12978194fb9ae9f60597b3ce1c1 P 1a9b5a4afd63410b813b41edbfe69e28 [term 1 LEADER]: Received request to transfer leadership to TS f366ff6c37a24eaaa6abcafab1467912
I20250109 08:58:49.540285 24842 consensus_queue.cc:1035] T a930173474344b2c93cc8ac295a74729 P 08c70c9a6a064d40bfa68fd105d86d86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250109 08:58:49.547114 24849 consensus_queue.cc:1035] T 98383febcb2145208b05b96598aed748 P f366ff6c37a24eaaa6abcafab1467912 [LEADER]: Connected to new peer: Peer: permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:58:49.552279 24815 consensus_queue.cc:1035] T 630fe12978194fb9ae9f60597b3ce1c1 P 1a9b5a4afd63410b813b41edbfe69e28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20250109 08:58:49.575357 24861 consensus_queue.cc:1035] T c1c586b8d3a64430bf1fe82b27fef388 P 1a9b5a4afd63410b813b41edbfe69e28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250109 08:58:49.589697 24815 raft_consensus.cc:988] T 630fe12978194fb9ae9f60597b3ce1c1 P 1a9b5a4afd63410b813b41edbfe69e28: : Instructing follower f366ff6c37a24eaaa6abcafab1467912 to start an election
I20250109 08:58:49.590308 24815 raft_consensus.cc:1076] T 630fe12978194fb9ae9f60597b3ce1c1 P 1a9b5a4afd63410b813b41edbfe69e28 [term 1 LEADER]: Signalling peer f366ff6c37a24eaaa6abcafab1467912 to start an election
I20250109 08:58:49.588795 24849 consensus_queue.cc:1035] T d8f9f0e755da4a7186c2c9316918ca73 P f366ff6c37a24eaaa6abcafab1467912 [LEADER]: Connected to new peer: Peer: permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20250109 08:58:49.594743 24695 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "630fe12978194fb9ae9f60597b3ce1c1"
dest_uuid: "f366ff6c37a24eaaa6abcafab1467912"
from {username='slave'} at 127.0.0.1:48098
I20250109 08:58:49.595499 24695 raft_consensus.cc:491] T 630fe12978194fb9ae9f60597b3ce1c1 P f366ff6c37a24eaaa6abcafab1467912 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:58:49.595952 24695 raft_consensus.cc:3054] T 630fe12978194fb9ae9f60597b3ce1c1 P f366ff6c37a24eaaa6abcafab1467912 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:49.606452 24861 raft_consensus.cc:988] T c1c586b8d3a64430bf1fe82b27fef388 P 1a9b5a4afd63410b813b41edbfe69e28: : Instructing follower 08c70c9a6a064d40bfa68fd105d86d86 to start an election
I20250109 08:58:49.607038 24861 raft_consensus.cc:1076] T c1c586b8d3a64430bf1fe82b27fef388 P 1a9b5a4afd63410b813b41edbfe69e28 [term 1 LEADER]: Signalling peer 08c70c9a6a064d40bfa68fd105d86d86 to start an election
I20250109 08:58:49.608421 24695 raft_consensus.cc:513] T 630fe12978194fb9ae9f60597b3ce1c1 P f366ff6c37a24eaaa6abcafab1467912 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:49.612175 24769 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "630fe12978194fb9ae9f60597b3ce1c1" candidate_uuid: "f366ff6c37a24eaaa6abcafab1467912" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1a9b5a4afd63410b813b41edbfe69e28"
I20250109 08:58:49.612977 24769 raft_consensus.cc:3049] T 630fe12978194fb9ae9f60597b3ce1c1 P 1a9b5a4afd63410b813b41edbfe69e28 [term 1 LEADER]: Stepping down as leader of term 1
I20250109 08:58:49.613301 24769 raft_consensus.cc:738] T 630fe12978194fb9ae9f60597b3ce1c1 P 1a9b5a4afd63410b813b41edbfe69e28 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1a9b5a4afd63410b813b41edbfe69e28, State: Running, Role: LEADER
I20250109 08:58:49.614120 24617 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "c1c586b8d3a64430bf1fe82b27fef388"
dest_uuid: "08c70c9a6a064d40bfa68fd105d86d86"
from {username='slave'} at 127.0.0.1:37628
I20250109 08:58:49.614235 24769 consensus_queue.cc:260] T 630fe12978194fb9ae9f60597b3ce1c1 P 1a9b5a4afd63410b813b41edbfe69e28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:49.614892 24617 raft_consensus.cc:491] T c1c586b8d3a64430bf1fe82b27fef388 P 08c70c9a6a064d40bfa68fd105d86d86 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:58:49.615343 24695 leader_election.cc:290] T 630fe12978194fb9ae9f60597b3ce1c1 P f366ff6c37a24eaaa6abcafab1467912 [CANDIDATE]: Term 2 election: Requested vote from peers 08c70c9a6a064d40bfa68fd105d86d86 (127.19.226.193:43567), 1a9b5a4afd63410b813b41edbfe69e28 (127.19.226.195:42529)
I20250109 08:58:49.615654 24769 raft_consensus.cc:3054] T 630fe12978194fb9ae9f60597b3ce1c1 P 1a9b5a4afd63410b813b41edbfe69e28 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:49.615950 24616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "630fe12978194fb9ae9f60597b3ce1c1" candidate_uuid: "f366ff6c37a24eaaa6abcafab1467912" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "08c70c9a6a064d40bfa68fd105d86d86"
I20250109 08:58:49.616638 24616 raft_consensus.cc:3054] T 630fe12978194fb9ae9f60597b3ce1c1 P 08c70c9a6a064d40bfa68fd105d86d86 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:49.615324 24617 raft_consensus.cc:3054] T c1c586b8d3a64430bf1fe82b27fef388 P 08c70c9a6a064d40bfa68fd105d86d86 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:49.622403 24769 raft_consensus.cc:2463] T 630fe12978194fb9ae9f60597b3ce1c1 P 1a9b5a4afd63410b813b41edbfe69e28 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f366ff6c37a24eaaa6abcafab1467912 in term 2.
I20250109 08:58:49.625253 24661 leader_election.cc:304] T 630fe12978194fb9ae9f60597b3ce1c1 P f366ff6c37a24eaaa6abcafab1467912 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a9b5a4afd63410b813b41edbfe69e28, f366ff6c37a24eaaa6abcafab1467912; no voters:
I20250109 08:58:49.626329 24849 raft_consensus.cc:2798] T 630fe12978194fb9ae9f60597b3ce1c1 P f366ff6c37a24eaaa6abcafab1467912 [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:58:49.626824 24849 raft_consensus.cc:695] T 630fe12978194fb9ae9f60597b3ce1c1 P f366ff6c37a24eaaa6abcafab1467912 [term 2 LEADER]: Becoming Leader. State: Replica: f366ff6c37a24eaaa6abcafab1467912, State: Running, Role: LEADER
I20250109 08:58:49.627897 24849 consensus_queue.cc:237] T 630fe12978194fb9ae9f60597b3ce1c1 P f366ff6c37a24eaaa6abcafab1467912 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:49.630517 24617 raft_consensus.cc:513] T c1c586b8d3a64430bf1fe82b27fef388 P 08c70c9a6a064d40bfa68fd105d86d86 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:49.633633 24617 leader_election.cc:290] T c1c586b8d3a64430bf1fe82b27fef388 P 08c70c9a6a064d40bfa68fd105d86d86 [CANDIDATE]: Term 2 election: Requested vote from peers f366ff6c37a24eaaa6abcafab1467912 (127.19.226.194:43355), 1a9b5a4afd63410b813b41edbfe69e28 (127.19.226.195:42529)
I20250109 08:58:49.634689 24694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1c586b8d3a64430bf1fe82b27fef388" candidate_uuid: "08c70c9a6a064d40bfa68fd105d86d86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f366ff6c37a24eaaa6abcafab1467912"
I20250109 08:58:49.635471 24694 raft_consensus.cc:3054] T c1c586b8d3a64430bf1fe82b27fef388 P f366ff6c37a24eaaa6abcafab1467912 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:49.637802 24516 catalog_manager.cc:5526] T 630fe12978194fb9ae9f60597b3ce1c1 P f366ff6c37a24eaaa6abcafab1467912 reported cstate change: term changed from 1 to 2, leader changed from 1a9b5a4afd63410b813b41edbfe69e28 (127.19.226.195) to f366ff6c37a24eaaa6abcafab1467912 (127.19.226.194). New cstate: current_term: 2 leader_uuid: "f366ff6c37a24eaaa6abcafab1467912" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:49.639009 24616 raft_consensus.cc:2463] T 630fe12978194fb9ae9f60597b3ce1c1 P 08c70c9a6a064d40bfa68fd105d86d86 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f366ff6c37a24eaaa6abcafab1467912 in term 2.
I20250109 08:58:49.640233 24769 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1c586b8d3a64430bf1fe82b27fef388" candidate_uuid: "08c70c9a6a064d40bfa68fd105d86d86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1a9b5a4afd63410b813b41edbfe69e28"
I20250109 08:58:49.640925 24769 raft_consensus.cc:3049] T c1c586b8d3a64430bf1fe82b27fef388 P 1a9b5a4afd63410b813b41edbfe69e28 [term 1 LEADER]: Stepping down as leader of term 1
I20250109 08:58:49.641228 24769 raft_consensus.cc:738] T c1c586b8d3a64430bf1fe82b27fef388 P 1a9b5a4afd63410b813b41edbfe69e28 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1a9b5a4afd63410b813b41edbfe69e28, State: Running, Role: LEADER
I20250109 08:58:49.642143 24769 consensus_queue.cc:260] T c1c586b8d3a64430bf1fe82b27fef388 P 1a9b5a4afd63410b813b41edbfe69e28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:49.644315 24769 raft_consensus.cc:3054] T c1c586b8d3a64430bf1fe82b27fef388 P 1a9b5a4afd63410b813b41edbfe69e28 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:49.650921 24694 raft_consensus.cc:2463] T c1c586b8d3a64430bf1fe82b27fef388 P f366ff6c37a24eaaa6abcafab1467912 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 08c70c9a6a064d40bfa68fd105d86d86 in term 2.
I20250109 08:58:49.652061 24695 raft_consensus.cc:1235] T c1c586b8d3a64430bf1fe82b27fef388 P f366ff6c37a24eaaa6abcafab1467912 [term 2 FOLLOWER]: Rejecting Update request from peer 1a9b5a4afd63410b813b41edbfe69e28 for earlier term 1. Current term is 2. Ops: []
I20250109 08:58:49.651872 24849 consensus_queue.cc:1035] T d8f9f0e755da4a7186c2c9316918ca73 P f366ff6c37a24eaaa6abcafab1467912 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:58:49.652411 24583 leader_election.cc:304] T c1c586b8d3a64430bf1fe82b27fef388 P 08c70c9a6a064d40bfa68fd105d86d86 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08c70c9a6a064d40bfa68fd105d86d86, f366ff6c37a24eaaa6abcafab1467912; no voters:
I20250109 08:58:49.653373 24822 raft_consensus.cc:2798] T c1c586b8d3a64430bf1fe82b27fef388 P 08c70c9a6a064d40bfa68fd105d86d86 [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:58:49.653818 24822 raft_consensus.cc:695] T c1c586b8d3a64430bf1fe82b27fef388 P 08c70c9a6a064d40bfa68fd105d86d86 [term 2 LEADER]: Becoming Leader. State: Replica: 08c70c9a6a064d40bfa68fd105d86d86, State: Running, Role: LEADER
I20250109 08:58:49.654878 24822 consensus_queue.cc:237] T c1c586b8d3a64430bf1fe82b27fef388 P 08c70c9a6a064d40bfa68fd105d86d86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:49.657302 24769 raft_consensus.cc:2463] T c1c586b8d3a64430bf1fe82b27fef388 P 1a9b5a4afd63410b813b41edbfe69e28 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 08c70c9a6a064d40bfa68fd105d86d86 in term 2.
I20250109 08:58:49.665146 24515 catalog_manager.cc:5526] T c1c586b8d3a64430bf1fe82b27fef388 P 08c70c9a6a064d40bfa68fd105d86d86 reported cstate change: term changed from 1 to 2, leader changed from 1a9b5a4afd63410b813b41edbfe69e28 (127.19.226.195) to 08c70c9a6a064d40bfa68fd105d86d86 (127.19.226.193). New cstate: current_term: 2 leader_uuid: "08c70c9a6a064d40bfa68fd105d86d86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:49.706777 24849 consensus_queue.cc:1035] T 970b2f8b130d431cb25d79d33d5f0e5c P f366ff6c37a24eaaa6abcafab1467912 [LEADER]: Connected to new peer: Peer: permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:58:49.714066 24813 raft_consensus.cc:988] T 970b2f8b130d431cb25d79d33d5f0e5c P f366ff6c37a24eaaa6abcafab1467912: : Instructing follower 08c70c9a6a064d40bfa68fd105d86d86 to start an election
I20250109 08:58:49.714594 24843 raft_consensus.cc:1076] T 970b2f8b130d431cb25d79d33d5f0e5c P f366ff6c37a24eaaa6abcafab1467912 [term 1 LEADER]: Signalling peer 08c70c9a6a064d40bfa68fd105d86d86 to start an election
I20250109 08:58:49.716848 24616 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "970b2f8b130d431cb25d79d33d5f0e5c"
dest_uuid: "08c70c9a6a064d40bfa68fd105d86d86"
from {username='slave'} at 127.0.0.1:35904
I20250109 08:58:49.717466 24616 raft_consensus.cc:491] T 970b2f8b130d431cb25d79d33d5f0e5c P 08c70c9a6a064d40bfa68fd105d86d86 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:58:49.717846 24616 raft_consensus.cc:3054] T 970b2f8b130d431cb25d79d33d5f0e5c P 08c70c9a6a064d40bfa68fd105d86d86 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:49.720383 24849 consensus_queue.cc:1035] T 970b2f8b130d431cb25d79d33d5f0e5c P f366ff6c37a24eaaa6abcafab1467912 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:58:49.725268 24616 raft_consensus.cc:513] T 970b2f8b130d431cb25d79d33d5f0e5c P 08c70c9a6a064d40bfa68fd105d86d86 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:49.728160 24694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "970b2f8b130d431cb25d79d33d5f0e5c" candidate_uuid: "08c70c9a6a064d40bfa68fd105d86d86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f366ff6c37a24eaaa6abcafab1467912"
I20250109 08:58:49.728806 24694 raft_consensus.cc:3049] T 970b2f8b130d431cb25d79d33d5f0e5c P f366ff6c37a24eaaa6abcafab1467912 [term 1 LEADER]: Stepping down as leader of term 1
I20250109 08:58:49.728261 24616 leader_election.cc:290] T 970b2f8b130d431cb25d79d33d5f0e5c P 08c70c9a6a064d40bfa68fd105d86d86 [CANDIDATE]: Term 2 election: Requested vote from peers f366ff6c37a24eaaa6abcafab1467912 (127.19.226.194:43355), 1a9b5a4afd63410b813b41edbfe69e28 (127.19.226.195:42529)
I20250109 08:58:49.729144 24694 raft_consensus.cc:738] T 970b2f8b130d431cb25d79d33d5f0e5c P f366ff6c37a24eaaa6abcafab1467912 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f366ff6c37a24eaaa6abcafab1467912, State: Running, Role: LEADER
I20250109 08:58:49.729512 24769 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "970b2f8b130d431cb25d79d33d5f0e5c" candidate_uuid: "08c70c9a6a064d40bfa68fd105d86d86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1a9b5a4afd63410b813b41edbfe69e28"
I20250109 08:58:49.730134 24769 raft_consensus.cc:3054] T 970b2f8b130d431cb25d79d33d5f0e5c P 1a9b5a4afd63410b813b41edbfe69e28 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:49.730058 24694 consensus_queue.cc:260] T 970b2f8b130d431cb25d79d33d5f0e5c P f366ff6c37a24eaaa6abcafab1467912 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:49.731402 24694 raft_consensus.cc:3054] T 970b2f8b130d431cb25d79d33d5f0e5c P f366ff6c37a24eaaa6abcafab1467912 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:49.737237 24769 raft_consensus.cc:2463] T 970b2f8b130d431cb25d79d33d5f0e5c P 1a9b5a4afd63410b813b41edbfe69e28 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 08c70c9a6a064d40bfa68fd105d86d86 in term 2.
I20250109 08:58:49.738368 24586 leader_election.cc:304] T 970b2f8b130d431cb25d79d33d5f0e5c P 08c70c9a6a064d40bfa68fd105d86d86 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08c70c9a6a064d40bfa68fd105d86d86, 1a9b5a4afd63410b813b41edbfe69e28; no voters:
I20250109 08:58:49.737558 24694 raft_consensus.cc:2463] T 970b2f8b130d431cb25d79d33d5f0e5c P f366ff6c37a24eaaa6abcafab1467912 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 08c70c9a6a064d40bfa68fd105d86d86 in term 2.
I20250109 08:58:49.739097 24822 raft_consensus.cc:2798] T 970b2f8b130d431cb25d79d33d5f0e5c P 08c70c9a6a064d40bfa68fd105d86d86 [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:58:49.739583 24822 raft_consensus.cc:695] T 970b2f8b130d431cb25d79d33d5f0e5c P 08c70c9a6a064d40bfa68fd105d86d86 [term 2 LEADER]: Becoming Leader. State: Replica: 08c70c9a6a064d40bfa68fd105d86d86, State: Running, Role: LEADER
I20250109 08:58:49.740434 24822 consensus_queue.cc:237] T 970b2f8b130d431cb25d79d33d5f0e5c P 08c70c9a6a064d40bfa68fd105d86d86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:49.746618 24515 catalog_manager.cc:5526] T 970b2f8b130d431cb25d79d33d5f0e5c P 08c70c9a6a064d40bfa68fd105d86d86 reported cstate change: term changed from 1 to 2, leader changed from f366ff6c37a24eaaa6abcafab1467912 (127.19.226.194) to 08c70c9a6a064d40bfa68fd105d86d86 (127.19.226.193). New cstate: current_term: 2 leader_uuid: "08c70c9a6a064d40bfa68fd105d86d86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:49.773634 24815 consensus_queue.cc:1035] T 4e2136a82d344ae6b0657803f1551906 P 1a9b5a4afd63410b813b41edbfe69e28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:58:49.785676 24861 consensus_queue.cc:1035] T 4e2136a82d344ae6b0657803f1551906 P 1a9b5a4afd63410b813b41edbfe69e28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:58:50.094074 24769 raft_consensus.cc:1270] T 630fe12978194fb9ae9f60597b3ce1c1 P 1a9b5a4afd63410b813b41edbfe69e28 [term 2 FOLLOWER]: Refusing update from remote peer f366ff6c37a24eaaa6abcafab1467912: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:58:50.095235 24843 consensus_queue.cc:1035] T 630fe12978194fb9ae9f60597b3ce1c1 P f366ff6c37a24eaaa6abcafab1467912 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:58:50.102725 24616 raft_consensus.cc:1270] T 630fe12978194fb9ae9f60597b3ce1c1 P 08c70c9a6a064d40bfa68fd105d86d86 [term 2 FOLLOWER]: Refusing update from remote peer f366ff6c37a24eaaa6abcafab1467912: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:58:50.104507 24849 consensus_queue.cc:1035] T 630fe12978194fb9ae9f60597b3ce1c1 P f366ff6c37a24eaaa6abcafab1467912 [LEADER]: Connected to new peer: Peer: permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:58:50.113816 24769 raft_consensus.cc:1270] T c1c586b8d3a64430bf1fe82b27fef388 P 1a9b5a4afd63410b813b41edbfe69e28 [term 2 FOLLOWER]: Refusing update from remote peer 08c70c9a6a064d40bfa68fd105d86d86: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:58:50.115006 24822 consensus_queue.cc:1035] T c1c586b8d3a64430bf1fe82b27fef388 P 08c70c9a6a064d40bfa68fd105d86d86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:58:50.122918 24694 raft_consensus.cc:1270] T c1c586b8d3a64430bf1fe82b27fef388 P f366ff6c37a24eaaa6abcafab1467912 [term 2 FOLLOWER]: Refusing update from remote peer 08c70c9a6a064d40bfa68fd105d86d86: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:58:50.124559 24822 consensus_queue.cc:1035] T c1c586b8d3a64430bf1fe82b27fef388 P 08c70c9a6a064d40bfa68fd105d86d86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:58:50.240629 24769 raft_consensus.cc:1270] T 970b2f8b130d431cb25d79d33d5f0e5c P 1a9b5a4afd63410b813b41edbfe69e28 [term 2 FOLLOWER]: Refusing update from remote peer 08c70c9a6a064d40bfa68fd105d86d86: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:58:50.241703 24822 consensus_queue.cc:1035] T 970b2f8b130d431cb25d79d33d5f0e5c P 08c70c9a6a064d40bfa68fd105d86d86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:58:50.248636 24694 raft_consensus.cc:1270] T 970b2f8b130d431cb25d79d33d5f0e5c P f366ff6c37a24eaaa6abcafab1467912 [term 2 FOLLOWER]: Refusing update from remote peer 08c70c9a6a064d40bfa68fd105d86d86: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:58:50.250255 24893 consensus_queue.cc:1035] T 970b2f8b130d431cb25d79d33d5f0e5c P 08c70c9a6a064d40bfa68fd105d86d86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20250109 08:58:52.597059 20363 test_util.cc:274] Using random seed: 804497115
I20250109 08:58:52.623492 24517 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56748:
name: "soft_deleted_table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250109 08:58:52.625792 24517 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250109 08:58:52.673373 24605 tablet_service.cc:1467] Processing CreateTablet for tablet a3207340610143c7bfb4943c87dc78bb (DEFAULT_TABLE table=soft_deleted_table [id=f05587e8b44d44c898cd6984d831bfbb]), partition=RANGE (key) PARTITION VALUES < 238609294
I20250109 08:58:52.675338 24752 tablet_service.cc:1467] Processing CreateTablet for tablet a3207340610143c7bfb4943c87dc78bb (DEFAULT_TABLE table=soft_deleted_table [id=f05587e8b44d44c898cd6984d831bfbb]), partition=RANGE (key) PARTITION VALUES < 238609294
I20250109 08:58:52.676940 24752 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3207340610143c7bfb4943c87dc78bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:52.677130 24605 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3207340610143c7bfb4943c87dc78bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:52.677778 24751 tablet_service.cc:1467] Processing CreateTablet for tablet 0364f388e25b466b8b39fdaa50aebf42 (DEFAULT_TABLE table=soft_deleted_table [id=f05587e8b44d44c898cd6984d831bfbb]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20250109 08:58:52.679126 24751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0364f388e25b466b8b39fdaa50aebf42. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:52.680042 24684 tablet_service.cc:1467] Processing CreateTablet for tablet 0364f388e25b466b8b39fdaa50aebf42 (DEFAULT_TABLE table=soft_deleted_table [id=f05587e8b44d44c898cd6984d831bfbb]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20250109 08:58:52.681455 24684 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0364f388e25b466b8b39fdaa50aebf42. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:52.685747 24677 tablet_service.cc:1467] Processing CreateTablet for tablet a3207340610143c7bfb4943c87dc78bb (DEFAULT_TABLE table=soft_deleted_table [id=f05587e8b44d44c898cd6984d831bfbb]), partition=RANGE (key) PARTITION VALUES < 238609294
I20250109 08:58:52.687019 24677 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3207340610143c7bfb4943c87dc78bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:52.690831 24607 tablet_service.cc:1467] Processing CreateTablet for tablet d8c90027b99f41b78f3b41c079e77ed7 (DEFAULT_TABLE table=soft_deleted_table [id=f05587e8b44d44c898cd6984d831bfbb]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20250109 08:58:52.692138 24607 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8c90027b99f41b78f3b41c079e77ed7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:52.693671 24603 tablet_service.cc:1467] Processing CreateTablet for tablet e8d95383d13941a99003d8e54fbed202 (DEFAULT_TABLE table=soft_deleted_table [id=f05587e8b44d44c898cd6984d831bfbb]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20250109 08:58:52.695096 24603 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8d95383d13941a99003d8e54fbed202. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:52.675776 24609 tablet_service.cc:1467] Processing CreateTablet for tablet 0364f388e25b466b8b39fdaa50aebf42 (DEFAULT_TABLE table=soft_deleted_table [id=f05587e8b44d44c898cd6984d831bfbb]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20250109 08:58:52.697896 24680 tablet_service.cc:1467] Processing CreateTablet for tablet d8c90027b99f41b78f3b41c079e77ed7 (DEFAULT_TABLE table=soft_deleted_table [id=f05587e8b44d44c898cd6984d831bfbb]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20250109 08:58:52.699250 24680 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8c90027b99f41b78f3b41c079e77ed7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:52.699630 24753 tablet_service.cc:1467] Processing CreateTablet for tablet d8c90027b99f41b78f3b41c079e77ed7 (DEFAULT_TABLE table=soft_deleted_table [id=f05587e8b44d44c898cd6984d831bfbb]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20250109 08:58:52.700973 24679 tablet_service.cc:1467] Processing CreateTablet for tablet e8d95383d13941a99003d8e54fbed202 (DEFAULT_TABLE table=soft_deleted_table [id=f05587e8b44d44c898cd6984d831bfbb]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20250109 08:58:52.702077 24753 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8c90027b99f41b78f3b41c079e77ed7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:52.702261 24679 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8d95383d13941a99003d8e54fbed202. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:52.702841 24755 tablet_service.cc:1467] Processing CreateTablet for tablet e8d95383d13941a99003d8e54fbed202 (DEFAULT_TABLE table=soft_deleted_table [id=f05587e8b44d44c898cd6984d831bfbb]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20250109 08:58:52.704135 24755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8d95383d13941a99003d8e54fbed202. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:52.704912 24609 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0364f388e25b466b8b39fdaa50aebf42. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:52.722038 24606 tablet_service.cc:1467] Processing CreateTablet for tablet ff0a7cf17ff34eedbdda1e5a4fac0dfa (DEFAULT_TABLE table=soft_deleted_table [id=f05587e8b44d44c898cd6984d831bfbb]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20250109 08:58:52.723481 24606 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff0a7cf17ff34eedbdda1e5a4fac0dfa. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:52.735350 24682 tablet_service.cc:1467] Processing CreateTablet for tablet ff0a7cf17ff34eedbdda1e5a4fac0dfa (DEFAULT_TABLE table=soft_deleted_table [id=f05587e8b44d44c898cd6984d831bfbb]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20250109 08:58:52.735971 24908 tablet_bootstrap.cc:492] T d8c90027b99f41b78f3b41c079e77ed7 P f366ff6c37a24eaaa6abcafab1467912: Bootstrap starting.
I20250109 08:58:52.737522 24682 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff0a7cf17ff34eedbdda1e5a4fac0dfa. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:52.738111 24756 tablet_service.cc:1467] Processing CreateTablet for tablet ff0a7cf17ff34eedbdda1e5a4fac0dfa (DEFAULT_TABLE table=soft_deleted_table [id=f05587e8b44d44c898cd6984d831bfbb]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20250109 08:58:52.739568 24756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff0a7cf17ff34eedbdda1e5a4fac0dfa. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:52.739965 24684 tablet_service.cc:1467] Processing CreateTablet for tablet 53eef9db17b548f081a60a39a43e4f19 (DEFAULT_TABLE table=soft_deleted_table [id=f05587e8b44d44c898cd6984d831bfbb]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20250109 08:58:52.741295 24684 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53eef9db17b548f081a60a39a43e4f19. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:52.741498 24608 tablet_service.cc:1467] Processing CreateTablet for tablet 53eef9db17b548f081a60a39a43e4f19 (DEFAULT_TABLE table=soft_deleted_table [id=f05587e8b44d44c898cd6984d831bfbb]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20250109 08:58:52.742705 24608 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53eef9db17b548f081a60a39a43e4f19. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:52.744000 24610 tablet_service.cc:1467] Processing CreateTablet for tablet 9e0af0555b484b748a03f99b45c463ef (DEFAULT_TABLE table=soft_deleted_table [id=f05587e8b44d44c898cd6984d831bfbb]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20250109 08:58:52.745312 24610 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9e0af0555b484b748a03f99b45c463ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:52.746054 24757 tablet_service.cc:1467] Processing CreateTablet for tablet 53eef9db17b548f081a60a39a43e4f19 (DEFAULT_TABLE table=soft_deleted_table [id=f05587e8b44d44c898cd6984d831bfbb]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20250109 08:58:52.747285 24757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53eef9db17b548f081a60a39a43e4f19. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:52.752502 24908 tablet_bootstrap.cc:654] T d8c90027b99f41b78f3b41c079e77ed7 P f366ff6c37a24eaaa6abcafab1467912: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:52.756237 24680 tablet_service.cc:1467] Processing CreateTablet for tablet 9e0af0555b484b748a03f99b45c463ef (DEFAULT_TABLE table=soft_deleted_table [id=f05587e8b44d44c898cd6984d831bfbb]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20250109 08:58:52.757719 24680 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9e0af0555b484b748a03f99b45c463ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:52.758042 24759 tablet_service.cc:1467] Processing CreateTablet for tablet 9e0af0555b484b748a03f99b45c463ef (DEFAULT_TABLE table=soft_deleted_table [id=f05587e8b44d44c898cd6984d831bfbb]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20250109 08:58:52.759385 24759 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9e0af0555b484b748a03f99b45c463ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:52.770817 24681 tablet_service.cc:1467] Processing CreateTablet for tablet 2141d5ba289f4dd99a8b0c4c1d11c66a (DEFAULT_TABLE table=soft_deleted_table [id=f05587e8b44d44c898cd6984d831bfbb]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20250109 08:58:52.772292 24681 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2141d5ba289f4dd99a8b0c4c1d11c66a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:52.771931 24603 tablet_service.cc:1467] Processing CreateTablet for tablet 2141d5ba289f4dd99a8b0c4c1d11c66a (DEFAULT_TABLE table=soft_deleted_table [id=f05587e8b44d44c898cd6984d831bfbb]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20250109 08:58:52.773281 24603 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2141d5ba289f4dd99a8b0c4c1d11c66a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:52.774569 24758 tablet_service.cc:1467] Processing CreateTablet for tablet 2141d5ba289f4dd99a8b0c4c1d11c66a (DEFAULT_TABLE table=soft_deleted_table [id=f05587e8b44d44c898cd6984d831bfbb]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20250109 08:58:52.776225 24758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2141d5ba289f4dd99a8b0c4c1d11c66a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:52.778590 24609 tablet_service.cc:1467] Processing CreateTablet for tablet 0faa532dc2514e6b9991e3832a029fab (DEFAULT_TABLE table=soft_deleted_table [id=f05587e8b44d44c898cd6984d831bfbb]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20250109 08:58:52.788779 24679 tablet_service.cc:1467] Processing CreateTablet for tablet 0faa532dc2514e6b9991e3832a029fab (DEFAULT_TABLE table=soft_deleted_table [id=f05587e8b44d44c898cd6984d831bfbb]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20250109 08:58:52.790035 24679 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0faa532dc2514e6b9991e3832a029fab. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:52.795529 24609 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0faa532dc2514e6b9991e3832a029fab. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:52.802204 24910 tablet_bootstrap.cc:492] T a3207340610143c7bfb4943c87dc78bb P 1a9b5a4afd63410b813b41edbfe69e28: Bootstrap starting.
I20250109 08:58:52.805938 24909 tablet_bootstrap.cc:492] T a3207340610143c7bfb4943c87dc78bb P 08c70c9a6a064d40bfa68fd105d86d86: Bootstrap starting.
I20250109 08:58:52.814733 24756 tablet_service.cc:1467] Processing CreateTablet for tablet 0faa532dc2514e6b9991e3832a029fab (DEFAULT_TABLE table=soft_deleted_table [id=f05587e8b44d44c898cd6984d831bfbb]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20250109 08:58:52.819479 24756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0faa532dc2514e6b9991e3832a029fab. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:58:52.817430 24908 tablet_bootstrap.cc:492] T d8c90027b99f41b78f3b41c079e77ed7 P f366ff6c37a24eaaa6abcafab1467912: No bootstrap required, opened a new log
I20250109 08:58:52.821062 24908 ts_tablet_manager.cc:1397] T d8c90027b99f41b78f3b41c079e77ed7 P f366ff6c37a24eaaa6abcafab1467912: Time spent bootstrapping tablet: real 0.085s user 0.016s sys 0.005s
I20250109 08:58:52.824012 24908 raft_consensus.cc:357] T d8c90027b99f41b78f3b41c079e77ed7 P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:52.832508 24908 raft_consensus.cc:383] T d8c90027b99f41b78f3b41c079e77ed7 P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:52.834774 24908 raft_consensus.cc:738] T d8c90027b99f41b78f3b41c079e77ed7 P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f366ff6c37a24eaaa6abcafab1467912, State: Initialized, Role: FOLLOWER
I20250109 08:58:52.837702 24910 tablet_bootstrap.cc:654] T a3207340610143c7bfb4943c87dc78bb P 1a9b5a4afd63410b813b41edbfe69e28: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:52.835518 24908 consensus_queue.cc:260] T d8c90027b99f41b78f3b41c079e77ed7 P f366ff6c37a24eaaa6abcafab1467912 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:52.840225 24908 ts_tablet_manager.cc:1428] T d8c90027b99f41b78f3b41c079e77ed7 P f366ff6c37a24eaaa6abcafab1467912: Time spent starting tablet: real 0.019s user 0.005s sys 0.000s
I20250109 08:58:52.841831 24908 tablet_bootstrap.cc:492] T 0364f388e25b466b8b39fdaa50aebf42 P f366ff6c37a24eaaa6abcafab1467912: Bootstrap starting.
I20250109 08:58:52.846374 24909 tablet_bootstrap.cc:654] T a3207340610143c7bfb4943c87dc78bb P 08c70c9a6a064d40bfa68fd105d86d86: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:52.848528 24908 tablet_bootstrap.cc:654] T 0364f388e25b466b8b39fdaa50aebf42 P f366ff6c37a24eaaa6abcafab1467912: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:52.857677 24910 tablet_bootstrap.cc:492] T a3207340610143c7bfb4943c87dc78bb P 1a9b5a4afd63410b813b41edbfe69e28: No bootstrap required, opened a new log
I20250109 08:58:52.857875 24909 tablet_bootstrap.cc:492] T a3207340610143c7bfb4943c87dc78bb P 08c70c9a6a064d40bfa68fd105d86d86: No bootstrap required, opened a new log
I20250109 08:58:52.857945 24908 tablet_bootstrap.cc:492] T 0364f388e25b466b8b39fdaa50aebf42 P f366ff6c37a24eaaa6abcafab1467912: No bootstrap required, opened a new log
I20250109 08:58:52.858201 24910 ts_tablet_manager.cc:1397] T a3207340610143c7bfb4943c87dc78bb P 1a9b5a4afd63410b813b41edbfe69e28: Time spent bootstrapping tablet: real 0.056s user 0.009s sys 0.009s
I20250109 08:58:52.858489 24909 ts_tablet_manager.cc:1397] T a3207340610143c7bfb4943c87dc78bb P 08c70c9a6a064d40bfa68fd105d86d86: Time spent bootstrapping tablet: real 0.053s user 0.006s sys 0.012s
I20250109 08:58:52.858539 24908 ts_tablet_manager.cc:1397] T 0364f388e25b466b8b39fdaa50aebf42 P f366ff6c37a24eaaa6abcafab1467912: Time spent bootstrapping tablet: real 0.017s user 0.007s sys 0.009s
I20250109 08:58:52.861248 24908 raft_consensus.cc:357] T 0364f388e25b466b8b39fdaa50aebf42 P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:52.861115 24910 raft_consensus.cc:357] T a3207340610143c7bfb4943c87dc78bb P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:52.861910 24908 raft_consensus.cc:383] T 0364f388e25b466b8b39fdaa50aebf42 P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:52.862026 24910 raft_consensus.cc:383] T a3207340610143c7bfb4943c87dc78bb P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:52.861685 24909 raft_consensus.cc:357] T a3207340610143c7bfb4943c87dc78bb P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:52.862232 24908 raft_consensus.cc:738] T 0364f388e25b466b8b39fdaa50aebf42 P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f366ff6c37a24eaaa6abcafab1467912, State: Initialized, Role: FOLLOWER
I20250109 08:58:52.862419 24910 raft_consensus.cc:738] T a3207340610143c7bfb4943c87dc78bb P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a9b5a4afd63410b813b41edbfe69e28, State: Initialized, Role: FOLLOWER
I20250109 08:58:52.862515 24909 raft_consensus.cc:383] T a3207340610143c7bfb4943c87dc78bb P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:52.862936 24909 raft_consensus.cc:738] T a3207340610143c7bfb4943c87dc78bb P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08c70c9a6a064d40bfa68fd105d86d86, State: Initialized, Role: FOLLOWER
I20250109 08:58:52.862995 24908 consensus_queue.cc:260] T 0364f388e25b466b8b39fdaa50aebf42 P f366ff6c37a24eaaa6abcafab1467912 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:52.863260 24910 consensus_queue.cc:260] T a3207340610143c7bfb4943c87dc78bb P 1a9b5a4afd63410b813b41edbfe69e28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:52.863656 24909 consensus_queue.cc:260] T a3207340610143c7bfb4943c87dc78bb P 08c70c9a6a064d40bfa68fd105d86d86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:52.865370 24908 ts_tablet_manager.cc:1428] T 0364f388e25b466b8b39fdaa50aebf42 P f366ff6c37a24eaaa6abcafab1467912: Time spent starting tablet: real 0.006s user 0.004s sys 0.001s
I20250109 08:58:52.866139 24909 ts_tablet_manager.cc:1428] T a3207340610143c7bfb4943c87dc78bb P 08c70c9a6a064d40bfa68fd105d86d86: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20250109 08:58:52.866323 24908 tablet_bootstrap.cc:492] T e8d95383d13941a99003d8e54fbed202 P f366ff6c37a24eaaa6abcafab1467912: Bootstrap starting.
I20250109 08:58:52.873131 24909 tablet_bootstrap.cc:492] T e8d95383d13941a99003d8e54fbed202 P 08c70c9a6a064d40bfa68fd105d86d86: Bootstrap starting.
I20250109 08:58:52.874238 24903 raft_consensus.cc:491] T 0364f388e25b466b8b39fdaa50aebf42 P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:52.874749 24903 raft_consensus.cc:513] T 0364f388e25b466b8b39fdaa50aebf42 P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:52.875319 24910 ts_tablet_manager.cc:1428] T a3207340610143c7bfb4943c87dc78bb P 1a9b5a4afd63410b813b41edbfe69e28: Time spent starting tablet: real 0.017s user 0.008s sys 0.000s
I20250109 08:58:52.875813 24908 tablet_bootstrap.cc:654] T e8d95383d13941a99003d8e54fbed202 P f366ff6c37a24eaaa6abcafab1467912: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:52.876436 24910 tablet_bootstrap.cc:492] T 0364f388e25b466b8b39fdaa50aebf42 P 1a9b5a4afd63410b813b41edbfe69e28: Bootstrap starting.
I20250109 08:58:52.881295 24909 tablet_bootstrap.cc:654] T e8d95383d13941a99003d8e54fbed202 P 08c70c9a6a064d40bfa68fd105d86d86: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:52.883587 24910 tablet_bootstrap.cc:654] T 0364f388e25b466b8b39fdaa50aebf42 P 1a9b5a4afd63410b813b41edbfe69e28: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:52.893699 24903 leader_election.cc:290] T 0364f388e25b466b8b39fdaa50aebf42 P f366ff6c37a24eaaa6abcafab1467912 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 08c70c9a6a064d40bfa68fd105d86d86 (127.19.226.193:43567), 1a9b5a4afd63410b813b41edbfe69e28 (127.19.226.195:42529)
I20250109 08:58:52.895440 24910 tablet_bootstrap.cc:492] T 0364f388e25b466b8b39fdaa50aebf42 P 1a9b5a4afd63410b813b41edbfe69e28: No bootstrap required, opened a new log
I20250109 08:58:52.896011 24910 ts_tablet_manager.cc:1397] T 0364f388e25b466b8b39fdaa50aebf42 P 1a9b5a4afd63410b813b41edbfe69e28: Time spent bootstrapping tablet: real 0.020s user 0.010s sys 0.007s
I20250109 08:58:52.895735 24767 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0364f388e25b466b8b39fdaa50aebf42" candidate_uuid: "f366ff6c37a24eaaa6abcafab1467912" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a9b5a4afd63410b813b41edbfe69e28" is_pre_election: true
W20250109 08:58:52.897465 24661 leader_election.cc:343] T 0364f388e25b466b8b39fdaa50aebf42 P f366ff6c37a24eaaa6abcafab1467912 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1a9b5a4afd63410b813b41edbfe69e28 (127.19.226.195:42529): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:52.901050 24910 raft_consensus.cc:357] T 0364f388e25b466b8b39fdaa50aebf42 P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:52.901929 24910 raft_consensus.cc:383] T 0364f388e25b466b8b39fdaa50aebf42 P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:52.902343 24910 raft_consensus.cc:738] T 0364f388e25b466b8b39fdaa50aebf42 P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a9b5a4afd63410b813b41edbfe69e28, State: Initialized, Role: FOLLOWER
I20250109 08:58:52.903112 24910 consensus_queue.cc:260] T 0364f388e25b466b8b39fdaa50aebf42 P 1a9b5a4afd63410b813b41edbfe69e28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:52.906198 24910 ts_tablet_manager.cc:1428] T 0364f388e25b466b8b39fdaa50aebf42 P 1a9b5a4afd63410b813b41edbfe69e28: Time spent starting tablet: real 0.008s user 0.006s sys 0.001s
I20250109 08:58:52.907177 24910 tablet_bootstrap.cc:492] T ff0a7cf17ff34eedbdda1e5a4fac0dfa P 1a9b5a4afd63410b813b41edbfe69e28: Bootstrap starting.
I20250109 08:58:52.913818 24910 tablet_bootstrap.cc:654] T ff0a7cf17ff34eedbdda1e5a4fac0dfa P 1a9b5a4afd63410b813b41edbfe69e28: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:52.922658 24620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0364f388e25b466b8b39fdaa50aebf42" candidate_uuid: "f366ff6c37a24eaaa6abcafab1467912" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08c70c9a6a064d40bfa68fd105d86d86" is_pre_election: true
I20250109 08:58:52.924219 24908 tablet_bootstrap.cc:492] T e8d95383d13941a99003d8e54fbed202 P f366ff6c37a24eaaa6abcafab1467912: No bootstrap required, opened a new log
W20250109 08:58:52.924290 24661 leader_election.cc:343] T 0364f388e25b466b8b39fdaa50aebf42 P f366ff6c37a24eaaa6abcafab1467912 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 08c70c9a6a064d40bfa68fd105d86d86 (127.19.226.193:43567): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:52.924921 24908 ts_tablet_manager.cc:1397] T e8d95383d13941a99003d8e54fbed202 P f366ff6c37a24eaaa6abcafab1467912: Time spent bootstrapping tablet: real 0.059s user 0.031s sys 0.021s
I20250109 08:58:52.924921 24661 leader_election.cc:304] T 0364f388e25b466b8b39fdaa50aebf42 P f366ff6c37a24eaaa6abcafab1467912 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f366ff6c37a24eaaa6abcafab1467912; no voters: 08c70c9a6a064d40bfa68fd105d86d86, 1a9b5a4afd63410b813b41edbfe69e28
I20250109 08:58:52.925781 24903 raft_consensus.cc:2743] T 0364f388e25b466b8b39fdaa50aebf42 P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:58:52.927606 24910 tablet_bootstrap.cc:492] T ff0a7cf17ff34eedbdda1e5a4fac0dfa P 1a9b5a4afd63410b813b41edbfe69e28: No bootstrap required, opened a new log
I20250109 08:58:52.928035 24910 ts_tablet_manager.cc:1397] T ff0a7cf17ff34eedbdda1e5a4fac0dfa P 1a9b5a4afd63410b813b41edbfe69e28: Time spent bootstrapping tablet: real 0.021s user 0.010s sys 0.006s
I20250109 08:58:52.927699 24908 raft_consensus.cc:357] T e8d95383d13941a99003d8e54fbed202 P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:52.928817 24908 raft_consensus.cc:383] T e8d95383d13941a99003d8e54fbed202 P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:52.929044 24908 raft_consensus.cc:738] T e8d95383d13941a99003d8e54fbed202 P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f366ff6c37a24eaaa6abcafab1467912, State: Initialized, Role: FOLLOWER
I20250109 08:58:52.929647 24909 tablet_bootstrap.cc:492] T e8d95383d13941a99003d8e54fbed202 P 08c70c9a6a064d40bfa68fd105d86d86: No bootstrap required, opened a new log
I20250109 08:58:52.929594 24908 consensus_queue.cc:260] T e8d95383d13941a99003d8e54fbed202 P f366ff6c37a24eaaa6abcafab1467912 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:52.930153 24909 ts_tablet_manager.cc:1397] T e8d95383d13941a99003d8e54fbed202 P 08c70c9a6a064d40bfa68fd105d86d86: Time spent bootstrapping tablet: real 0.057s user 0.005s sys 0.019s
I20250109 08:58:52.930770 24910 raft_consensus.cc:357] T ff0a7cf17ff34eedbdda1e5a4fac0dfa P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:52.931550 24910 raft_consensus.cc:383] T ff0a7cf17ff34eedbdda1e5a4fac0dfa P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:52.931808 24910 raft_consensus.cc:738] T ff0a7cf17ff34eedbdda1e5a4fac0dfa P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a9b5a4afd63410b813b41edbfe69e28, State: Initialized, Role: FOLLOWER
I20250109 08:58:52.931864 24908 ts_tablet_manager.cc:1428] T e8d95383d13941a99003d8e54fbed202 P f366ff6c37a24eaaa6abcafab1467912: Time spent starting tablet: real 0.006s user 0.003s sys 0.000s
I20250109 08:58:52.932636 24910 consensus_queue.cc:260] T ff0a7cf17ff34eedbdda1e5a4fac0dfa P 1a9b5a4afd63410b813b41edbfe69e28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:52.932971 24908 tablet_bootstrap.cc:492] T 9e0af0555b484b748a03f99b45c463ef P f366ff6c37a24eaaa6abcafab1467912: Bootstrap starting.
I20250109 08:58:52.932968 24909 raft_consensus.cc:357] T e8d95383d13941a99003d8e54fbed202 P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:52.933929 24909 raft_consensus.cc:383] T e8d95383d13941a99003d8e54fbed202 P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:52.934219 24909 raft_consensus.cc:738] T e8d95383d13941a99003d8e54fbed202 P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08c70c9a6a064d40bfa68fd105d86d86, State: Initialized, Role: FOLLOWER
I20250109 08:58:52.934989 24909 consensus_queue.cc:260] T e8d95383d13941a99003d8e54fbed202 P 08c70c9a6a064d40bfa68fd105d86d86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:52.937341 24909 ts_tablet_manager.cc:1428] T e8d95383d13941a99003d8e54fbed202 P 08c70c9a6a064d40bfa68fd105d86d86: Time spent starting tablet: real 0.007s user 0.005s sys 0.003s
I20250109 08:58:52.938249 24909 tablet_bootstrap.cc:492] T 0364f388e25b466b8b39fdaa50aebf42 P 08c70c9a6a064d40bfa68fd105d86d86: Bootstrap starting.
I20250109 08:58:52.940443 24908 tablet_bootstrap.cc:654] T 9e0af0555b484b748a03f99b45c463ef P f366ff6c37a24eaaa6abcafab1467912: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:52.942663 24910 ts_tablet_manager.cc:1428] T ff0a7cf17ff34eedbdda1e5a4fac0dfa P 1a9b5a4afd63410b813b41edbfe69e28: Time spent starting tablet: real 0.014s user 0.007s sys 0.000s
I20250109 08:58:52.944257 24909 tablet_bootstrap.cc:654] T 0364f388e25b466b8b39fdaa50aebf42 P 08c70c9a6a064d40bfa68fd105d86d86: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:52.946642 24910 tablet_bootstrap.cc:492] T 9e0af0555b484b748a03f99b45c463ef P 1a9b5a4afd63410b813b41edbfe69e28: Bootstrap starting.
I20250109 08:58:52.952071 24908 tablet_bootstrap.cc:492] T 9e0af0555b484b748a03f99b45c463ef P f366ff6c37a24eaaa6abcafab1467912: No bootstrap required, opened a new log
I20250109 08:58:52.952666 24908 ts_tablet_manager.cc:1397] T 9e0af0555b484b748a03f99b45c463ef P f366ff6c37a24eaaa6abcafab1467912: Time spent bootstrapping tablet: real 0.020s user 0.018s sys 0.000s
I20250109 08:58:52.953543 24910 tablet_bootstrap.cc:654] T 9e0af0555b484b748a03f99b45c463ef P 1a9b5a4afd63410b813b41edbfe69e28: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:52.957116 24908 raft_consensus.cc:357] T 9e0af0555b484b748a03f99b45c463ef P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
W20250109 08:58:52.957597 24566 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20250109 08:58:52.958256 24908 raft_consensus.cc:383] T 9e0af0555b484b748a03f99b45c463ef P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:52.958647 24908 raft_consensus.cc:738] T 9e0af0555b484b748a03f99b45c463ef P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f366ff6c37a24eaaa6abcafab1467912, State: Initialized, Role: FOLLOWER
I20250109 08:58:52.959471 24908 consensus_queue.cc:260] T 9e0af0555b484b748a03f99b45c463ef P f366ff6c37a24eaaa6abcafab1467912 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:52.961632 24842 raft_consensus.cc:491] T a3207340610143c7bfb4943c87dc78bb P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:52.962042 24908 ts_tablet_manager.cc:1428] T 9e0af0555b484b748a03f99b45c463ef P f366ff6c37a24eaaa6abcafab1467912: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20250109 08:58:52.962757 24842 raft_consensus.cc:513] T a3207340610143c7bfb4943c87dc78bb P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:52.963645 24908 tablet_bootstrap.cc:492] T 53eef9db17b548f081a60a39a43e4f19 P f366ff6c37a24eaaa6abcafab1467912: Bootstrap starting.
I20250109 08:58:52.964561 24842 leader_election.cc:290] T a3207340610143c7bfb4943c87dc78bb P 08c70c9a6a064d40bfa68fd105d86d86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f366ff6c37a24eaaa6abcafab1467912 (127.19.226.194:43355), 1a9b5a4afd63410b813b41edbfe69e28 (127.19.226.195:42529)
I20250109 08:58:52.960346 24909 tablet_bootstrap.cc:492] T 0364f388e25b466b8b39fdaa50aebf42 P 08c70c9a6a064d40bfa68fd105d86d86: No bootstrap required, opened a new log
I20250109 08:58:52.965651 24694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3207340610143c7bfb4943c87dc78bb" candidate_uuid: "08c70c9a6a064d40bfa68fd105d86d86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f366ff6c37a24eaaa6abcafab1467912" is_pre_election: true
I20250109 08:58:52.966588 24909 ts_tablet_manager.cc:1397] T 0364f388e25b466b8b39fdaa50aebf42 P 08c70c9a6a064d40bfa68fd105d86d86: Time spent bootstrapping tablet: real 0.029s user 0.010s sys 0.007s
I20250109 08:58:52.965651 24767 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3207340610143c7bfb4943c87dc78bb" candidate_uuid: "08c70c9a6a064d40bfa68fd105d86d86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a9b5a4afd63410b813b41edbfe69e28" is_pre_election: true
I20250109 08:58:52.967115 24767 raft_consensus.cc:2463] T a3207340610143c7bfb4943c87dc78bb P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 08c70c9a6a064d40bfa68fd105d86d86 in term 0.
W20250109 08:58:52.967366 24583 leader_election.cc:343] T a3207340610143c7bfb4943c87dc78bb P 08c70c9a6a064d40bfa68fd105d86d86 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f366ff6c37a24eaaa6abcafab1467912 (127.19.226.194:43355): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:52.968235 24586 leader_election.cc:304] T a3207340610143c7bfb4943c87dc78bb P 08c70c9a6a064d40bfa68fd105d86d86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 08c70c9a6a064d40bfa68fd105d86d86, 1a9b5a4afd63410b813b41edbfe69e28; no voters: f366ff6c37a24eaaa6abcafab1467912
I20250109 08:58:52.969197 24842 raft_consensus.cc:2798] T a3207340610143c7bfb4943c87dc78bb P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:58:52.969533 24842 raft_consensus.cc:491] T a3207340610143c7bfb4943c87dc78bb P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:58:52.969888 24842 raft_consensus.cc:3054] T a3207340610143c7bfb4943c87dc78bb P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:52.970659 24903 raft_consensus.cc:491] T e8d95383d13941a99003d8e54fbed202 P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:52.971119 24903 raft_consensus.cc:513] T e8d95383d13941a99003d8e54fbed202 P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:52.973737 24620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8d95383d13941a99003d8e54fbed202" candidate_uuid: "f366ff6c37a24eaaa6abcafab1467912" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08c70c9a6a064d40bfa68fd105d86d86" is_pre_election: true
I20250109 08:58:52.974368 24620 raft_consensus.cc:2463] T e8d95383d13941a99003d8e54fbed202 P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f366ff6c37a24eaaa6abcafab1467912 in term 0.
I20250109 08:58:52.969465 24909 raft_consensus.cc:357] T 0364f388e25b466b8b39fdaa50aebf42 P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:52.974838 24909 raft_consensus.cc:383] T 0364f388e25b466b8b39fdaa50aebf42 P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:52.975124 24909 raft_consensus.cc:738] T 0364f388e25b466b8b39fdaa50aebf42 P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08c70c9a6a064d40bfa68fd105d86d86, State: Initialized, Role: FOLLOWER
I20250109 08:58:52.975555 24767 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8d95383d13941a99003d8e54fbed202" candidate_uuid: "f366ff6c37a24eaaa6abcafab1467912" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a9b5a4afd63410b813b41edbfe69e28" is_pre_election: true
I20250109 08:58:52.975752 24909 consensus_queue.cc:260] T 0364f388e25b466b8b39fdaa50aebf42 P 08c70c9a6a064d40bfa68fd105d86d86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:52.976753 24661 leader_election.cc:304] T e8d95383d13941a99003d8e54fbed202 P f366ff6c37a24eaaa6abcafab1467912 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08c70c9a6a064d40bfa68fd105d86d86, f366ff6c37a24eaaa6abcafab1467912; no voters:
I20250109 08:58:52.973873 24903 leader_election.cc:290] T e8d95383d13941a99003d8e54fbed202 P f366ff6c37a24eaaa6abcafab1467912 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 08c70c9a6a064d40bfa68fd105d86d86 (127.19.226.193:43567), 1a9b5a4afd63410b813b41edbfe69e28 (127.19.226.195:42529)
I20250109 08:58:52.978029 24908 tablet_bootstrap.cc:654] T 53eef9db17b548f081a60a39a43e4f19 P f366ff6c37a24eaaa6abcafab1467912: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:52.978094 24903 raft_consensus.cc:491] T d8c90027b99f41b78f3b41c079e77ed7 P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:52.978624 24903 raft_consensus.cc:513] T d8c90027b99f41b78f3b41c079e77ed7 P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:52.979813 24849 raft_consensus.cc:2798] T e8d95383d13941a99003d8e54fbed202 P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:58:52.980213 24849 raft_consensus.cc:491] T e8d95383d13941a99003d8e54fbed202 P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:58:52.977717 24909 ts_tablet_manager.cc:1428] T 0364f388e25b466b8b39fdaa50aebf42 P 08c70c9a6a064d40bfa68fd105d86d86: Time spent starting tablet: real 0.011s user 0.005s sys 0.000s
I20250109 08:58:52.980543 24903 leader_election.cc:290] T d8c90027b99f41b78f3b41c079e77ed7 P f366ff6c37a24eaaa6abcafab1467912 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 08c70c9a6a064d40bfa68fd105d86d86 (127.19.226.193:43567), 1a9b5a4afd63410b813b41edbfe69e28 (127.19.226.195:42529)
I20250109 08:58:52.980571 24849 raft_consensus.cc:3054] T e8d95383d13941a99003d8e54fbed202 P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:52.981251 24909 tablet_bootstrap.cc:492] T ff0a7cf17ff34eedbdda1e5a4fac0dfa P 08c70c9a6a064d40bfa68fd105d86d86: Bootstrap starting.
I20250109 08:58:52.988171 24909 tablet_bootstrap.cc:654] T ff0a7cf17ff34eedbdda1e5a4fac0dfa P 08c70c9a6a064d40bfa68fd105d86d86: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:52.988226 24849 raft_consensus.cc:513] T e8d95383d13941a99003d8e54fbed202 P f366ff6c37a24eaaa6abcafab1467912 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:52.995817 24849 leader_election.cc:290] T e8d95383d13941a99003d8e54fbed202 P f366ff6c37a24eaaa6abcafab1467912 [CANDIDATE]: Term 1 election: Requested vote from peers 08c70c9a6a064d40bfa68fd105d86d86 (127.19.226.193:43567), 1a9b5a4afd63410b813b41edbfe69e28 (127.19.226.195:42529)
I20250109 08:58:53.001482 24842 raft_consensus.cc:513] T a3207340610143c7bfb4943c87dc78bb P 08c70c9a6a064d40bfa68fd105d86d86 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:53.002319 24910 tablet_bootstrap.cc:492] T 9e0af0555b484b748a03f99b45c463ef P 1a9b5a4afd63410b813b41edbfe69e28: No bootstrap required, opened a new log
I20250109 08:58:53.003944 24910 ts_tablet_manager.cc:1397] T 9e0af0555b484b748a03f99b45c463ef P 1a9b5a4afd63410b813b41edbfe69e28: Time spent bootstrapping tablet: real 0.057s user 0.010s sys 0.008s
I20250109 08:58:53.005221 24842 leader_election.cc:290] T a3207340610143c7bfb4943c87dc78bb P 08c70c9a6a064d40bfa68fd105d86d86 [CANDIDATE]: Term 1 election: Requested vote from peers f366ff6c37a24eaaa6abcafab1467912 (127.19.226.194:43355), 1a9b5a4afd63410b813b41edbfe69e28 (127.19.226.195:42529)
I20250109 08:58:53.007524 24767 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3207340610143c7bfb4943c87dc78bb" candidate_uuid: "08c70c9a6a064d40bfa68fd105d86d86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a9b5a4afd63410b813b41edbfe69e28"
I20250109 08:58:53.008203 24767 raft_consensus.cc:3054] T a3207340610143c7bfb4943c87dc78bb P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:53.009397 24910 raft_consensus.cc:357] T 9e0af0555b484b748a03f99b45c463ef P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:53.010221 24910 raft_consensus.cc:383] T 9e0af0555b484b748a03f99b45c463ef P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:53.010500 24910 raft_consensus.cc:738] T 9e0af0555b484b748a03f99b45c463ef P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a9b5a4afd63410b813b41edbfe69e28, State: Initialized, Role: FOLLOWER
I20250109 08:58:53.013794 24620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8c90027b99f41b78f3b41c079e77ed7" candidate_uuid: "f366ff6c37a24eaaa6abcafab1467912" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08c70c9a6a064d40bfa68fd105d86d86" is_pre_election: true
I20250109 08:58:53.015444 24769 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8c90027b99f41b78f3b41c079e77ed7" candidate_uuid: "f366ff6c37a24eaaa6abcafab1467912" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a9b5a4afd63410b813b41edbfe69e28" is_pre_election: true
I20250109 08:58:53.017630 24908 tablet_bootstrap.cc:492] T 53eef9db17b548f081a60a39a43e4f19 P f366ff6c37a24eaaa6abcafab1467912: No bootstrap required, opened a new log
I20250109 08:58:53.018151 24908 ts_tablet_manager.cc:1397] T 53eef9db17b548f081a60a39a43e4f19 P f366ff6c37a24eaaa6abcafab1467912: Time spent bootstrapping tablet: real 0.055s user 0.008s sys 0.014s
I20250109 08:58:53.018116 24694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3207340610143c7bfb4943c87dc78bb" candidate_uuid: "08c70c9a6a064d40bfa68fd105d86d86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f366ff6c37a24eaaa6abcafab1467912"
I20250109 08:58:53.018748 24620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8d95383d13941a99003d8e54fbed202" candidate_uuid: "f366ff6c37a24eaaa6abcafab1467912" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08c70c9a6a064d40bfa68fd105d86d86"
I20250109 08:58:53.019321 24620 raft_consensus.cc:3054] T e8d95383d13941a99003d8e54fbed202 P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:53.019695 24909 tablet_bootstrap.cc:492] T ff0a7cf17ff34eedbdda1e5a4fac0dfa P 08c70c9a6a064d40bfa68fd105d86d86: No bootstrap required, opened a new log
W20250109 08:58:53.019796 24583 leader_election.cc:343] T a3207340610143c7bfb4943c87dc78bb P 08c70c9a6a064d40bfa68fd105d86d86 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f366ff6c37a24eaaa6abcafab1467912 (127.19.226.194:43355): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:53.020256 24909 ts_tablet_manager.cc:1397] T ff0a7cf17ff34eedbdda1e5a4fac0dfa P 08c70c9a6a064d40bfa68fd105d86d86: Time spent bootstrapping tablet: real 0.039s user 0.006s sys 0.015s
I20250109 08:58:53.018230 24767 raft_consensus.cc:2463] T a3207340610143c7bfb4943c87dc78bb P 1a9b5a4afd63410b813b41edbfe69e28 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 08c70c9a6a064d40bfa68fd105d86d86 in term 1.
I20250109 08:58:53.021605 24586 leader_election.cc:304] T a3207340610143c7bfb4943c87dc78bb P 08c70c9a6a064d40bfa68fd105d86d86 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 08c70c9a6a064d40bfa68fd105d86d86, 1a9b5a4afd63410b813b41edbfe69e28; no voters: f366ff6c37a24eaaa6abcafab1467912
I20250109 08:58:53.022059 24908 raft_consensus.cc:357] T 53eef9db17b548f081a60a39a43e4f19 P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:53.022413 24926 raft_consensus.cc:2798] T a3207340610143c7bfb4943c87dc78bb P 08c70c9a6a064d40bfa68fd105d86d86 [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:58:53.022833 24908 raft_consensus.cc:383] T 53eef9db17b548f081a60a39a43e4f19 P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:53.022679 24909 raft_consensus.cc:357] T ff0a7cf17ff34eedbdda1e5a4fac0dfa P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:53.023236 24908 raft_consensus.cc:738] T 53eef9db17b548f081a60a39a43e4f19 P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f366ff6c37a24eaaa6abcafab1467912, State: Initialized, Role: FOLLOWER
I20250109 08:58:53.023506 24909 raft_consensus.cc:383] T ff0a7cf17ff34eedbdda1e5a4fac0dfa P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:53.023927 24909 raft_consensus.cc:738] T ff0a7cf17ff34eedbdda1e5a4fac0dfa P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08c70c9a6a064d40bfa68fd105d86d86, State: Initialized, Role: FOLLOWER
I20250109 08:58:53.023967 24926 raft_consensus.cc:695] T a3207340610143c7bfb4943c87dc78bb P 08c70c9a6a064d40bfa68fd105d86d86 [term 1 LEADER]: Becoming Leader. State: Replica: 08c70c9a6a064d40bfa68fd105d86d86, State: Running, Role: LEADER
I20250109 08:58:53.023947 24908 consensus_queue.cc:260] T 53eef9db17b548f081a60a39a43e4f19 P f366ff6c37a24eaaa6abcafab1467912 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:53.024786 24909 consensus_queue.cc:260] T ff0a7cf17ff34eedbdda1e5a4fac0dfa P 08c70c9a6a064d40bfa68fd105d86d86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:53.025349 24926 consensus_queue.cc:237] T a3207340610143c7bfb4943c87dc78bb P 08c70c9a6a064d40bfa68fd105d86d86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:53.026793 24908 ts_tablet_manager.cc:1428] T 53eef9db17b548f081a60a39a43e4f19 P f366ff6c37a24eaaa6abcafab1467912: Time spent starting tablet: real 0.008s user 0.003s sys 0.003s
I20250109 08:58:53.027204 24909 ts_tablet_manager.cc:1428] T ff0a7cf17ff34eedbdda1e5a4fac0dfa P 08c70c9a6a064d40bfa68fd105d86d86: Time spent starting tablet: real 0.007s user 0.004s sys 0.000s
I20250109 08:58:53.012863 24910 consensus_queue.cc:260] T 9e0af0555b484b748a03f99b45c463ef P 1a9b5a4afd63410b813b41edbfe69e28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:53.028195 24909 tablet_bootstrap.cc:492] T 9e0af0555b484b748a03f99b45c463ef P 08c70c9a6a064d40bfa68fd105d86d86: Bootstrap starting.
I20250109 08:58:53.034225 24908 tablet_bootstrap.cc:492] T 2141d5ba289f4dd99a8b0c4c1d11c66a P f366ff6c37a24eaaa6abcafab1467912: Bootstrap starting.
I20250109 08:58:53.035513 24909 tablet_bootstrap.cc:654] T 9e0af0555b484b748a03f99b45c463ef P 08c70c9a6a064d40bfa68fd105d86d86: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:53.039940 24910 ts_tablet_manager.cc:1428] T 9e0af0555b484b748a03f99b45c463ef P 1a9b5a4afd63410b813b41edbfe69e28: Time spent starting tablet: real 0.036s user 0.006s sys 0.000s
I20250109 08:58:53.040819 24620 raft_consensus.cc:2463] T e8d95383d13941a99003d8e54fbed202 P 08c70c9a6a064d40bfa68fd105d86d86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f366ff6c37a24eaaa6abcafab1467912 in term 1.
I20250109 08:58:53.042781 24767 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8d95383d13941a99003d8e54fbed202" candidate_uuid: "f366ff6c37a24eaaa6abcafab1467912" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a9b5a4afd63410b813b41edbfe69e28"
I20250109 08:58:53.047703 24910 tablet_bootstrap.cc:492] T 0faa532dc2514e6b9991e3832a029fab P 1a9b5a4afd63410b813b41edbfe69e28: Bootstrap starting.
I20250109 08:58:53.049774 24909 tablet_bootstrap.cc:492] T 9e0af0555b484b748a03f99b45c463ef P 08c70c9a6a064d40bfa68fd105d86d86: No bootstrap required, opened a new log
W20250109 08:58:53.048547 24661 leader_election.cc:343] T e8d95383d13941a99003d8e54fbed202 P f366ff6c37a24eaaa6abcafab1467912 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1a9b5a4afd63410b813b41edbfe69e28 (127.19.226.195:42529): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:53.050288 24909 ts_tablet_manager.cc:1397] T 9e0af0555b484b748a03f99b45c463ef P 08c70c9a6a064d40bfa68fd105d86d86: Time spent bootstrapping tablet: real 0.022s user 0.011s sys 0.007s
W20250109 08:58:53.050844 24661 leader_election.cc:343] T d8c90027b99f41b78f3b41c079e77ed7 P f366ff6c37a24eaaa6abcafab1467912 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1a9b5a4afd63410b813b41edbfe69e28 (127.19.226.195:42529): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:53.046890 24908 tablet_bootstrap.cc:654] T 2141d5ba289f4dd99a8b0c4c1d11c66a P f366ff6c37a24eaaa6abcafab1467912: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:53.053516 24895 raft_consensus.cc:491] T ff0a7cf17ff34eedbdda1e5a4fac0dfa P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:53.053236 24909 raft_consensus.cc:357] T 9e0af0555b484b748a03f99b45c463ef P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
W20250109 08:58:53.055940 24661 leader_election.cc:343] T e8d95383d13941a99003d8e54fbed202 P f366ff6c37a24eaaa6abcafab1467912 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1a9b5a4afd63410b813b41edbfe69e28 (127.19.226.195:42529): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:53.055999 24909 raft_consensus.cc:383] T 9e0af0555b484b748a03f99b45c463ef P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:53.055909 24895 raft_consensus.cc:513] T ff0a7cf17ff34eedbdda1e5a4fac0dfa P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:53.056751 24909 raft_consensus.cc:738] T 9e0af0555b484b748a03f99b45c463ef P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08c70c9a6a064d40bfa68fd105d86d86, State: Initialized, Role: FOLLOWER
I20250109 08:58:53.057626 24909 consensus_queue.cc:260] T 9e0af0555b484b748a03f99b45c463ef P 08c70c9a6a064d40bfa68fd105d86d86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:53.059453 24694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff0a7cf17ff34eedbdda1e5a4fac0dfa" candidate_uuid: "1a9b5a4afd63410b813b41edbfe69e28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f366ff6c37a24eaaa6abcafab1467912" is_pre_election: true
I20250109 08:58:53.060344 24909 ts_tablet_manager.cc:1428] T 9e0af0555b484b748a03f99b45c463ef P 08c70c9a6a064d40bfa68fd105d86d86: Time spent starting tablet: real 0.010s user 0.007s sys 0.000s
W20250109 08:58:53.061064 24732 leader_election.cc:343] T ff0a7cf17ff34eedbdda1e5a4fac0dfa P 1a9b5a4afd63410b813b41edbfe69e28 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f366ff6c37a24eaaa6abcafab1467912 (127.19.226.194:43355): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:53.058175 24895 leader_election.cc:290] T ff0a7cf17ff34eedbdda1e5a4fac0dfa P 1a9b5a4afd63410b813b41edbfe69e28 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 08c70c9a6a064d40bfa68fd105d86d86 (127.19.226.193:43567), f366ff6c37a24eaaa6abcafab1467912 (127.19.226.194:43355)
I20250109 08:58:53.055411 24910 tablet_bootstrap.cc:654] T 0faa532dc2514e6b9991e3832a029fab P 1a9b5a4afd63410b813b41edbfe69e28: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:53.059530 24620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff0a7cf17ff34eedbdda1e5a4fac0dfa" candidate_uuid: "1a9b5a4afd63410b813b41edbfe69e28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08c70c9a6a064d40bfa68fd105d86d86" is_pre_election: true
I20250109 08:58:53.062182 24620 raft_consensus.cc:2463] T ff0a7cf17ff34eedbdda1e5a4fac0dfa P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a9b5a4afd63410b813b41edbfe69e28 in term 0.
I20250109 08:58:53.063191 24735 leader_election.cc:304] T ff0a7cf17ff34eedbdda1e5a4fac0dfa P 1a9b5a4afd63410b813b41edbfe69e28 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 08c70c9a6a064d40bfa68fd105d86d86, 1a9b5a4afd63410b813b41edbfe69e28; no voters: f366ff6c37a24eaaa6abcafab1467912
I20250109 08:58:53.061499 24909 tablet_bootstrap.cc:492] T 0faa532dc2514e6b9991e3832a029fab P 08c70c9a6a064d40bfa68fd105d86d86: Bootstrap starting.
I20250109 08:58:53.068567 24895 raft_consensus.cc:2798] T ff0a7cf17ff34eedbdda1e5a4fac0dfa P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:58:53.068979 24895 raft_consensus.cc:491] T ff0a7cf17ff34eedbdda1e5a4fac0dfa P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:58:53.069319 24895 raft_consensus.cc:3054] T ff0a7cf17ff34eedbdda1e5a4fac0dfa P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:53.071913 24909 tablet_bootstrap.cc:654] T 0faa532dc2514e6b9991e3832a029fab P 08c70c9a6a064d40bfa68fd105d86d86: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:53.077260 24910 tablet_bootstrap.cc:492] T 0faa532dc2514e6b9991e3832a029fab P 1a9b5a4afd63410b813b41edbfe69e28: No bootstrap required, opened a new log
I20250109 08:58:53.077788 24910 ts_tablet_manager.cc:1397] T 0faa532dc2514e6b9991e3832a029fab P 1a9b5a4afd63410b813b41edbfe69e28: Time spent bootstrapping tablet: real 0.030s user 0.010s sys 0.000s
I20250109 08:58:53.080858 24910 raft_consensus.cc:357] T 0faa532dc2514e6b9991e3832a029fab P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:53.078239 24895 raft_consensus.cc:513] T ff0a7cf17ff34eedbdda1e5a4fac0dfa P 1a9b5a4afd63410b813b41edbfe69e28 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:53.081895 24910 raft_consensus.cc:383] T 0faa532dc2514e6b9991e3832a029fab P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20250109 08:58:53.081719 24661 leader_election.cc:343] T d8c90027b99f41b78f3b41c079e77ed7 P f366ff6c37a24eaaa6abcafab1467912 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 08c70c9a6a064d40bfa68fd105d86d86 (127.19.226.193:43567): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:53.082314 24910 raft_consensus.cc:738] T 0faa532dc2514e6b9991e3832a029fab P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a9b5a4afd63410b813b41edbfe69e28, State: Initialized, Role: FOLLOWER
I20250109 08:58:53.083092 24910 consensus_queue.cc:260] T 0faa532dc2514e6b9991e3832a029fab P 1a9b5a4afd63410b813b41edbfe69e28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:53.084795 24620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff0a7cf17ff34eedbdda1e5a4fac0dfa" candidate_uuid: "1a9b5a4afd63410b813b41edbfe69e28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08c70c9a6a064d40bfa68fd105d86d86"
I20250109 08:58:53.085501 24620 raft_consensus.cc:3054] T ff0a7cf17ff34eedbdda1e5a4fac0dfa P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:53.093875 24694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff0a7cf17ff34eedbdda1e5a4fac0dfa" candidate_uuid: "1a9b5a4afd63410b813b41edbfe69e28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f366ff6c37a24eaaa6abcafab1467912"
W20250109 08:58:53.095569 24732 leader_election.cc:343] T ff0a7cf17ff34eedbdda1e5a4fac0dfa P 1a9b5a4afd63410b813b41edbfe69e28 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f366ff6c37a24eaaa6abcafab1467912 (127.19.226.194:43355): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:53.096230 24895 leader_election.cc:290] T ff0a7cf17ff34eedbdda1e5a4fac0dfa P 1a9b5a4afd63410b813b41edbfe69e28 [CANDIDATE]: Term 1 election: Requested vote from peers 08c70c9a6a064d40bfa68fd105d86d86 (127.19.226.193:43567), f366ff6c37a24eaaa6abcafab1467912 (127.19.226.194:43355)
I20250109 08:58:53.098454 24661 leader_election.cc:304] T d8c90027b99f41b78f3b41c079e77ed7 P f366ff6c37a24eaaa6abcafab1467912 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f366ff6c37a24eaaa6abcafab1467912; no voters: 08c70c9a6a064d40bfa68fd105d86d86, 1a9b5a4afd63410b813b41edbfe69e28
I20250109 08:58:53.099512 24923 raft_consensus.cc:2743] T d8c90027b99f41b78f3b41c079e77ed7 P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:58:53.099685 24661 leader_election.cc:304] T e8d95383d13941a99003d8e54fbed202 P f366ff6c37a24eaaa6abcafab1467912 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 08c70c9a6a064d40bfa68fd105d86d86, f366ff6c37a24eaaa6abcafab1467912; no voters: 1a9b5a4afd63410b813b41edbfe69e28
I20250109 08:58:53.101308 24849 raft_consensus.cc:2798] T e8d95383d13941a99003d8e54fbed202 P f366ff6c37a24eaaa6abcafab1467912 [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:58:53.101763 24849 raft_consensus.cc:695] T e8d95383d13941a99003d8e54fbed202 P f366ff6c37a24eaaa6abcafab1467912 [term 1 LEADER]: Becoming Leader. State: Replica: f366ff6c37a24eaaa6abcafab1467912, State: Running, Role: LEADER
I20250109 08:58:53.102524 24849 consensus_queue.cc:237] T e8d95383d13941a99003d8e54fbed202 P f366ff6c37a24eaaa6abcafab1467912 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:53.105214 24909 tablet_bootstrap.cc:492] T 0faa532dc2514e6b9991e3832a029fab P 08c70c9a6a064d40bfa68fd105d86d86: No bootstrap required, opened a new log
I20250109 08:58:53.105793 24909 ts_tablet_manager.cc:1397] T 0faa532dc2514e6b9991e3832a029fab P 08c70c9a6a064d40bfa68fd105d86d86: Time spent bootstrapping tablet: real 0.045s user 0.011s sys 0.028s
I20250109 08:58:53.108888 24909 raft_consensus.cc:357] T 0faa532dc2514e6b9991e3832a029fab P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:53.109627 24909 raft_consensus.cc:383] T 0faa532dc2514e6b9991e3832a029fab P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:53.109897 24909 raft_consensus.cc:738] T 0faa532dc2514e6b9991e3832a029fab P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08c70c9a6a064d40bfa68fd105d86d86, State: Initialized, Role: FOLLOWER
I20250109 08:58:53.110682 24909 consensus_queue.cc:260] T 0faa532dc2514e6b9991e3832a029fab P 08c70c9a6a064d40bfa68fd105d86d86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:53.113159 24909 ts_tablet_manager.cc:1428] T 0faa532dc2514e6b9991e3832a029fab P 08c70c9a6a064d40bfa68fd105d86d86: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20250109 08:58:53.114200 24909 tablet_bootstrap.cc:492] T 53eef9db17b548f081a60a39a43e4f19 P 08c70c9a6a064d40bfa68fd105d86d86: Bootstrap starting.
I20250109 08:58:53.117944 24910 ts_tablet_manager.cc:1428] T 0faa532dc2514e6b9991e3832a029fab P 1a9b5a4afd63410b813b41edbfe69e28: Time spent starting tablet: real 0.040s user 0.013s sys 0.011s
I20250109 08:58:53.119410 24910 tablet_bootstrap.cc:492] T 53eef9db17b548f081a60a39a43e4f19 P 1a9b5a4afd63410b813b41edbfe69e28: Bootstrap starting.
I20250109 08:58:53.130973 24518 catalog_manager.cc:5526] T a3207340610143c7bfb4943c87dc78bb P 08c70c9a6a064d40bfa68fd105d86d86 reported cstate change: term changed from 0 to 1, leader changed from <none> to 08c70c9a6a064d40bfa68fd105d86d86 (127.19.226.193). New cstate: current_term: 1 leader_uuid: "08c70c9a6a064d40bfa68fd105d86d86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:53.134531 24926 raft_consensus.cc:491] T 0364f388e25b466b8b39fdaa50aebf42 P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:53.133472 24933 raft_consensus.cc:491] T 0364f388e25b466b8b39fdaa50aebf42 P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:53.135054 24926 raft_consensus.cc:513] T 0364f388e25b466b8b39fdaa50aebf42 P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:53.135429 24933 raft_consensus.cc:513] T 0364f388e25b466b8b39fdaa50aebf42 P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:53.137439 24926 leader_election.cc:290] T 0364f388e25b466b8b39fdaa50aebf42 P 08c70c9a6a064d40bfa68fd105d86d86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f366ff6c37a24eaaa6abcafab1467912 (127.19.226.194:43355), 1a9b5a4afd63410b813b41edbfe69e28 (127.19.226.195:42529)
I20250109 08:58:53.137694 24694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0364f388e25b466b8b39fdaa50aebf42" candidate_uuid: "08c70c9a6a064d40bfa68fd105d86d86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f366ff6c37a24eaaa6abcafab1467912" is_pre_election: true
I20250109 08:58:53.138350 24694 raft_consensus.cc:2463] T 0364f388e25b466b8b39fdaa50aebf42 P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 08c70c9a6a064d40bfa68fd105d86d86 in term 0.
I20250109 08:58:53.140190 24767 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0364f388e25b466b8b39fdaa50aebf42" candidate_uuid: "08c70c9a6a064d40bfa68fd105d86d86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a9b5a4afd63410b813b41edbfe69e28" is_pre_election: true
I20250109 08:58:53.139654 24515 catalog_manager.cc:5526] T e8d95383d13941a99003d8e54fbed202 P f366ff6c37a24eaaa6abcafab1467912 reported cstate change: term changed from 0 to 1, leader changed from <none> to f366ff6c37a24eaaa6abcafab1467912 (127.19.226.194). New cstate: current_term: 1 leader_uuid: "f366ff6c37a24eaaa6abcafab1467912" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:53.140841 24767 raft_consensus.cc:2463] T 0364f388e25b466b8b39fdaa50aebf42 P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 08c70c9a6a064d40bfa68fd105d86d86 in term 0.
I20250109 08:58:53.142235 24586 leader_election.cc:304] T 0364f388e25b466b8b39fdaa50aebf42 P 08c70c9a6a064d40bfa68fd105d86d86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08c70c9a6a064d40bfa68fd105d86d86, 1a9b5a4afd63410b813b41edbfe69e28; no voters:
I20250109 08:58:53.142973 24926 raft_consensus.cc:2798] T 0364f388e25b466b8b39fdaa50aebf42 P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:58:53.143360 24926 raft_consensus.cc:491] T 0364f388e25b466b8b39fdaa50aebf42 P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:58:53.143750 24926 raft_consensus.cc:3054] T 0364f388e25b466b8b39fdaa50aebf42 P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:53.145417 24616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0364f388e25b466b8b39fdaa50aebf42" candidate_uuid: "1a9b5a4afd63410b813b41edbfe69e28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08c70c9a6a064d40bfa68fd105d86d86" is_pre_election: true
I20250109 08:58:53.146903 24909 tablet_bootstrap.cc:654] T 53eef9db17b548f081a60a39a43e4f19 P 08c70c9a6a064d40bfa68fd105d86d86: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:53.148029 24694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0364f388e25b466b8b39fdaa50aebf42" candidate_uuid: "1a9b5a4afd63410b813b41edbfe69e28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f366ff6c37a24eaaa6abcafab1467912" is_pre_election: true
I20250109 08:58:53.148677 24694 raft_consensus.cc:2463] T 0364f388e25b466b8b39fdaa50aebf42 P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a9b5a4afd63410b813b41edbfe69e28 in term 0.
I20250109 08:58:53.150491 24732 leader_election.cc:304] T 0364f388e25b466b8b39fdaa50aebf42 P 1a9b5a4afd63410b813b41edbfe69e28 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a9b5a4afd63410b813b41edbfe69e28, f366ff6c37a24eaaa6abcafab1467912; no voters:
I20250109 08:58:53.151342 24895 raft_consensus.cc:2798] T 0364f388e25b466b8b39fdaa50aebf42 P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:58:53.151820 24895 raft_consensus.cc:491] T 0364f388e25b466b8b39fdaa50aebf42 P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:58:53.151346 24926 raft_consensus.cc:513] T 0364f388e25b466b8b39fdaa50aebf42 P 08c70c9a6a064d40bfa68fd105d86d86 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:53.152132 24895 raft_consensus.cc:3054] T 0364f388e25b466b8b39fdaa50aebf42 P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:53.152499 24910 tablet_bootstrap.cc:654] T 53eef9db17b548f081a60a39a43e4f19 P 1a9b5a4afd63410b813b41edbfe69e28: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:53.153925 24926 leader_election.cc:290] T 0364f388e25b466b8b39fdaa50aebf42 P 08c70c9a6a064d40bfa68fd105d86d86 [CANDIDATE]: Term 1 election: Requested vote from peers f366ff6c37a24eaaa6abcafab1467912 (127.19.226.194:43355), 1a9b5a4afd63410b813b41edbfe69e28 (127.19.226.195:42529)
I20250109 08:58:53.158095 24694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0364f388e25b466b8b39fdaa50aebf42" candidate_uuid: "08c70c9a6a064d40bfa68fd105d86d86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f366ff6c37a24eaaa6abcafab1467912"
I20250109 08:58:53.158520 24910 tablet_bootstrap.cc:492] T 53eef9db17b548f081a60a39a43e4f19 P 1a9b5a4afd63410b813b41edbfe69e28: No bootstrap required, opened a new log
I20250109 08:58:53.158124 24616 raft_consensus.cc:2388] T 0364f388e25b466b8b39fdaa50aebf42 P 08c70c9a6a064d40bfa68fd105d86d86 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1a9b5a4afd63410b813b41edbfe69e28 in current term 1: Already voted for candidate 08c70c9a6a064d40bfa68fd105d86d86 in this term.
I20250109 08:58:53.159071 24910 ts_tablet_manager.cc:1397] T 53eef9db17b548f081a60a39a43e4f19 P 1a9b5a4afd63410b813b41edbfe69e28: Time spent bootstrapping tablet: real 0.040s user 0.011s sys 0.001s
I20250109 08:58:53.157300 24769 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0364f388e25b466b8b39fdaa50aebf42" candidate_uuid: "08c70c9a6a064d40bfa68fd105d86d86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a9b5a4afd63410b813b41edbfe69e28"
I20250109 08:58:53.157232 24895 raft_consensus.cc:513] T 0364f388e25b466b8b39fdaa50aebf42 P 1a9b5a4afd63410b813b41edbfe69e28 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:53.160974 24620 raft_consensus.cc:2463] T ff0a7cf17ff34eedbdda1e5a4fac0dfa P 08c70c9a6a064d40bfa68fd105d86d86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a9b5a4afd63410b813b41edbfe69e28 in term 1.
I20250109 08:58:53.163244 24616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0364f388e25b466b8b39fdaa50aebf42" candidate_uuid: "1a9b5a4afd63410b813b41edbfe69e28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08c70c9a6a064d40bfa68fd105d86d86"
I20250109 08:58:53.161309 24769 raft_consensus.cc:2388] T 0364f388e25b466b8b39fdaa50aebf42 P 1a9b5a4afd63410b813b41edbfe69e28 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 08c70c9a6a064d40bfa68fd105d86d86 in current term 1: Already voted for candidate 1a9b5a4afd63410b813b41edbfe69e28 in this term.
I20250109 08:58:53.163496 24910 raft_consensus.cc:357] T 53eef9db17b548f081a60a39a43e4f19 P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:53.159997 24933 leader_election.cc:290] T 0364f388e25b466b8b39fdaa50aebf42 P 1a9b5a4afd63410b813b41edbfe69e28 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 08c70c9a6a064d40bfa68fd105d86d86 (127.19.226.193:43567), f366ff6c37a24eaaa6abcafab1467912 (127.19.226.194:43355)
I20250109 08:58:53.158690 24694 raft_consensus.cc:3054] T 0364f388e25b466b8b39fdaa50aebf42 P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:53.164445 24910 raft_consensus.cc:383] T 53eef9db17b548f081a60a39a43e4f19 P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:53.165711 24910 raft_consensus.cc:738] T 53eef9db17b548f081a60a39a43e4f19 P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a9b5a4afd63410b813b41edbfe69e28, State: Initialized, Role: FOLLOWER
I20250109 08:58:53.166491 24910 consensus_queue.cc:260] T 53eef9db17b548f081a60a39a43e4f19 P 1a9b5a4afd63410b813b41edbfe69e28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:53.168715 24616 raft_consensus.cc:2388] T 0364f388e25b466b8b39fdaa50aebf42 P 08c70c9a6a064d40bfa68fd105d86d86 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 1a9b5a4afd63410b813b41edbfe69e28 in current term 1: Already voted for candidate 08c70c9a6a064d40bfa68fd105d86d86 in this term.
I20250109 08:58:53.171293 24849 raft_consensus.cc:491] T 9e0af0555b484b748a03f99b45c463ef P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:53.171455 24693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0364f388e25b466b8b39fdaa50aebf42" candidate_uuid: "1a9b5a4afd63410b813b41edbfe69e28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f366ff6c37a24eaaa6abcafab1467912"
I20250109 08:58:53.172966 24732 leader_election.cc:304] T 0364f388e25b466b8b39fdaa50aebf42 P 1a9b5a4afd63410b813b41edbfe69e28 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1a9b5a4afd63410b813b41edbfe69e28; no voters: 08c70c9a6a064d40bfa68fd105d86d86, f366ff6c37a24eaaa6abcafab1467912
I20250109 08:58:53.171808 24849 raft_consensus.cc:513] T 9e0af0555b484b748a03f99b45c463ef P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:53.173782 24933 raft_consensus.cc:2743] T 0364f388e25b466b8b39fdaa50aebf42 P 1a9b5a4afd63410b813b41edbfe69e28 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20250109 08:58:53.173485 24910 ts_tablet_manager.cc:1428] T 53eef9db17b548f081a60a39a43e4f19 P 1a9b5a4afd63410b813b41edbfe69e28: Time spent starting tablet: real 0.013s user 0.006s sys 0.001s
I20250109 08:58:53.175181 24849 leader_election.cc:290] T 9e0af0555b484b748a03f99b45c463ef P f366ff6c37a24eaaa6abcafab1467912 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 08c70c9a6a064d40bfa68fd105d86d86 (127.19.226.193:43567), 1a9b5a4afd63410b813b41edbfe69e28 (127.19.226.195:42529)
I20250109 08:58:53.175381 24910 tablet_bootstrap.cc:492] T e8d95383d13941a99003d8e54fbed202 P 1a9b5a4afd63410b813b41edbfe69e28: Bootstrap starting.
I20250109 08:58:53.179231 24616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e0af0555b484b748a03f99b45c463ef" candidate_uuid: "f366ff6c37a24eaaa6abcafab1467912" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08c70c9a6a064d40bfa68fd105d86d86" is_pre_election: true
I20250109 08:58:53.179883 24616 raft_consensus.cc:2463] T 9e0af0555b484b748a03f99b45c463ef P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f366ff6c37a24eaaa6abcafab1467912 in term 0.
I20250109 08:58:53.181686 24769 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e0af0555b484b748a03f99b45c463ef" candidate_uuid: "f366ff6c37a24eaaa6abcafab1467912" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a9b5a4afd63410b813b41edbfe69e28" is_pre_election: true
I20250109 08:58:53.182274 24769 raft_consensus.cc:2463] T 9e0af0555b484b748a03f99b45c463ef P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f366ff6c37a24eaaa6abcafab1467912 in term 0.
I20250109 08:58:53.183387 24661 leader_election.cc:304] T 9e0af0555b484b748a03f99b45c463ef P f366ff6c37a24eaaa6abcafab1467912 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a9b5a4afd63410b813b41edbfe69e28, f366ff6c37a24eaaa6abcafab1467912; no voters:
I20250109 08:58:53.183887 24910 tablet_bootstrap.cc:654] T e8d95383d13941a99003d8e54fbed202 P 1a9b5a4afd63410b813b41edbfe69e28: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:53.184087 24903 raft_consensus.cc:2798] T 9e0af0555b484b748a03f99b45c463ef P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:58:53.184531 24903 raft_consensus.cc:491] T 9e0af0555b484b748a03f99b45c463ef P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:58:53.184854 24903 raft_consensus.cc:3054] T 9e0af0555b484b748a03f99b45c463ef P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:53.188344 24735 leader_election.cc:304] T ff0a7cf17ff34eedbdda1e5a4fac0dfa P 1a9b5a4afd63410b813b41edbfe69e28 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 08c70c9a6a064d40bfa68fd105d86d86, 1a9b5a4afd63410b813b41edbfe69e28; no voters: f366ff6c37a24eaaa6abcafab1467912
I20250109 08:58:53.189226 24933 raft_consensus.cc:2798] T ff0a7cf17ff34eedbdda1e5a4fac0dfa P 1a9b5a4afd63410b813b41edbfe69e28 [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:58:53.191242 24933 raft_consensus.cc:695] T ff0a7cf17ff34eedbdda1e5a4fac0dfa P 1a9b5a4afd63410b813b41edbfe69e28 [term 1 LEADER]: Becoming Leader. State: Replica: 1a9b5a4afd63410b813b41edbfe69e28, State: Running, Role: LEADER
I20250109 08:58:53.191998 24903 raft_consensus.cc:513] T 9e0af0555b484b748a03f99b45c463ef P f366ff6c37a24eaaa6abcafab1467912 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:53.192119 24933 consensus_queue.cc:237] T ff0a7cf17ff34eedbdda1e5a4fac0dfa P 1a9b5a4afd63410b813b41edbfe69e28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:53.162938 24908 tablet_bootstrap.cc:492] T 2141d5ba289f4dd99a8b0c4c1d11c66a P f366ff6c37a24eaaa6abcafab1467912: No bootstrap required, opened a new log
I20250109 08:58:53.194896 24908 ts_tablet_manager.cc:1397] T 2141d5ba289f4dd99a8b0c4c1d11c66a P f366ff6c37a24eaaa6abcafab1467912: Time spent bootstrapping tablet: real 0.161s user 0.022s sys 0.034s
I20250109 08:58:53.194860 24620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e0af0555b484b748a03f99b45c463ef" candidate_uuid: "f366ff6c37a24eaaa6abcafab1467912" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08c70c9a6a064d40bfa68fd105d86d86"
I20250109 08:58:53.195533 24620 raft_consensus.cc:3054] T 9e0af0555b484b748a03f99b45c463ef P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:53.198185 24694 raft_consensus.cc:2463] T 0364f388e25b466b8b39fdaa50aebf42 P f366ff6c37a24eaaa6abcafab1467912 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 08c70c9a6a064d40bfa68fd105d86d86 in term 1.
I20250109 08:58:53.199604 24583 leader_election.cc:304] T 0364f388e25b466b8b39fdaa50aebf42 P 08c70c9a6a064d40bfa68fd105d86d86 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 08c70c9a6a064d40bfa68fd105d86d86, f366ff6c37a24eaaa6abcafab1467912; no voters: 1a9b5a4afd63410b813b41edbfe69e28
I20250109 08:58:53.192140 24895 leader_election.cc:290] T 0364f388e25b466b8b39fdaa50aebf42 P 1a9b5a4afd63410b813b41edbfe69e28 [CANDIDATE]: Term 1 election: Requested vote from peers 08c70c9a6a064d40bfa68fd105d86d86 (127.19.226.193:43567), f366ff6c37a24eaaa6abcafab1467912 (127.19.226.194:43355)
I20250109 08:58:53.200721 24926 raft_consensus.cc:2798] T 0364f388e25b466b8b39fdaa50aebf42 P 08c70c9a6a064d40bfa68fd105d86d86 [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:58:53.201185 24926 raft_consensus.cc:695] T 0364f388e25b466b8b39fdaa50aebf42 P 08c70c9a6a064d40bfa68fd105d86d86 [term 1 LEADER]: Becoming Leader. State: Replica: 08c70c9a6a064d40bfa68fd105d86d86, State: Running, Role: LEADER
I20250109 08:58:53.201362 24908 raft_consensus.cc:357] T 2141d5ba289f4dd99a8b0c4c1d11c66a P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:53.201973 24926 consensus_queue.cc:237] T 0364f388e25b466b8b39fdaa50aebf42 P 08c70c9a6a064d40bfa68fd105d86d86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:53.202505 24908 raft_consensus.cc:383] T 2141d5ba289f4dd99a8b0c4c1d11c66a P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:53.202958 24908 raft_consensus.cc:738] T 2141d5ba289f4dd99a8b0c4c1d11c66a P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f366ff6c37a24eaaa6abcafab1467912, State: Initialized, Role: FOLLOWER
I20250109 08:58:53.203133 24909 tablet_bootstrap.cc:492] T 53eef9db17b548f081a60a39a43e4f19 P 08c70c9a6a064d40bfa68fd105d86d86: No bootstrap required, opened a new log
I20250109 08:58:53.203658 24909 ts_tablet_manager.cc:1397] T 53eef9db17b548f081a60a39a43e4f19 P 08c70c9a6a064d40bfa68fd105d86d86: Time spent bootstrapping tablet: real 0.090s user 0.016s sys 0.022s
I20250109 08:58:53.205847 24903 leader_election.cc:290] T 9e0af0555b484b748a03f99b45c463ef P f366ff6c37a24eaaa6abcafab1467912 [CANDIDATE]: Term 1 election: Requested vote from peers 08c70c9a6a064d40bfa68fd105d86d86 (127.19.226.193:43567), 1a9b5a4afd63410b813b41edbfe69e28 (127.19.226.195:42529)
I20250109 08:58:53.207237 24769 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e0af0555b484b748a03f99b45c463ef" candidate_uuid: "f366ff6c37a24eaaa6abcafab1467912" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a9b5a4afd63410b813b41edbfe69e28"
I20250109 08:58:53.207881 24769 raft_consensus.cc:3054] T 9e0af0555b484b748a03f99b45c463ef P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:53.211081 24518 catalog_manager.cc:5526] T ff0a7cf17ff34eedbdda1e5a4fac0dfa P 1a9b5a4afd63410b813b41edbfe69e28 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1a9b5a4afd63410b813b41edbfe69e28 (127.19.226.195). New cstate: current_term: 1 leader_uuid: "1a9b5a4afd63410b813b41edbfe69e28" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } health_report { overall_health: HEALTHY } } }
I20250109 08:58:53.211109 24909 raft_consensus.cc:357] T 53eef9db17b548f081a60a39a43e4f19 P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:53.215219 24909 raft_consensus.cc:383] T 53eef9db17b548f081a60a39a43e4f19 P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:53.215541 24909 raft_consensus.cc:738] T 53eef9db17b548f081a60a39a43e4f19 P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08c70c9a6a064d40bfa68fd105d86d86, State: Initialized, Role: FOLLOWER
I20250109 08:58:53.216280 24909 consensus_queue.cc:260] T 53eef9db17b548f081a60a39a43e4f19 P 08c70c9a6a064d40bfa68fd105d86d86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:53.216775 24620 raft_consensus.cc:2463] T 9e0af0555b484b748a03f99b45c463ef P 08c70c9a6a064d40bfa68fd105d86d86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f366ff6c37a24eaaa6abcafab1467912 in term 1.
I20250109 08:58:53.217835 24661 leader_election.cc:304] T 9e0af0555b484b748a03f99b45c463ef P f366ff6c37a24eaaa6abcafab1467912 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08c70c9a6a064d40bfa68fd105d86d86, f366ff6c37a24eaaa6abcafab1467912; no voters:
I20250109 08:58:53.218653 24903 raft_consensus.cc:2798] T 9e0af0555b484b748a03f99b45c463ef P f366ff6c37a24eaaa6abcafab1467912 [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:58:53.218813 24909 ts_tablet_manager.cc:1428] T 53eef9db17b548f081a60a39a43e4f19 P 08c70c9a6a064d40bfa68fd105d86d86: Time spent starting tablet: real 0.015s user 0.000s sys 0.007s
I20250109 08:58:53.219668 24903 raft_consensus.cc:695] T 9e0af0555b484b748a03f99b45c463ef P f366ff6c37a24eaaa6abcafab1467912 [term 1 LEADER]: Becoming Leader. State: Replica: f366ff6c37a24eaaa6abcafab1467912, State: Running, Role: LEADER
I20250109 08:58:53.219889 24909 tablet_bootstrap.cc:492] T 2141d5ba289f4dd99a8b0c4c1d11c66a P 08c70c9a6a064d40bfa68fd105d86d86: Bootstrap starting.
I20250109 08:58:53.220526 24903 consensus_queue.cc:237] T 9e0af0555b484b748a03f99b45c463ef P f366ff6c37a24eaaa6abcafab1467912 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:53.214607 24769 raft_consensus.cc:2463] T 9e0af0555b484b748a03f99b45c463ef P 1a9b5a4afd63410b813b41edbfe69e28 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f366ff6c37a24eaaa6abcafab1467912 in term 1.
I20250109 08:58:53.227381 24909 tablet_bootstrap.cc:654] T 2141d5ba289f4dd99a8b0c4c1d11c66a P 08c70c9a6a064d40bfa68fd105d86d86: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:53.203934 24908 consensus_queue.cc:260] T 2141d5ba289f4dd99a8b0c4c1d11c66a P f366ff6c37a24eaaa6abcafab1467912 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:53.228449 24849 raft_consensus.cc:491] T 2141d5ba289f4dd99a8b0c4c1d11c66a P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:53.229820 24849 raft_consensus.cc:513] T 2141d5ba289f4dd99a8b0c4c1d11c66a P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:53.230216 24515 catalog_manager.cc:5526] T 0364f388e25b466b8b39fdaa50aebf42 P 08c70c9a6a064d40bfa68fd105d86d86 reported cstate change: term changed from 0 to 1, leader changed from <none> to 08c70c9a6a064d40bfa68fd105d86d86 (127.19.226.193). New cstate: current_term: 1 leader_uuid: "08c70c9a6a064d40bfa68fd105d86d86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:53.234939 24620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2141d5ba289f4dd99a8b0c4c1d11c66a" candidate_uuid: "f366ff6c37a24eaaa6abcafab1467912" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08c70c9a6a064d40bfa68fd105d86d86" is_pre_election: true
I20250109 08:58:53.235209 24910 tablet_bootstrap.cc:492] T e8d95383d13941a99003d8e54fbed202 P 1a9b5a4afd63410b813b41edbfe69e28: No bootstrap required, opened a new log
I20250109 08:58:53.235764 24910 ts_tablet_manager.cc:1397] T e8d95383d13941a99003d8e54fbed202 P 1a9b5a4afd63410b813b41edbfe69e28: Time spent bootstrapping tablet: real 0.061s user 0.022s sys 0.007s
I20250109 08:58:53.238731 24910 raft_consensus.cc:357] T e8d95383d13941a99003d8e54fbed202 P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:53.239470 24910 raft_consensus.cc:383] T e8d95383d13941a99003d8e54fbed202 P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:53.239742 24910 raft_consensus.cc:738] T e8d95383d13941a99003d8e54fbed202 P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a9b5a4afd63410b813b41edbfe69e28, State: Initialized, Role: FOLLOWER
I20250109 08:58:53.240423 24910 consensus_queue.cc:260] T e8d95383d13941a99003d8e54fbed202 P 1a9b5a4afd63410b813b41edbfe69e28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
W20250109 08:58:53.241155 24661 leader_election.cc:343] T 2141d5ba289f4dd99a8b0c4c1d11c66a P f366ff6c37a24eaaa6abcafab1467912 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 08c70c9a6a064d40bfa68fd105d86d86 (127.19.226.193:43567): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:53.243273 24910 ts_tablet_manager.cc:1428] T e8d95383d13941a99003d8e54fbed202 P 1a9b5a4afd63410b813b41edbfe69e28: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20250109 08:58:53.244331 24910 tablet_bootstrap.cc:492] T 2141d5ba289f4dd99a8b0c4c1d11c66a P 1a9b5a4afd63410b813b41edbfe69e28: Bootstrap starting.
I20250109 08:58:53.248044 24908 ts_tablet_manager.cc:1428] T 2141d5ba289f4dd99a8b0c4c1d11c66a P f366ff6c37a24eaaa6abcafab1467912: Time spent starting tablet: real 0.053s user 0.006s sys 0.000s
I20250109 08:58:53.249042 24908 tablet_bootstrap.cc:492] T a3207340610143c7bfb4943c87dc78bb P f366ff6c37a24eaaa6abcafab1467912: Bootstrap starting.
I20250109 08:58:53.250722 24910 tablet_bootstrap.cc:654] T 2141d5ba289f4dd99a8b0c4c1d11c66a P 1a9b5a4afd63410b813b41edbfe69e28: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:53.252692 24518 catalog_manager.cc:5526] T 9e0af0555b484b748a03f99b45c463ef P f366ff6c37a24eaaa6abcafab1467912 reported cstate change: term changed from 0 to 1, leader changed from <none> to f366ff6c37a24eaaa6abcafab1467912 (127.19.226.194). New cstate: current_term: 1 leader_uuid: "f366ff6c37a24eaaa6abcafab1467912" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:53.254226 24849 leader_election.cc:290] T 2141d5ba289f4dd99a8b0c4c1d11c66a P f366ff6c37a24eaaa6abcafab1467912 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 08c70c9a6a064d40bfa68fd105d86d86 (127.19.226.193:43567), 1a9b5a4afd63410b813b41edbfe69e28 (127.19.226.195:42529)
I20250109 08:58:53.255208 24769 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2141d5ba289f4dd99a8b0c4c1d11c66a" candidate_uuid: "f366ff6c37a24eaaa6abcafab1467912" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a9b5a4afd63410b813b41edbfe69e28" is_pre_election: true
W20250109 08:58:53.256390 24661 leader_election.cc:343] T 2141d5ba289f4dd99a8b0c4c1d11c66a P f366ff6c37a24eaaa6abcafab1467912 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1a9b5a4afd63410b813b41edbfe69e28 (127.19.226.195:42529): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:58:53.256798 24661 leader_election.cc:304] T 2141d5ba289f4dd99a8b0c4c1d11c66a P f366ff6c37a24eaaa6abcafab1467912 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f366ff6c37a24eaaa6abcafab1467912; no voters: 08c70c9a6a064d40bfa68fd105d86d86, 1a9b5a4afd63410b813b41edbfe69e28
I20250109 08:58:53.257805 24909 tablet_bootstrap.cc:492] T 2141d5ba289f4dd99a8b0c4c1d11c66a P 08c70c9a6a064d40bfa68fd105d86d86: No bootstrap required, opened a new log
I20250109 08:58:53.257838 24849 raft_consensus.cc:2743] T 2141d5ba289f4dd99a8b0c4c1d11c66a P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:58:53.258453 24909 ts_tablet_manager.cc:1397] T 2141d5ba289f4dd99a8b0c4c1d11c66a P 08c70c9a6a064d40bfa68fd105d86d86: Time spent bootstrapping tablet: real 0.039s user 0.007s sys 0.026s
I20250109 08:58:53.261180 24909 raft_consensus.cc:357] T 2141d5ba289f4dd99a8b0c4c1d11c66a P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:53.262127 24909 raft_consensus.cc:383] T 2141d5ba289f4dd99a8b0c4c1d11c66a P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:53.262379 24909 raft_consensus.cc:738] T 2141d5ba289f4dd99a8b0c4c1d11c66a P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08c70c9a6a064d40bfa68fd105d86d86, State: Initialized, Role: FOLLOWER
I20250109 08:58:53.262964 24908 tablet_bootstrap.cc:654] T a3207340610143c7bfb4943c87dc78bb P f366ff6c37a24eaaa6abcafab1467912: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:53.262970 24909 consensus_queue.cc:260] T 2141d5ba289f4dd99a8b0c4c1d11c66a P 08c70c9a6a064d40bfa68fd105d86d86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:53.283047 24910 tablet_bootstrap.cc:492] T 2141d5ba289f4dd99a8b0c4c1d11c66a P 1a9b5a4afd63410b813b41edbfe69e28: No bootstrap required, opened a new log
I20250109 08:58:53.283546 24910 ts_tablet_manager.cc:1397] T 2141d5ba289f4dd99a8b0c4c1d11c66a P 1a9b5a4afd63410b813b41edbfe69e28: Time spent bootstrapping tablet: real 0.039s user 0.014s sys 0.022s
I20250109 08:58:53.286052 24909 ts_tablet_manager.cc:1428] T 2141d5ba289f4dd99a8b0c4c1d11c66a P 08c70c9a6a064d40bfa68fd105d86d86: Time spent starting tablet: real 0.027s user 0.014s sys 0.006s
I20250109 08:58:53.286017 24910 raft_consensus.cc:357] T 2141d5ba289f4dd99a8b0c4c1d11c66a P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:53.286825 24910 raft_consensus.cc:383] T 2141d5ba289f4dd99a8b0c4c1d11c66a P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:53.287102 24910 raft_consensus.cc:738] T 2141d5ba289f4dd99a8b0c4c1d11c66a P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a9b5a4afd63410b813b41edbfe69e28, State: Initialized, Role: FOLLOWER
I20250109 08:58:53.287256 24908 tablet_bootstrap.cc:492] T a3207340610143c7bfb4943c87dc78bb P f366ff6c37a24eaaa6abcafab1467912: No bootstrap required, opened a new log
I20250109 08:58:53.288236 24908 ts_tablet_manager.cc:1397] T a3207340610143c7bfb4943c87dc78bb P f366ff6c37a24eaaa6abcafab1467912: Time spent bootstrapping tablet: real 0.039s user 0.007s sys 0.007s
I20250109 08:58:53.287830 24910 consensus_queue.cc:260] T 2141d5ba289f4dd99a8b0c4c1d11c66a P 1a9b5a4afd63410b813b41edbfe69e28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:53.290374 24910 ts_tablet_manager.cc:1428] T 2141d5ba289f4dd99a8b0c4c1d11c66a P 1a9b5a4afd63410b813b41edbfe69e28: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20250109 08:58:53.291249 24910 tablet_bootstrap.cc:492] T d8c90027b99f41b78f3b41c079e77ed7 P 1a9b5a4afd63410b813b41edbfe69e28: Bootstrap starting.
I20250109 08:58:53.290995 24908 raft_consensus.cc:357] T a3207340610143c7bfb4943c87dc78bb P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:53.291783 24908 raft_consensus.cc:383] T a3207340610143c7bfb4943c87dc78bb P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:53.292064 24908 raft_consensus.cc:738] T a3207340610143c7bfb4943c87dc78bb P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f366ff6c37a24eaaa6abcafab1467912, State: Initialized, Role: FOLLOWER
I20250109 08:58:53.292724 24908 consensus_queue.cc:260] T a3207340610143c7bfb4943c87dc78bb P f366ff6c37a24eaaa6abcafab1467912 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:53.296082 24909 tablet_bootstrap.cc:492] T d8c90027b99f41b78f3b41c079e77ed7 P 08c70c9a6a064d40bfa68fd105d86d86: Bootstrap starting.
I20250109 08:58:53.296747 24910 tablet_bootstrap.cc:654] T d8c90027b99f41b78f3b41c079e77ed7 P 1a9b5a4afd63410b813b41edbfe69e28: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:53.302403 24909 tablet_bootstrap.cc:654] T d8c90027b99f41b78f3b41c079e77ed7 P 08c70c9a6a064d40bfa68fd105d86d86: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:53.299906 24908 ts_tablet_manager.cc:1428] T a3207340610143c7bfb4943c87dc78bb P f366ff6c37a24eaaa6abcafab1467912: Time spent starting tablet: real 0.011s user 0.005s sys 0.000s
I20250109 08:58:53.306761 24908 tablet_bootstrap.cc:492] T 0faa532dc2514e6b9991e3832a029fab P f366ff6c37a24eaaa6abcafab1467912: Bootstrap starting.
I20250109 08:58:53.308182 24910 tablet_bootstrap.cc:492] T d8c90027b99f41b78f3b41c079e77ed7 P 1a9b5a4afd63410b813b41edbfe69e28: No bootstrap required, opened a new log
I20250109 08:58:53.308744 24910 ts_tablet_manager.cc:1397] T d8c90027b99f41b78f3b41c079e77ed7 P 1a9b5a4afd63410b813b41edbfe69e28: Time spent bootstrapping tablet: real 0.018s user 0.009s sys 0.007s
I20250109 08:58:53.311355 24910 raft_consensus.cc:357] T d8c90027b99f41b78f3b41c079e77ed7 P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:53.312075 24910 raft_consensus.cc:383] T d8c90027b99f41b78f3b41c079e77ed7 P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:53.312345 24910 raft_consensus.cc:738] T d8c90027b99f41b78f3b41c079e77ed7 P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a9b5a4afd63410b813b41edbfe69e28, State: Initialized, Role: FOLLOWER
I20250109 08:58:53.312796 24908 tablet_bootstrap.cc:654] T 0faa532dc2514e6b9991e3832a029fab P f366ff6c37a24eaaa6abcafab1467912: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:53.313233 24910 consensus_queue.cc:260] T d8c90027b99f41b78f3b41c079e77ed7 P 1a9b5a4afd63410b813b41edbfe69e28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:53.315887 24910 ts_tablet_manager.cc:1428] T d8c90027b99f41b78f3b41c079e77ed7 P 1a9b5a4afd63410b813b41edbfe69e28: Time spent starting tablet: real 0.007s user 0.002s sys 0.004s
I20250109 08:58:53.318513 24909 tablet_bootstrap.cc:492] T d8c90027b99f41b78f3b41c079e77ed7 P 08c70c9a6a064d40bfa68fd105d86d86: No bootstrap required, opened a new log
I20250109 08:58:53.318902 24909 ts_tablet_manager.cc:1397] T d8c90027b99f41b78f3b41c079e77ed7 P 08c70c9a6a064d40bfa68fd105d86d86: Time spent bootstrapping tablet: real 0.023s user 0.010s sys 0.010s
I20250109 08:58:53.319641 24908 tablet_bootstrap.cc:492] T 0faa532dc2514e6b9991e3832a029fab P f366ff6c37a24eaaa6abcafab1467912: No bootstrap required, opened a new log
I20250109 08:58:53.320096 24908 ts_tablet_manager.cc:1397] T 0faa532dc2514e6b9991e3832a029fab P f366ff6c37a24eaaa6abcafab1467912: Time spent bootstrapping tablet: real 0.014s user 0.011s sys 0.000s
I20250109 08:58:53.320834 24909 raft_consensus.cc:357] T d8c90027b99f41b78f3b41c079e77ed7 P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:53.321357 24909 raft_consensus.cc:383] T d8c90027b99f41b78f3b41c079e77ed7 P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:53.321563 24909 raft_consensus.cc:738] T d8c90027b99f41b78f3b41c079e77ed7 P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08c70c9a6a064d40bfa68fd105d86d86, State: Initialized, Role: FOLLOWER
I20250109 08:58:53.322170 24909 consensus_queue.cc:260] T d8c90027b99f41b78f3b41c079e77ed7 P 08c70c9a6a064d40bfa68fd105d86d86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:53.322544 24908 raft_consensus.cc:357] T 0faa532dc2514e6b9991e3832a029fab P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:53.323314 24908 raft_consensus.cc:383] T 0faa532dc2514e6b9991e3832a029fab P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:53.323570 24908 raft_consensus.cc:738] T 0faa532dc2514e6b9991e3832a029fab P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f366ff6c37a24eaaa6abcafab1467912, State: Initialized, Role: FOLLOWER
I20250109 08:58:53.324172 24908 consensus_queue.cc:260] T 0faa532dc2514e6b9991e3832a029fab P f366ff6c37a24eaaa6abcafab1467912 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:53.325701 24909 ts_tablet_manager.cc:1428] T d8c90027b99f41b78f3b41c079e77ed7 P 08c70c9a6a064d40bfa68fd105d86d86: Time spent starting tablet: real 0.007s user 0.005s sys 0.001s
I20250109 08:58:53.326268 24908 ts_tablet_manager.cc:1428] T 0faa532dc2514e6b9991e3832a029fab P f366ff6c37a24eaaa6abcafab1467912: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
I20250109 08:58:53.327316 24908 tablet_bootstrap.cc:492] T ff0a7cf17ff34eedbdda1e5a4fac0dfa P f366ff6c37a24eaaa6abcafab1467912: Bootstrap starting.
I20250109 08:58:53.335315 24908 tablet_bootstrap.cc:654] T ff0a7cf17ff34eedbdda1e5a4fac0dfa P f366ff6c37a24eaaa6abcafab1467912: Neither blocks nor log segments found. Creating new log.
I20250109 08:58:53.339145 24908 tablet_bootstrap.cc:492] T ff0a7cf17ff34eedbdda1e5a4fac0dfa P f366ff6c37a24eaaa6abcafab1467912: No bootstrap required, opened a new log
I20250109 08:58:53.339506 24908 ts_tablet_manager.cc:1397] T ff0a7cf17ff34eedbdda1e5a4fac0dfa P f366ff6c37a24eaaa6abcafab1467912: Time spent bootstrapping tablet: real 0.012s user 0.008s sys 0.000s
I20250109 08:58:53.341351 24908 raft_consensus.cc:357] T ff0a7cf17ff34eedbdda1e5a4fac0dfa P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:53.341861 24908 raft_consensus.cc:383] T ff0a7cf17ff34eedbdda1e5a4fac0dfa P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:58:53.342046 24908 raft_consensus.cc:738] T ff0a7cf17ff34eedbdda1e5a4fac0dfa P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f366ff6c37a24eaaa6abcafab1467912, State: Initialized, Role: FOLLOWER
I20250109 08:58:53.342515 24908 consensus_queue.cc:260] T ff0a7cf17ff34eedbdda1e5a4fac0dfa P f366ff6c37a24eaaa6abcafab1467912 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:53.343992 24908 ts_tablet_manager.cc:1428] T ff0a7cf17ff34eedbdda1e5a4fac0dfa P f366ff6c37a24eaaa6abcafab1467912: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20250109 08:58:53.366185 24926 raft_consensus.cc:491] T 0faa532dc2514e6b9991e3832a029fab P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:53.366631 24926 raft_consensus.cc:513] T 0faa532dc2514e6b9991e3832a029fab P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:53.368288 24926 leader_election.cc:290] T 0faa532dc2514e6b9991e3832a029fab P 08c70c9a6a064d40bfa68fd105d86d86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f366ff6c37a24eaaa6abcafab1467912 (127.19.226.194:43355), 1a9b5a4afd63410b813b41edbfe69e28 (127.19.226.195:42529)
I20250109 08:58:53.369005 24694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0faa532dc2514e6b9991e3832a029fab" candidate_uuid: "08c70c9a6a064d40bfa68fd105d86d86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f366ff6c37a24eaaa6abcafab1467912" is_pre_election: true
I20250109 08:58:53.369246 24769 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0faa532dc2514e6b9991e3832a029fab" candidate_uuid: "08c70c9a6a064d40bfa68fd105d86d86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a9b5a4afd63410b813b41edbfe69e28" is_pre_election: true
I20250109 08:58:53.369597 24694 raft_consensus.cc:2463] T 0faa532dc2514e6b9991e3832a029fab P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 08c70c9a6a064d40bfa68fd105d86d86 in term 0.
I20250109 08:58:53.369812 24769 raft_consensus.cc:2463] T 0faa532dc2514e6b9991e3832a029fab P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 08c70c9a6a064d40bfa68fd105d86d86 in term 0.
I20250109 08:58:53.370481 24583 leader_election.cc:304] T 0faa532dc2514e6b9991e3832a029fab P 08c70c9a6a064d40bfa68fd105d86d86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08c70c9a6a064d40bfa68fd105d86d86, f366ff6c37a24eaaa6abcafab1467912; no voters:
I20250109 08:58:53.371079 24926 raft_consensus.cc:2798] T 0faa532dc2514e6b9991e3832a029fab P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:58:53.371407 24926 raft_consensus.cc:491] T 0faa532dc2514e6b9991e3832a029fab P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:58:53.371634 24926 raft_consensus.cc:3054] T 0faa532dc2514e6b9991e3832a029fab P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:53.373539 24925 raft_consensus.cc:491] T 53eef9db17b548f081a60a39a43e4f19 P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:53.373946 24925 raft_consensus.cc:513] T 53eef9db17b548f081a60a39a43e4f19 P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:53.375465 24925 leader_election.cc:290] T 53eef9db17b548f081a60a39a43e4f19 P 08c70c9a6a064d40bfa68fd105d86d86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f366ff6c37a24eaaa6abcafab1467912 (127.19.226.194:43355), 1a9b5a4afd63410b813b41edbfe69e28 (127.19.226.195:42529)
I20250109 08:58:53.376331 24694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53eef9db17b548f081a60a39a43e4f19" candidate_uuid: "08c70c9a6a064d40bfa68fd105d86d86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f366ff6c37a24eaaa6abcafab1467912" is_pre_election: true
I20250109 08:58:53.376523 24769 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53eef9db17b548f081a60a39a43e4f19" candidate_uuid: "08c70c9a6a064d40bfa68fd105d86d86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a9b5a4afd63410b813b41edbfe69e28" is_pre_election: true
I20250109 08:58:53.376930 24694 raft_consensus.cc:2463] T 53eef9db17b548f081a60a39a43e4f19 P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 08c70c9a6a064d40bfa68fd105d86d86 in term 0.
I20250109 08:58:53.377189 24769 raft_consensus.cc:2463] T 53eef9db17b548f081a60a39a43e4f19 P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 08c70c9a6a064d40bfa68fd105d86d86 in term 0.
I20250109 08:58:53.377025 24926 raft_consensus.cc:513] T 0faa532dc2514e6b9991e3832a029fab P 08c70c9a6a064d40bfa68fd105d86d86 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:53.377813 24583 leader_election.cc:304] T 53eef9db17b548f081a60a39a43e4f19 P 08c70c9a6a064d40bfa68fd105d86d86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08c70c9a6a064d40bfa68fd105d86d86, f366ff6c37a24eaaa6abcafab1467912; no voters:
I20250109 08:58:53.378561 24925 raft_consensus.cc:2798] T 53eef9db17b548f081a60a39a43e4f19 P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:58:53.378927 24925 raft_consensus.cc:491] T 53eef9db17b548f081a60a39a43e4f19 P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:58:53.378933 24926 leader_election.cc:290] T 0faa532dc2514e6b9991e3832a029fab P 08c70c9a6a064d40bfa68fd105d86d86 [CANDIDATE]: Term 1 election: Requested vote from peers f366ff6c37a24eaaa6abcafab1467912 (127.19.226.194:43355), 1a9b5a4afd63410b813b41edbfe69e28 (127.19.226.195:42529)
I20250109 08:58:53.379366 24925 raft_consensus.cc:3054] T 53eef9db17b548f081a60a39a43e4f19 P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:53.379767 24694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0faa532dc2514e6b9991e3832a029fab" candidate_uuid: "08c70c9a6a064d40bfa68fd105d86d86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f366ff6c37a24eaaa6abcafab1467912"
I20250109 08:58:53.380190 24694 raft_consensus.cc:3054] T 0faa532dc2514e6b9991e3832a029fab P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:53.381603 24769 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0faa532dc2514e6b9991e3832a029fab" candidate_uuid: "08c70c9a6a064d40bfa68fd105d86d86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a9b5a4afd63410b813b41edbfe69e28"
I20250109 08:58:53.382052 24769 raft_consensus.cc:3054] T 0faa532dc2514e6b9991e3832a029fab P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:53.384829 24849 raft_consensus.cc:491] T ff0a7cf17ff34eedbdda1e5a4fac0dfa P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:53.384835 24694 raft_consensus.cc:2463] T 0faa532dc2514e6b9991e3832a029fab P f366ff6c37a24eaaa6abcafab1467912 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 08c70c9a6a064d40bfa68fd105d86d86 in term 1.
I20250109 08:58:53.385502 24849 raft_consensus.cc:513] T ff0a7cf17ff34eedbdda1e5a4fac0dfa P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:53.386220 24583 leader_election.cc:304] T 0faa532dc2514e6b9991e3832a029fab P 08c70c9a6a064d40bfa68fd105d86d86 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08c70c9a6a064d40bfa68fd105d86d86, f366ff6c37a24eaaa6abcafab1467912; no voters:
I20250109 08:58:53.386554 24925 raft_consensus.cc:513] T 53eef9db17b548f081a60a39a43e4f19 P 08c70c9a6a064d40bfa68fd105d86d86 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:53.387081 24926 raft_consensus.cc:2798] T 0faa532dc2514e6b9991e3832a029fab P 08c70c9a6a064d40bfa68fd105d86d86 [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:58:53.387475 24849 leader_election.cc:290] T ff0a7cf17ff34eedbdda1e5a4fac0dfa P f366ff6c37a24eaaa6abcafab1467912 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 08c70c9a6a064d40bfa68fd105d86d86 (127.19.226.193:43567), 1a9b5a4afd63410b813b41edbfe69e28 (127.19.226.195:42529)
I20250109 08:58:53.388511 24620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff0a7cf17ff34eedbdda1e5a4fac0dfa" candidate_uuid: "f366ff6c37a24eaaa6abcafab1467912" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08c70c9a6a064d40bfa68fd105d86d86" is_pre_election: true
I20250109 08:58:53.387574 24926 raft_consensus.cc:695] T 0faa532dc2514e6b9991e3832a029fab P 08c70c9a6a064d40bfa68fd105d86d86 [term 1 LEADER]: Becoming Leader. State: Replica: 08c70c9a6a064d40bfa68fd105d86d86, State: Running, Role: LEADER
I20250109 08:58:53.389544 24933 raft_consensus.cc:491] T d8c90027b99f41b78f3b41c079e77ed7 P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:53.390059 24933 raft_consensus.cc:513] T d8c90027b99f41b78f3b41c079e77ed7 P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:53.390302 24926 consensus_queue.cc:237] T 0faa532dc2514e6b9991e3832a029fab P 08c70c9a6a064d40bfa68fd105d86d86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:53.390887 24769 raft_consensus.cc:2463] T 0faa532dc2514e6b9991e3832a029fab P 1a9b5a4afd63410b813b41edbfe69e28 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 08c70c9a6a064d40bfa68fd105d86d86 in term 1.
I20250109 08:58:53.391893 24933 leader_election.cc:290] T d8c90027b99f41b78f3b41c079e77ed7 P 1a9b5a4afd63410b813b41edbfe69e28 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 08c70c9a6a064d40bfa68fd105d86d86 (127.19.226.193:43567), f366ff6c37a24eaaa6abcafab1467912 (127.19.226.194:43355)
I20250109 08:58:53.392511 24925 leader_election.cc:290] T 53eef9db17b548f081a60a39a43e4f19 P 08c70c9a6a064d40bfa68fd105d86d86 [CANDIDATE]: Term 1 election: Requested vote from peers f366ff6c37a24eaaa6abcafab1467912 (127.19.226.194:43355), 1a9b5a4afd63410b813b41edbfe69e28 (127.19.226.195:42529)
I20250109 08:58:53.392746 24616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8c90027b99f41b78f3b41c079e77ed7" candidate_uuid: "1a9b5a4afd63410b813b41edbfe69e28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08c70c9a6a064d40bfa68fd105d86d86" is_pre_election: true
I20250109 08:58:53.389513 24694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53eef9db17b548f081a60a39a43e4f19" candidate_uuid: "08c70c9a6a064d40bfa68fd105d86d86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f366ff6c37a24eaaa6abcafab1467912"
I20250109 08:58:53.392903 24693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8c90027b99f41b78f3b41c079e77ed7" candidate_uuid: "1a9b5a4afd63410b813b41edbfe69e28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f366ff6c37a24eaaa6abcafab1467912" is_pre_election: true
I20250109 08:58:53.393421 24616 raft_consensus.cc:2463] T d8c90027b99f41b78f3b41c079e77ed7 P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a9b5a4afd63410b813b41edbfe69e28 in term 0.
I20250109 08:58:53.393636 24694 raft_consensus.cc:3054] T 53eef9db17b548f081a60a39a43e4f19 P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:53.393791 24693 raft_consensus.cc:2463] T d8c90027b99f41b78f3b41c079e77ed7 P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a9b5a4afd63410b813b41edbfe69e28 in term 0.
I20250109 08:58:53.394610 24735 leader_election.cc:304] T d8c90027b99f41b78f3b41c079e77ed7 P 1a9b5a4afd63410b813b41edbfe69e28 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08c70c9a6a064d40bfa68fd105d86d86, 1a9b5a4afd63410b813b41edbfe69e28; no voters:
I20250109 08:58:53.395706 24933 raft_consensus.cc:2798] T d8c90027b99f41b78f3b41c079e77ed7 P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:58:53.396062 24933 raft_consensus.cc:491] T d8c90027b99f41b78f3b41c079e77ed7 P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:58:53.389302 24620 raft_consensus.cc:2388] T ff0a7cf17ff34eedbdda1e5a4fac0dfa P 08c70c9a6a064d40bfa68fd105d86d86 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f366ff6c37a24eaaa6abcafab1467912 in current term 1: Already voted for candidate 1a9b5a4afd63410b813b41edbfe69e28 in this term.
I20250109 08:58:53.396411 24933 raft_consensus.cc:3054] T d8c90027b99f41b78f3b41c079e77ed7 P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:53.389814 24767 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff0a7cf17ff34eedbdda1e5a4fac0dfa" candidate_uuid: "f366ff6c37a24eaaa6abcafab1467912" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a9b5a4afd63410b813b41edbfe69e28" is_pre_election: true
I20250109 08:58:53.399219 24767 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53eef9db17b548f081a60a39a43e4f19" candidate_uuid: "08c70c9a6a064d40bfa68fd105d86d86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a9b5a4afd63410b813b41edbfe69e28"
I20250109 08:58:53.399820 24767 raft_consensus.cc:3054] T 53eef9db17b548f081a60a39a43e4f19 P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:53.400453 24694 raft_consensus.cc:2463] T 53eef9db17b548f081a60a39a43e4f19 P f366ff6c37a24eaaa6abcafab1467912 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 08c70c9a6a064d40bfa68fd105d86d86 in term 1.
I20250109 08:58:53.402781 24583 leader_election.cc:304] T 53eef9db17b548f081a60a39a43e4f19 P 08c70c9a6a064d40bfa68fd105d86d86 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08c70c9a6a064d40bfa68fd105d86d86, f366ff6c37a24eaaa6abcafab1467912; no voters:
I20250109 08:58:53.403578 24925 raft_consensus.cc:2798] T 53eef9db17b548f081a60a39a43e4f19 P 08c70c9a6a064d40bfa68fd105d86d86 [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:58:53.404014 24925 raft_consensus.cc:695] T 53eef9db17b548f081a60a39a43e4f19 P 08c70c9a6a064d40bfa68fd105d86d86 [term 1 LEADER]: Becoming Leader. State: Replica: 08c70c9a6a064d40bfa68fd105d86d86, State: Running, Role: LEADER
I20250109 08:58:53.405155 24661 leader_election.cc:304] T ff0a7cf17ff34eedbdda1e5a4fac0dfa P f366ff6c37a24eaaa6abcafab1467912 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f366ff6c37a24eaaa6abcafab1467912; no voters: 08c70c9a6a064d40bfa68fd105d86d86, 1a9b5a4afd63410b813b41edbfe69e28
I20250109 08:58:53.405063 24925 consensus_queue.cc:237] T 53eef9db17b548f081a60a39a43e4f19 P 08c70c9a6a064d40bfa68fd105d86d86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:53.406320 24849 raft_consensus.cc:3054] T ff0a7cf17ff34eedbdda1e5a4fac0dfa P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:53.407582 24767 raft_consensus.cc:2463] T 53eef9db17b548f081a60a39a43e4f19 P 1a9b5a4afd63410b813b41edbfe69e28 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 08c70c9a6a064d40bfa68fd105d86d86 in term 1.
I20250109 08:58:53.409727 24933 raft_consensus.cc:513] T d8c90027b99f41b78f3b41c079e77ed7 P 1a9b5a4afd63410b813b41edbfe69e28 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:53.413578 24694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8c90027b99f41b78f3b41c079e77ed7" candidate_uuid: "1a9b5a4afd63410b813b41edbfe69e28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f366ff6c37a24eaaa6abcafab1467912"
I20250109 08:58:53.414215 24620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8c90027b99f41b78f3b41c079e77ed7" candidate_uuid: "1a9b5a4afd63410b813b41edbfe69e28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08c70c9a6a064d40bfa68fd105d86d86"
I20250109 08:58:53.414369 24933 leader_election.cc:290] T d8c90027b99f41b78f3b41c079e77ed7 P 1a9b5a4afd63410b813b41edbfe69e28 [CANDIDATE]: Term 1 election: Requested vote from peers 08c70c9a6a064d40bfa68fd105d86d86 (127.19.226.193:43567), f366ff6c37a24eaaa6abcafab1467912 (127.19.226.194:43355)
I20250109 08:58:53.415027 24620 raft_consensus.cc:3054] T d8c90027b99f41b78f3b41c079e77ed7 P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:53.414291 24694 raft_consensus.cc:3054] T d8c90027b99f41b78f3b41c079e77ed7 P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:53.416910 24849 raft_consensus.cc:2743] T ff0a7cf17ff34eedbdda1e5a4fac0dfa P f366ff6c37a24eaaa6abcafab1467912 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:58:53.422324 24620 raft_consensus.cc:2463] T d8c90027b99f41b78f3b41c079e77ed7 P 08c70c9a6a064d40bfa68fd105d86d86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a9b5a4afd63410b813b41edbfe69e28 in term 1.
I20250109 08:58:53.423241 24735 leader_election.cc:304] T d8c90027b99f41b78f3b41c079e77ed7 P 1a9b5a4afd63410b813b41edbfe69e28 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08c70c9a6a064d40bfa68fd105d86d86, 1a9b5a4afd63410b813b41edbfe69e28; no voters:
I20250109 08:58:53.422739 24517 catalog_manager.cc:5526] T 0faa532dc2514e6b9991e3832a029fab P 08c70c9a6a064d40bfa68fd105d86d86 reported cstate change: term changed from 0 to 1, leader changed from <none> to 08c70c9a6a064d40bfa68fd105d86d86 (127.19.226.193). New cstate: current_term: 1 leader_uuid: "08c70c9a6a064d40bfa68fd105d86d86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:53.423903 24933 raft_consensus.cc:2798] T d8c90027b99f41b78f3b41c079e77ed7 P 1a9b5a4afd63410b813b41edbfe69e28 [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:58:53.424382 24933 raft_consensus.cc:695] T d8c90027b99f41b78f3b41c079e77ed7 P 1a9b5a4afd63410b813b41edbfe69e28 [term 1 LEADER]: Becoming Leader. State: Replica: 1a9b5a4afd63410b813b41edbfe69e28, State: Running, Role: LEADER
I20250109 08:58:53.425139 24933 consensus_queue.cc:237] T d8c90027b99f41b78f3b41c079e77ed7 P 1a9b5a4afd63410b813b41edbfe69e28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:53.427161 24694 raft_consensus.cc:2463] T d8c90027b99f41b78f3b41c079e77ed7 P f366ff6c37a24eaaa6abcafab1467912 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a9b5a4afd63410b813b41edbfe69e28 in term 1.
I20250109 08:58:53.434587 24515 catalog_manager.cc:5526] T d8c90027b99f41b78f3b41c079e77ed7 P 1a9b5a4afd63410b813b41edbfe69e28 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1a9b5a4afd63410b813b41edbfe69e28 (127.19.226.195). New cstate: current_term: 1 leader_uuid: "1a9b5a4afd63410b813b41edbfe69e28" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } health_report { overall_health: HEALTHY } } }
I20250109 08:58:53.440061 24517 catalog_manager.cc:5526] T 53eef9db17b548f081a60a39a43e4f19 P 08c70c9a6a064d40bfa68fd105d86d86 reported cstate change: term changed from 0 to 1, leader changed from <none> to 08c70c9a6a064d40bfa68fd105d86d86 (127.19.226.193). New cstate: current_term: 1 leader_uuid: "08c70c9a6a064d40bfa68fd105d86d86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:53.451534 24933 raft_consensus.cc:491] T e8d95383d13941a99003d8e54fbed202 P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:53.451982 24933 raft_consensus.cc:513] T e8d95383d13941a99003d8e54fbed202 P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:53.453832 24933 leader_election.cc:290] T e8d95383d13941a99003d8e54fbed202 P 1a9b5a4afd63410b813b41edbfe69e28 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 08c70c9a6a064d40bfa68fd105d86d86 (127.19.226.193:43567), f366ff6c37a24eaaa6abcafab1467912 (127.19.226.194:43355)
I20250109 08:58:53.454461 24620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8d95383d13941a99003d8e54fbed202" candidate_uuid: "1a9b5a4afd63410b813b41edbfe69e28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08c70c9a6a064d40bfa68fd105d86d86" is_pre_election: true
I20250109 08:58:53.454697 24694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8d95383d13941a99003d8e54fbed202" candidate_uuid: "1a9b5a4afd63410b813b41edbfe69e28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f366ff6c37a24eaaa6abcafab1467912" is_pre_election: true
I20250109 08:58:53.455127 24620 raft_consensus.cc:2388] T e8d95383d13941a99003d8e54fbed202 P 08c70c9a6a064d40bfa68fd105d86d86 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1a9b5a4afd63410b813b41edbfe69e28 in current term 1: Already voted for candidate f366ff6c37a24eaaa6abcafab1467912 in this term.
I20250109 08:58:53.455955 24735 leader_election.cc:304] T e8d95383d13941a99003d8e54fbed202 P 1a9b5a4afd63410b813b41edbfe69e28 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1a9b5a4afd63410b813b41edbfe69e28; no voters: 08c70c9a6a064d40bfa68fd105d86d86, f366ff6c37a24eaaa6abcafab1467912
I20250109 08:58:53.456573 24933 raft_consensus.cc:3054] T e8d95383d13941a99003d8e54fbed202 P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:53.461473 24933 raft_consensus.cc:2743] T e8d95383d13941a99003d8e54fbed202 P 1a9b5a4afd63410b813b41edbfe69e28 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:58:53.488910 24926 consensus_queue.cc:1035] T a3207340610143c7bfb4943c87dc78bb P 08c70c9a6a064d40bfa68fd105d86d86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:58:53.497159 24694 raft_consensus.cc:3054] T a3207340610143c7bfb4943c87dc78bb P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:53.505934 24926 consensus_queue.cc:1035] T a3207340610143c7bfb4943c87dc78bb P 08c70c9a6a064d40bfa68fd105d86d86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250109 08:58:53.538455 24849 consensus_queue.cc:1035] T e8d95383d13941a99003d8e54fbed202 P f366ff6c37a24eaaa6abcafab1467912 [LEADER]: Connected to new peer: Peer: permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:58:53.547446 24849 consensus_queue.cc:1035] T e8d95383d13941a99003d8e54fbed202 P f366ff6c37a24eaaa6abcafab1467912 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:58:53.603420 24842 consensus_queue.cc:1035] T 0364f388e25b466b8b39fdaa50aebf42 P 08c70c9a6a064d40bfa68fd105d86d86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:58:53.612519 24842 consensus_queue.cc:1035] T 0364f388e25b466b8b39fdaa50aebf42 P 08c70c9a6a064d40bfa68fd105d86d86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:58:53.615955 24933 consensus_queue.cc:1035] T ff0a7cf17ff34eedbdda1e5a4fac0dfa P 1a9b5a4afd63410b813b41edbfe69e28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:58:53.628954 24933 consensus_queue.cc:1035] T ff0a7cf17ff34eedbdda1e5a4fac0dfa P 1a9b5a4afd63410b813b41edbfe69e28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:58:53.656997 24903 consensus_queue.cc:1035] T 9e0af0555b484b748a03f99b45c463ef P f366ff6c37a24eaaa6abcafab1467912 [LEADER]: Connected to new peer: Peer: permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:58:53.666838 24903 consensus_queue.cc:1035] T 9e0af0555b484b748a03f99b45c463ef P f366ff6c37a24eaaa6abcafab1467912 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:58:53.707612 24842 raft_consensus.cc:491] T 2141d5ba289f4dd99a8b0c4c1d11c66a P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:58:53.708166 24842 raft_consensus.cc:513] T 2141d5ba289f4dd99a8b0c4c1d11c66a P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:53.709899 24842 leader_election.cc:290] T 2141d5ba289f4dd99a8b0c4c1d11c66a P 08c70c9a6a064d40bfa68fd105d86d86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f366ff6c37a24eaaa6abcafab1467912 (127.19.226.194:43355), 1a9b5a4afd63410b813b41edbfe69e28 (127.19.226.195:42529)
I20250109 08:58:53.710741 24694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2141d5ba289f4dd99a8b0c4c1d11c66a" candidate_uuid: "08c70c9a6a064d40bfa68fd105d86d86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f366ff6c37a24eaaa6abcafab1467912" is_pre_election: true
I20250109 08:58:53.710865 24769 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2141d5ba289f4dd99a8b0c4c1d11c66a" candidate_uuid: "08c70c9a6a064d40bfa68fd105d86d86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a9b5a4afd63410b813b41edbfe69e28" is_pre_election: true
I20250109 08:58:53.711328 24694 raft_consensus.cc:2463] T 2141d5ba289f4dd99a8b0c4c1d11c66a P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 08c70c9a6a064d40bfa68fd105d86d86 in term 0.
I20250109 08:58:53.711439 24769 raft_consensus.cc:2463] T 2141d5ba289f4dd99a8b0c4c1d11c66a P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 08c70c9a6a064d40bfa68fd105d86d86 in term 0.
I20250109 08:58:53.712201 24583 leader_election.cc:304] T 2141d5ba289f4dd99a8b0c4c1d11c66a P 08c70c9a6a064d40bfa68fd105d86d86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08c70c9a6a064d40bfa68fd105d86d86, f366ff6c37a24eaaa6abcafab1467912; no voters:
I20250109 08:58:53.712916 24842 raft_consensus.cc:2798] T 2141d5ba289f4dd99a8b0c4c1d11c66a P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:58:53.713246 24842 raft_consensus.cc:491] T 2141d5ba289f4dd99a8b0c4c1d11c66a P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:58:53.713496 24842 raft_consensus.cc:3054] T 2141d5ba289f4dd99a8b0c4c1d11c66a P 08c70c9a6a064d40bfa68fd105d86d86 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:53.717949 24842 raft_consensus.cc:513] T 2141d5ba289f4dd99a8b0c4c1d11c66a P 08c70c9a6a064d40bfa68fd105d86d86 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:53.719491 24842 leader_election.cc:290] T 2141d5ba289f4dd99a8b0c4c1d11c66a P 08c70c9a6a064d40bfa68fd105d86d86 [CANDIDATE]: Term 1 election: Requested vote from peers f366ff6c37a24eaaa6abcafab1467912 (127.19.226.194:43355), 1a9b5a4afd63410b813b41edbfe69e28 (127.19.226.195:42529)
I20250109 08:58:53.720152 24694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2141d5ba289f4dd99a8b0c4c1d11c66a" candidate_uuid: "08c70c9a6a064d40bfa68fd105d86d86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f366ff6c37a24eaaa6abcafab1467912"
I20250109 08:58:53.720356 24769 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2141d5ba289f4dd99a8b0c4c1d11c66a" candidate_uuid: "08c70c9a6a064d40bfa68fd105d86d86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a9b5a4afd63410b813b41edbfe69e28"
I20250109 08:58:53.720752 24694 raft_consensus.cc:3054] T 2141d5ba289f4dd99a8b0c4c1d11c66a P f366ff6c37a24eaaa6abcafab1467912 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:53.721052 24769 raft_consensus.cc:3054] T 2141d5ba289f4dd99a8b0c4c1d11c66a P 1a9b5a4afd63410b813b41edbfe69e28 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:58:53.725247 24694 raft_consensus.cc:2463] T 2141d5ba289f4dd99a8b0c4c1d11c66a P f366ff6c37a24eaaa6abcafab1467912 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 08c70c9a6a064d40bfa68fd105d86d86 in term 1.
I20250109 08:58:53.725621 24769 raft_consensus.cc:2463] T 2141d5ba289f4dd99a8b0c4c1d11c66a P 1a9b5a4afd63410b813b41edbfe69e28 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 08c70c9a6a064d40bfa68fd105d86d86 in term 1.
I20250109 08:58:53.726348 24583 leader_election.cc:304] T 2141d5ba289f4dd99a8b0c4c1d11c66a P 08c70c9a6a064d40bfa68fd105d86d86 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08c70c9a6a064d40bfa68fd105d86d86, f366ff6c37a24eaaa6abcafab1467912; no voters:
I20250109 08:58:53.727064 24842 raft_consensus.cc:2798] T 2141d5ba289f4dd99a8b0c4c1d11c66a P 08c70c9a6a064d40bfa68fd105d86d86 [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:58:53.727425 24842 raft_consensus.cc:695] T 2141d5ba289f4dd99a8b0c4c1d11c66a P 08c70c9a6a064d40bfa68fd105d86d86 [term 1 LEADER]: Becoming Leader. State: Replica: 08c70c9a6a064d40bfa68fd105d86d86, State: Running, Role: LEADER
I20250109 08:58:53.728128 24842 consensus_queue.cc:237] T 2141d5ba289f4dd99a8b0c4c1d11c66a P 08c70c9a6a064d40bfa68fd105d86d86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:53.735196 24515 catalog_manager.cc:5526] T 2141d5ba289f4dd99a8b0c4c1d11c66a P 08c70c9a6a064d40bfa68fd105d86d86 reported cstate change: term changed from 0 to 1, leader changed from <none> to 08c70c9a6a064d40bfa68fd105d86d86 (127.19.226.193). New cstate: current_term: 1 leader_uuid: "08c70c9a6a064d40bfa68fd105d86d86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:53.795946 24842 consensus_queue.cc:1035] T 0faa532dc2514e6b9991e3832a029fab P 08c70c9a6a064d40bfa68fd105d86d86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:58:53.806684 24842 consensus_queue.cc:1035] T 0faa532dc2514e6b9991e3832a029fab P 08c70c9a6a064d40bfa68fd105d86d86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:58:53.826884 24769 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "d8c90027b99f41b78f3b41c079e77ed7"
dest_uuid: "1a9b5a4afd63410b813b41edbfe69e28"
mode: GRACEFUL
new_leader_uuid: "08c70c9a6a064d40bfa68fd105d86d86"
from {username='slave'} at 127.0.0.1:37366
I20250109 08:58:53.827353 24769 raft_consensus.cc:604] T d8c90027b99f41b78f3b41c079e77ed7 P 1a9b5a4afd63410b813b41edbfe69e28 [term 1 LEADER]: Received request to transfer leadership to TS 08c70c9a6a064d40bfa68fd105d86d86
I20250109 08:58:53.831771 24694 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "e8d95383d13941a99003d8e54fbed202"
dest_uuid: "f366ff6c37a24eaaa6abcafab1467912"
mode: GRACEFUL
new_leader_uuid: "08c70c9a6a064d40bfa68fd105d86d86"
from {username='slave'} at 127.0.0.1:39922
I20250109 08:58:53.832245 24694 raft_consensus.cc:604] T e8d95383d13941a99003d8e54fbed202 P f366ff6c37a24eaaa6abcafab1467912 [term 1 LEADER]: Received request to transfer leadership to TS 08c70c9a6a064d40bfa68fd105d86d86
I20250109 08:58:53.836678 24769 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "ff0a7cf17ff34eedbdda1e5a4fac0dfa"
dest_uuid: "1a9b5a4afd63410b813b41edbfe69e28"
mode: GRACEFUL
new_leader_uuid: "f366ff6c37a24eaaa6abcafab1467912"
from {username='slave'} at 127.0.0.1:37366
I20250109 08:58:53.837127 24769 raft_consensus.cc:604] T ff0a7cf17ff34eedbdda1e5a4fac0dfa P 1a9b5a4afd63410b813b41edbfe69e28 [term 1 LEADER]: Received request to transfer leadership to TS f366ff6c37a24eaaa6abcafab1467912
I20250109 08:58:53.855319 24620 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "53eef9db17b548f081a60a39a43e4f19"
dest_uuid: "08c70c9a6a064d40bfa68fd105d86d86"
mode: GRACEFUL
new_leader_uuid: "f366ff6c37a24eaaa6abcafab1467912"
from {username='slave'} at 127.0.0.1:35910
I20250109 08:58:53.855834 24620 raft_consensus.cc:604] T 53eef9db17b548f081a60a39a43e4f19 P 08c70c9a6a064d40bfa68fd105d86d86 [term 1 LEADER]: Received request to transfer leadership to TS f366ff6c37a24eaaa6abcafab1467912
I20250109 08:58:53.862311 24620 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "2141d5ba289f4dd99a8b0c4c1d11c66a"
dest_uuid: "08c70c9a6a064d40bfa68fd105d86d86"
mode: GRACEFUL
new_leader_uuid: "f366ff6c37a24eaaa6abcafab1467912"
from {username='slave'} at 127.0.0.1:35910
I20250109 08:58:53.862792 24620 raft_consensus.cc:604] T 2141d5ba289f4dd99a8b0c4c1d11c66a P 08c70c9a6a064d40bfa68fd105d86d86 [term 1 LEADER]: Received request to transfer leadership to TS f366ff6c37a24eaaa6abcafab1467912
I20250109 08:58:53.866677 24620 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "0faa532dc2514e6b9991e3832a029fab"
dest_uuid: "08c70c9a6a064d40bfa68fd105d86d86"
mode: GRACEFUL
new_leader_uuid: "1a9b5a4afd63410b813b41edbfe69e28"
from {username='slave'} at 127.0.0.1:35910
I20250109 08:58:53.867106 24620 raft_consensus.cc:604] T 0faa532dc2514e6b9991e3832a029fab P 08c70c9a6a064d40bfa68fd105d86d86 [term 1 LEADER]: Received request to transfer leadership to TS 1a9b5a4afd63410b813b41edbfe69e28
I20250109 08:58:53.873935 24842 consensus_queue.cc:1035] T 53eef9db17b548f081a60a39a43e4f19 P 08c70c9a6a064d40bfa68fd105d86d86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:58:53.885059 24925 consensus_queue.cc:1035] T 53eef9db17b548f081a60a39a43e4f19 P 08c70c9a6a064d40bfa68fd105d86d86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:58:53.893018 24842 raft_consensus.cc:988] T 53eef9db17b548f081a60a39a43e4f19 P 08c70c9a6a064d40bfa68fd105d86d86: : Instructing follower f366ff6c37a24eaaa6abcafab1467912 to start an election
I20250109 08:58:53.893481 24925 raft_consensus.cc:1076] T 53eef9db17b548f081a60a39a43e4f19 P 08c70c9a6a064d40bfa68fd105d86d86 [term 1 LEADER]: Signalling peer f366ff6c37a24eaaa6abcafab1467912 to start an election
I20250109 08:58:53.894899 24694 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "53eef9db17b548f081a60a39a43e4f19"
dest_uuid: "f366ff6c37a24eaaa6abcafab1467912"
from {username='slave'} at 127.0.0.1:39906
I20250109 08:58:53.895386 24694 raft_consensus.cc:491] T 53eef9db17b548f081a60a39a43e4f19 P f366ff6c37a24eaaa6abcafab1467912 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:58:53.895665 24694 raft_consensus.cc:3054] T 53eef9db17b548f081a60a39a43e4f19 P f366ff6c37a24eaaa6abcafab1467912 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:53.901073 24694 raft_consensus.cc:513] T 53eef9db17b548f081a60a39a43e4f19 P f366ff6c37a24eaaa6abcafab1467912 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:53.902498 24694 leader_election.cc:290] T 53eef9db17b548f081a60a39a43e4f19 P f366ff6c37a24eaaa6abcafab1467912 [CANDIDATE]: Term 2 election: Requested vote from peers 08c70c9a6a064d40bfa68fd105d86d86 (127.19.226.193:43567), 1a9b5a4afd63410b813b41edbfe69e28 (127.19.226.195:42529)
I20250109 08:58:53.903200 24620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53eef9db17b548f081a60a39a43e4f19" candidate_uuid: "f366ff6c37a24eaaa6abcafab1467912" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "08c70c9a6a064d40bfa68fd105d86d86"
I20250109 08:58:53.903383 24769 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53eef9db17b548f081a60a39a43e4f19" candidate_uuid: "f366ff6c37a24eaaa6abcafab1467912" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1a9b5a4afd63410b813b41edbfe69e28"
I20250109 08:58:53.903822 24620 raft_consensus.cc:3049] T 53eef9db17b548f081a60a39a43e4f19 P 08c70c9a6a064d40bfa68fd105d86d86 [term 1 LEADER]: Stepping down as leader of term 1
I20250109 08:58:53.903920 24769 raft_consensus.cc:3054] T 53eef9db17b548f081a60a39a43e4f19 P 1a9b5a4afd63410b813b41edbfe69e28 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:53.904124 24620 raft_consensus.cc:738] T 53eef9db17b548f081a60a39a43e4f19 P 08c70c9a6a064d40bfa68fd105d86d86 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 08c70c9a6a064d40bfa68fd105d86d86, State: Running, Role: LEADER
I20250109 08:58:53.904834 24620 consensus_queue.cc:260] T 53eef9db17b548f081a60a39a43e4f19 P 08c70c9a6a064d40bfa68fd105d86d86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:53.905740 24620 raft_consensus.cc:3054] T 53eef9db17b548f081a60a39a43e4f19 P 08c70c9a6a064d40bfa68fd105d86d86 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:53.908424 24769 raft_consensus.cc:2463] T 53eef9db17b548f081a60a39a43e4f19 P 1a9b5a4afd63410b813b41edbfe69e28 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f366ff6c37a24eaaa6abcafab1467912 in term 2.
I20250109 08:58:53.909454 24661 leader_election.cc:304] T 53eef9db17b548f081a60a39a43e4f19 P f366ff6c37a24eaaa6abcafab1467912 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a9b5a4afd63410b813b41edbfe69e28, f366ff6c37a24eaaa6abcafab1467912; no voters:
I20250109 08:58:53.910084 24903 raft_consensus.cc:2798] T 53eef9db17b548f081a60a39a43e4f19 P f366ff6c37a24eaaa6abcafab1467912 [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:58:53.910557 24903 raft_consensus.cc:695] T 53eef9db17b548f081a60a39a43e4f19 P f366ff6c37a24eaaa6abcafab1467912 [term 2 LEADER]: Becoming Leader. State: Replica: f366ff6c37a24eaaa6abcafab1467912, State: Running, Role: LEADER
I20250109 08:58:53.910575 24620 raft_consensus.cc:2463] T 53eef9db17b548f081a60a39a43e4f19 P 08c70c9a6a064d40bfa68fd105d86d86 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f366ff6c37a24eaaa6abcafab1467912 in term 2.
I20250109 08:58:53.911453 24903 consensus_queue.cc:237] T 53eef9db17b548f081a60a39a43e4f19 P f366ff6c37a24eaaa6abcafab1467912 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:53.918215 24515 catalog_manager.cc:5526] T 53eef9db17b548f081a60a39a43e4f19 P f366ff6c37a24eaaa6abcafab1467912 reported cstate change: term changed from 1 to 2, leader changed from 08c70c9a6a064d40bfa68fd105d86d86 (127.19.226.193) to f366ff6c37a24eaaa6abcafab1467912 (127.19.226.194). New cstate: current_term: 2 leader_uuid: "f366ff6c37a24eaaa6abcafab1467912" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:53.986836 24895 consensus_queue.cc:1035] T d8c90027b99f41b78f3b41c079e77ed7 P 1a9b5a4afd63410b813b41edbfe69e28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:58:53.992748 24962 raft_consensus.cc:988] T d8c90027b99f41b78f3b41c079e77ed7 P 1a9b5a4afd63410b813b41edbfe69e28: : Instructing follower 08c70c9a6a064d40bfa68fd105d86d86 to start an election
I20250109 08:58:53.993216 24933 raft_consensus.cc:1076] T d8c90027b99f41b78f3b41c079e77ed7 P 1a9b5a4afd63410b813b41edbfe69e28 [term 1 LEADER]: Signalling peer 08c70c9a6a064d40bfa68fd105d86d86 to start an election
I20250109 08:58:53.994658 24616 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d8c90027b99f41b78f3b41c079e77ed7"
dest_uuid: "08c70c9a6a064d40bfa68fd105d86d86"
from {username='slave'} at 127.0.0.1:37628
I20250109 08:58:53.995838 24616 raft_consensus.cc:491] T d8c90027b99f41b78f3b41c079e77ed7 P 08c70c9a6a064d40bfa68fd105d86d86 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:58:53.996217 24616 raft_consensus.cc:3054] T d8c90027b99f41b78f3b41c079e77ed7 P 08c70c9a6a064d40bfa68fd105d86d86 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:53.998659 24933 consensus_queue.cc:1035] T d8c90027b99f41b78f3b41c079e77ed7 P 1a9b5a4afd63410b813b41edbfe69e28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:58:54.005146 24616 raft_consensus.cc:513] T d8c90027b99f41b78f3b41c079e77ed7 P 08c70c9a6a064d40bfa68fd105d86d86 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:54.008471 24769 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8c90027b99f41b78f3b41c079e77ed7" candidate_uuid: "08c70c9a6a064d40bfa68fd105d86d86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1a9b5a4afd63410b813b41edbfe69e28"
I20250109 08:58:54.009008 24769 raft_consensus.cc:3049] T d8c90027b99f41b78f3b41c079e77ed7 P 1a9b5a4afd63410b813b41edbfe69e28 [term 1 LEADER]: Stepping down as leader of term 1
I20250109 08:58:54.009315 24769 raft_consensus.cc:738] T d8c90027b99f41b78f3b41c079e77ed7 P 1a9b5a4afd63410b813b41edbfe69e28 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1a9b5a4afd63410b813b41edbfe69e28, State: Running, Role: LEADER
I20250109 08:58:54.009229 24694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8c90027b99f41b78f3b41c079e77ed7" candidate_uuid: "08c70c9a6a064d40bfa68fd105d86d86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f366ff6c37a24eaaa6abcafab1467912"
I20250109 08:58:54.010105 24694 raft_consensus.cc:3054] T d8c90027b99f41b78f3b41c079e77ed7 P f366ff6c37a24eaaa6abcafab1467912 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:54.010317 24769 consensus_queue.cc:260] T d8c90027b99f41b78f3b41c079e77ed7 P 1a9b5a4afd63410b813b41edbfe69e28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:54.011190 24769 raft_consensus.cc:3054] T d8c90027b99f41b78f3b41c079e77ed7 P 1a9b5a4afd63410b813b41edbfe69e28 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:54.015784 24616 leader_election.cc:290] T d8c90027b99f41b78f3b41c079e77ed7 P 08c70c9a6a064d40bfa68fd105d86d86 [CANDIDATE]: Term 2 election: Requested vote from peers f366ff6c37a24eaaa6abcafab1467912 (127.19.226.194:43355), 1a9b5a4afd63410b813b41edbfe69e28 (127.19.226.195:42529)
I20250109 08:58:54.017019 24694 raft_consensus.cc:2463] T d8c90027b99f41b78f3b41c079e77ed7 P f366ff6c37a24eaaa6abcafab1467912 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 08c70c9a6a064d40bfa68fd105d86d86 in term 2.
I20250109 08:58:54.017019 24769 raft_consensus.cc:2463] T d8c90027b99f41b78f3b41c079e77ed7 P 1a9b5a4afd63410b813b41edbfe69e28 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 08c70c9a6a064d40bfa68fd105d86d86 in term 2.
I20250109 08:58:54.018898 24583 leader_election.cc:304] T d8c90027b99f41b78f3b41c079e77ed7 P 08c70c9a6a064d40bfa68fd105d86d86 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08c70c9a6a064d40bfa68fd105d86d86, f366ff6c37a24eaaa6abcafab1467912; no voters:
I20250109 08:58:54.019698 24842 raft_consensus.cc:2798] T d8c90027b99f41b78f3b41c079e77ed7 P 08c70c9a6a064d40bfa68fd105d86d86 [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:58:54.020123 24842 raft_consensus.cc:695] T d8c90027b99f41b78f3b41c079e77ed7 P 08c70c9a6a064d40bfa68fd105d86d86 [term 2 LEADER]: Becoming Leader. State: Replica: 08c70c9a6a064d40bfa68fd105d86d86, State: Running, Role: LEADER
I20250109 08:58:54.020795 24842 consensus_queue.cc:237] T d8c90027b99f41b78f3b41c079e77ed7 P 08c70c9a6a064d40bfa68fd105d86d86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:54.027215 24518 catalog_manager.cc:5526] T d8c90027b99f41b78f3b41c079e77ed7 P 08c70c9a6a064d40bfa68fd105d86d86 reported cstate change: term changed from 1 to 2, leader changed from 1a9b5a4afd63410b813b41edbfe69e28 (127.19.226.195) to 08c70c9a6a064d40bfa68fd105d86d86 (127.19.226.193). New cstate: current_term: 2 leader_uuid: "08c70c9a6a064d40bfa68fd105d86d86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:54.053100 24849 raft_consensus.cc:988] T e8d95383d13941a99003d8e54fbed202 P f366ff6c37a24eaaa6abcafab1467912: : Instructing follower 08c70c9a6a064d40bfa68fd105d86d86 to start an election
I20250109 08:58:54.053575 24903 raft_consensus.cc:1076] T e8d95383d13941a99003d8e54fbed202 P f366ff6c37a24eaaa6abcafab1467912 [term 1 LEADER]: Signalling peer 08c70c9a6a064d40bfa68fd105d86d86 to start an election
I20250109 08:58:54.055436 24616 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "e8d95383d13941a99003d8e54fbed202"
dest_uuid: "08c70c9a6a064d40bfa68fd105d86d86"
from {username='slave'} at 127.0.0.1:35904
I20250109 08:58:54.056012 24616 raft_consensus.cc:491] T e8d95383d13941a99003d8e54fbed202 P 08c70c9a6a064d40bfa68fd105d86d86 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:58:54.056429 24616 raft_consensus.cc:3054] T e8d95383d13941a99003d8e54fbed202 P 08c70c9a6a064d40bfa68fd105d86d86 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:54.061424 24616 raft_consensus.cc:513] T e8d95383d13941a99003d8e54fbed202 P 08c70c9a6a064d40bfa68fd105d86d86 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:54.062827 24616 leader_election.cc:290] T e8d95383d13941a99003d8e54fbed202 P 08c70c9a6a064d40bfa68fd105d86d86 [CANDIDATE]: Term 2 election: Requested vote from peers f366ff6c37a24eaaa6abcafab1467912 (127.19.226.194:43355), 1a9b5a4afd63410b813b41edbfe69e28 (127.19.226.195:42529)
I20250109 08:58:54.063540 24694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8d95383d13941a99003d8e54fbed202" candidate_uuid: "08c70c9a6a064d40bfa68fd105d86d86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f366ff6c37a24eaaa6abcafab1467912"
I20250109 08:58:54.063719 24769 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8d95383d13941a99003d8e54fbed202" candidate_uuid: "08c70c9a6a064d40bfa68fd105d86d86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1a9b5a4afd63410b813b41edbfe69e28"
I20250109 08:58:54.064208 24694 raft_consensus.cc:3049] T e8d95383d13941a99003d8e54fbed202 P f366ff6c37a24eaaa6abcafab1467912 [term 1 LEADER]: Stepping down as leader of term 1
I20250109 08:58:54.064222 24769 raft_consensus.cc:3054] T e8d95383d13941a99003d8e54fbed202 P 1a9b5a4afd63410b813b41edbfe69e28 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:54.064627 24694 raft_consensus.cc:738] T e8d95383d13941a99003d8e54fbed202 P f366ff6c37a24eaaa6abcafab1467912 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f366ff6c37a24eaaa6abcafab1467912, State: Running, Role: LEADER
I20250109 08:58:54.065384 24694 consensus_queue.cc:260] T e8d95383d13941a99003d8e54fbed202 P f366ff6c37a24eaaa6abcafab1467912 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:54.066524 24694 raft_consensus.cc:3054] T e8d95383d13941a99003d8e54fbed202 P f366ff6c37a24eaaa6abcafab1467912 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:54.070567 24769 raft_consensus.cc:2463] T e8d95383d13941a99003d8e54fbed202 P 1a9b5a4afd63410b813b41edbfe69e28 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 08c70c9a6a064d40bfa68fd105d86d86 in term 2.
I20250109 08:58:54.071631 24586 leader_election.cc:304] T e8d95383d13941a99003d8e54fbed202 P 08c70c9a6a064d40bfa68fd105d86d86 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08c70c9a6a064d40bfa68fd105d86d86, 1a9b5a4afd63410b813b41edbfe69e28; no voters:
I20250109 08:58:54.072180 24694 raft_consensus.cc:2463] T e8d95383d13941a99003d8e54fbed202 P f366ff6c37a24eaaa6abcafab1467912 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 08c70c9a6a064d40bfa68fd105d86d86 in term 2.
I20250109 08:58:54.072264 24842 raft_consensus.cc:2798] T e8d95383d13941a99003d8e54fbed202 P 08c70c9a6a064d40bfa68fd105d86d86 [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:58:54.072898 24842 raft_consensus.cc:695] T e8d95383d13941a99003d8e54fbed202 P 08c70c9a6a064d40bfa68fd105d86d86 [term 2 LEADER]: Becoming Leader. State: Replica: 08c70c9a6a064d40bfa68fd105d86d86, State: Running, Role: LEADER
I20250109 08:58:54.073678 24842 consensus_queue.cc:237] T e8d95383d13941a99003d8e54fbed202 P 08c70c9a6a064d40bfa68fd105d86d86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:54.079772 24518 catalog_manager.cc:5526] T e8d95383d13941a99003d8e54fbed202 P 08c70c9a6a064d40bfa68fd105d86d86 reported cstate change: term changed from 1 to 2, leader changed from f366ff6c37a24eaaa6abcafab1467912 (127.19.226.194) to 08c70c9a6a064d40bfa68fd105d86d86 (127.19.226.193). New cstate: current_term: 2 leader_uuid: "08c70c9a6a064d40bfa68fd105d86d86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:54.165119 24975 raft_consensus.cc:988] T ff0a7cf17ff34eedbdda1e5a4fac0dfa P 1a9b5a4afd63410b813b41edbfe69e28: : Instructing follower f366ff6c37a24eaaa6abcafab1467912 to start an election
I20250109 08:58:54.165567 24975 raft_consensus.cc:1076] T ff0a7cf17ff34eedbdda1e5a4fac0dfa P 1a9b5a4afd63410b813b41edbfe69e28 [term 1 LEADER]: Signalling peer f366ff6c37a24eaaa6abcafab1467912 to start an election
I20250109 08:58:54.166889 24694 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "ff0a7cf17ff34eedbdda1e5a4fac0dfa"
dest_uuid: "f366ff6c37a24eaaa6abcafab1467912"
from {username='slave'} at 127.0.0.1:48098
I20250109 08:58:54.167299 24694 raft_consensus.cc:491] T ff0a7cf17ff34eedbdda1e5a4fac0dfa P f366ff6c37a24eaaa6abcafab1467912 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:58:54.167567 24694 raft_consensus.cc:3054] T ff0a7cf17ff34eedbdda1e5a4fac0dfa P f366ff6c37a24eaaa6abcafab1467912 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:54.172068 24694 raft_consensus.cc:513] T ff0a7cf17ff34eedbdda1e5a4fac0dfa P f366ff6c37a24eaaa6abcafab1467912 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:54.173400 24694 leader_election.cc:290] T ff0a7cf17ff34eedbdda1e5a4fac0dfa P f366ff6c37a24eaaa6abcafab1467912 [CANDIDATE]: Term 2 election: Requested vote from peers 08c70c9a6a064d40bfa68fd105d86d86 (127.19.226.193:43567), 1a9b5a4afd63410b813b41edbfe69e28 (127.19.226.195:42529)
I20250109 08:58:54.173985 24616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff0a7cf17ff34eedbdda1e5a4fac0dfa" candidate_uuid: "f366ff6c37a24eaaa6abcafab1467912" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "08c70c9a6a064d40bfa68fd105d86d86"
I20250109 08:58:54.174630 24767 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff0a7cf17ff34eedbdda1e5a4fac0dfa" candidate_uuid: "f366ff6c37a24eaaa6abcafab1467912" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1a9b5a4afd63410b813b41edbfe69e28"
I20250109 08:58:54.174656 24616 raft_consensus.cc:3054] T ff0a7cf17ff34eedbdda1e5a4fac0dfa P 08c70c9a6a064d40bfa68fd105d86d86 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:54.175154 24767 raft_consensus.cc:3049] T ff0a7cf17ff34eedbdda1e5a4fac0dfa P 1a9b5a4afd63410b813b41edbfe69e28 [term 1 LEADER]: Stepping down as leader of term 1
I20250109 08:58:54.175392 24767 raft_consensus.cc:738] T ff0a7cf17ff34eedbdda1e5a4fac0dfa P 1a9b5a4afd63410b813b41edbfe69e28 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1a9b5a4afd63410b813b41edbfe69e28, State: Running, Role: LEADER
I20250109 08:58:54.175947 24767 consensus_queue.cc:260] T ff0a7cf17ff34eedbdda1e5a4fac0dfa P 1a9b5a4afd63410b813b41edbfe69e28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:54.176891 24767 raft_consensus.cc:3054] T ff0a7cf17ff34eedbdda1e5a4fac0dfa P 1a9b5a4afd63410b813b41edbfe69e28 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:54.179525 24616 raft_consensus.cc:2463] T ff0a7cf17ff34eedbdda1e5a4fac0dfa P 08c70c9a6a064d40bfa68fd105d86d86 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f366ff6c37a24eaaa6abcafab1467912 in term 2.
I20250109 08:58:54.180541 24661 leader_election.cc:304] T ff0a7cf17ff34eedbdda1e5a4fac0dfa P f366ff6c37a24eaaa6abcafab1467912 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08c70c9a6a064d40bfa68fd105d86d86, f366ff6c37a24eaaa6abcafab1467912; no voters:
I20250109 08:58:54.181298 24903 raft_consensus.cc:2798] T ff0a7cf17ff34eedbdda1e5a4fac0dfa P f366ff6c37a24eaaa6abcafab1467912 [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:58:54.181649 24903 raft_consensus.cc:695] T ff0a7cf17ff34eedbdda1e5a4fac0dfa P f366ff6c37a24eaaa6abcafab1467912 [term 2 LEADER]: Becoming Leader. State: Replica: f366ff6c37a24eaaa6abcafab1467912, State: Running, Role: LEADER
I20250109 08:58:54.181810 24767 raft_consensus.cc:2463] T ff0a7cf17ff34eedbdda1e5a4fac0dfa P 1a9b5a4afd63410b813b41edbfe69e28 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f366ff6c37a24eaaa6abcafab1467912 in term 2.
I20250109 08:58:54.182329 24903 consensus_queue.cc:237] T ff0a7cf17ff34eedbdda1e5a4fac0dfa P f366ff6c37a24eaaa6abcafab1467912 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:54.188302 24842 raft_consensus.cc:988] T 0faa532dc2514e6b9991e3832a029fab P 08c70c9a6a064d40bfa68fd105d86d86: : Instructing follower 1a9b5a4afd63410b813b41edbfe69e28 to start an election
I20250109 08:58:54.188712 24925 raft_consensus.cc:1076] T 0faa532dc2514e6b9991e3832a029fab P 08c70c9a6a064d40bfa68fd105d86d86 [term 1 LEADER]: Signalling peer 1a9b5a4afd63410b813b41edbfe69e28 to start an election
I20250109 08:58:54.189056 24518 catalog_manager.cc:5526] T ff0a7cf17ff34eedbdda1e5a4fac0dfa P f366ff6c37a24eaaa6abcafab1467912 reported cstate change: term changed from 1 to 2, leader changed from 1a9b5a4afd63410b813b41edbfe69e28 (127.19.226.195) to f366ff6c37a24eaaa6abcafab1467912 (127.19.226.194). New cstate: current_term: 2 leader_uuid: "f366ff6c37a24eaaa6abcafab1467912" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:54.189985 24767 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "0faa532dc2514e6b9991e3832a029fab"
dest_uuid: "1a9b5a4afd63410b813b41edbfe69e28"
from {username='slave'} at 127.0.0.1:37354
I20250109 08:58:54.190507 24767 raft_consensus.cc:491] T 0faa532dc2514e6b9991e3832a029fab P 1a9b5a4afd63410b813b41edbfe69e28 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:58:54.190774 24767 raft_consensus.cc:3054] T 0faa532dc2514e6b9991e3832a029fab P 1a9b5a4afd63410b813b41edbfe69e28 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:54.196049 24767 raft_consensus.cc:513] T 0faa532dc2514e6b9991e3832a029fab P 1a9b5a4afd63410b813b41edbfe69e28 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:54.197945 24767 leader_election.cc:290] T 0faa532dc2514e6b9991e3832a029fab P 1a9b5a4afd63410b813b41edbfe69e28 [CANDIDATE]: Term 2 election: Requested vote from peers 08c70c9a6a064d40bfa68fd105d86d86 (127.19.226.193:43567), f366ff6c37a24eaaa6abcafab1467912 (127.19.226.194:43355)
I20250109 08:58:54.198606 24616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0faa532dc2514e6b9991e3832a029fab" candidate_uuid: "1a9b5a4afd63410b813b41edbfe69e28" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "08c70c9a6a064d40bfa68fd105d86d86"
I20250109 08:58:54.198894 24694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0faa532dc2514e6b9991e3832a029fab" candidate_uuid: "1a9b5a4afd63410b813b41edbfe69e28" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f366ff6c37a24eaaa6abcafab1467912"
I20250109 08:58:54.199307 24616 raft_consensus.cc:3049] T 0faa532dc2514e6b9991e3832a029fab P 08c70c9a6a064d40bfa68fd105d86d86 [term 1 LEADER]: Stepping down as leader of term 1
I20250109 08:58:54.199410 24694 raft_consensus.cc:3054] T 0faa532dc2514e6b9991e3832a029fab P f366ff6c37a24eaaa6abcafab1467912 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:54.199626 24616 raft_consensus.cc:738] T 0faa532dc2514e6b9991e3832a029fab P 08c70c9a6a064d40bfa68fd105d86d86 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 08c70c9a6a064d40bfa68fd105d86d86, State: Running, Role: LEADER
I20250109 08:58:54.200343 24616 consensus_queue.cc:260] T 0faa532dc2514e6b9991e3832a029fab P 08c70c9a6a064d40bfa68fd105d86d86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:54.201334 24616 raft_consensus.cc:3054] T 0faa532dc2514e6b9991e3832a029fab P 08c70c9a6a064d40bfa68fd105d86d86 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:54.203821 24694 raft_consensus.cc:2463] T 0faa532dc2514e6b9991e3832a029fab P f366ff6c37a24eaaa6abcafab1467912 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a9b5a4afd63410b813b41edbfe69e28 in term 2.
I20250109 08:58:54.205032 24732 leader_election.cc:304] T 0faa532dc2514e6b9991e3832a029fab P 1a9b5a4afd63410b813b41edbfe69e28 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a9b5a4afd63410b813b41edbfe69e28, f366ff6c37a24eaaa6abcafab1467912; no voters:
I20250109 08:58:54.205698 24975 raft_consensus.cc:2798] T 0faa532dc2514e6b9991e3832a029fab P 1a9b5a4afd63410b813b41edbfe69e28 [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:58:54.205905 24616 raft_consensus.cc:2463] T 0faa532dc2514e6b9991e3832a029fab P 08c70c9a6a064d40bfa68fd105d86d86 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a9b5a4afd63410b813b41edbfe69e28 in term 2.
I20250109 08:58:54.206060 24975 raft_consensus.cc:695] T 0faa532dc2514e6b9991e3832a029fab P 1a9b5a4afd63410b813b41edbfe69e28 [term 2 LEADER]: Becoming Leader. State: Replica: 1a9b5a4afd63410b813b41edbfe69e28, State: Running, Role: LEADER
I20250109 08:58:54.207404 24975 consensus_queue.cc:237] T 0faa532dc2514e6b9991e3832a029fab P 1a9b5a4afd63410b813b41edbfe69e28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:54.213392 24518 catalog_manager.cc:5526] T 0faa532dc2514e6b9991e3832a029fab P 1a9b5a4afd63410b813b41edbfe69e28 reported cstate change: term changed from 1 to 2, leader changed from 08c70c9a6a064d40bfa68fd105d86d86 (127.19.226.193) to 1a9b5a4afd63410b813b41edbfe69e28 (127.19.226.195). New cstate: current_term: 2 leader_uuid: "1a9b5a4afd63410b813b41edbfe69e28" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } health_report { overall_health: HEALTHY } } }
I20250109 08:58:54.325884 24925 consensus_queue.cc:1035] T 2141d5ba289f4dd99a8b0c4c1d11c66a P 08c70c9a6a064d40bfa68fd105d86d86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:58:54.337435 24925 consensus_queue.cc:1035] T 2141d5ba289f4dd99a8b0c4c1d11c66a P 08c70c9a6a064d40bfa68fd105d86d86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:58:54.346377 24925 raft_consensus.cc:988] T 2141d5ba289f4dd99a8b0c4c1d11c66a P 08c70c9a6a064d40bfa68fd105d86d86: : Instructing follower f366ff6c37a24eaaa6abcafab1467912 to start an election
I20250109 08:58:54.346791 24926 raft_consensus.cc:1076] T 2141d5ba289f4dd99a8b0c4c1d11c66a P 08c70c9a6a064d40bfa68fd105d86d86 [term 1 LEADER]: Signalling peer f366ff6c37a24eaaa6abcafab1467912 to start an election
I20250109 08:58:54.348173 24694 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2141d5ba289f4dd99a8b0c4c1d11c66a"
dest_uuid: "f366ff6c37a24eaaa6abcafab1467912"
from {username='slave'} at 127.0.0.1:39906
I20250109 08:58:54.348681 24694 raft_consensus.cc:491] T 2141d5ba289f4dd99a8b0c4c1d11c66a P f366ff6c37a24eaaa6abcafab1467912 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:58:54.348959 24694 raft_consensus.cc:3054] T 2141d5ba289f4dd99a8b0c4c1d11c66a P f366ff6c37a24eaaa6abcafab1467912 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:54.353164 24694 raft_consensus.cc:513] T 2141d5ba289f4dd99a8b0c4c1d11c66a P f366ff6c37a24eaaa6abcafab1467912 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:54.354559 24694 leader_election.cc:290] T 2141d5ba289f4dd99a8b0c4c1d11c66a P f366ff6c37a24eaaa6abcafab1467912 [CANDIDATE]: Term 2 election: Requested vote from peers 08c70c9a6a064d40bfa68fd105d86d86 (127.19.226.193:43567), 1a9b5a4afd63410b813b41edbfe69e28 (127.19.226.195:42529)
I20250109 08:58:54.355438 24616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2141d5ba289f4dd99a8b0c4c1d11c66a" candidate_uuid: "f366ff6c37a24eaaa6abcafab1467912" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "08c70c9a6a064d40bfa68fd105d86d86"
I20250109 08:58:54.356107 24616 raft_consensus.cc:3049] T 2141d5ba289f4dd99a8b0c4c1d11c66a P 08c70c9a6a064d40bfa68fd105d86d86 [term 1 LEADER]: Stepping down as leader of term 1
I20250109 08:58:54.355953 24767 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2141d5ba289f4dd99a8b0c4c1d11c66a" candidate_uuid: "f366ff6c37a24eaaa6abcafab1467912" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1a9b5a4afd63410b813b41edbfe69e28"
I20250109 08:58:54.356479 24616 raft_consensus.cc:738] T 2141d5ba289f4dd99a8b0c4c1d11c66a P 08c70c9a6a064d40bfa68fd105d86d86 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 08c70c9a6a064d40bfa68fd105d86d86, State: Running, Role: LEADER
I20250109 08:58:54.356673 24767 raft_consensus.cc:3054] T 2141d5ba289f4dd99a8b0c4c1d11c66a P 1a9b5a4afd63410b813b41edbfe69e28 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:54.357293 24616 consensus_queue.cc:260] T 2141d5ba289f4dd99a8b0c4c1d11c66a P 08c70c9a6a064d40bfa68fd105d86d86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:54.358608 24616 raft_consensus.cc:3054] T 2141d5ba289f4dd99a8b0c4c1d11c66a P 08c70c9a6a064d40bfa68fd105d86d86 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:54.363689 24767 raft_consensus.cc:2463] T 2141d5ba289f4dd99a8b0c4c1d11c66a P 1a9b5a4afd63410b813b41edbfe69e28 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f366ff6c37a24eaaa6abcafab1467912 in term 2.
I20250109 08:58:54.364795 24661 leader_election.cc:304] T 2141d5ba289f4dd99a8b0c4c1d11c66a P f366ff6c37a24eaaa6abcafab1467912 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a9b5a4afd63410b813b41edbfe69e28, f366ff6c37a24eaaa6abcafab1467912; no voters:
I20250109 08:58:54.364966 24616 raft_consensus.cc:2463] T 2141d5ba289f4dd99a8b0c4c1d11c66a P 08c70c9a6a064d40bfa68fd105d86d86 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f366ff6c37a24eaaa6abcafab1467912 in term 2.
I20250109 08:58:54.365581 24903 raft_consensus.cc:2798] T 2141d5ba289f4dd99a8b0c4c1d11c66a P f366ff6c37a24eaaa6abcafab1467912 [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:58:54.366016 24903 raft_consensus.cc:695] T 2141d5ba289f4dd99a8b0c4c1d11c66a P f366ff6c37a24eaaa6abcafab1467912 [term 2 LEADER]: Becoming Leader. State: Replica: f366ff6c37a24eaaa6abcafab1467912, State: Running, Role: LEADER
I20250109 08:58:54.366700 24903 consensus_queue.cc:237] T 2141d5ba289f4dd99a8b0c4c1d11c66a P f366ff6c37a24eaaa6abcafab1467912 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:54.372808 24515 catalog_manager.cc:5526] T 2141d5ba289f4dd99a8b0c4c1d11c66a P f366ff6c37a24eaaa6abcafab1467912 reported cstate change: term changed from 1 to 2, leader changed from 08c70c9a6a064d40bfa68fd105d86d86 (127.19.226.193) to f366ff6c37a24eaaa6abcafab1467912 (127.19.226.194). New cstate: current_term: 2 leader_uuid: "f366ff6c37a24eaaa6abcafab1467912" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } health_report { overall_health: UNKNOWN } } }
I20250109 08:58:54.458034 24694 raft_consensus.cc:1270] T d8c90027b99f41b78f3b41c079e77ed7 P f366ff6c37a24eaaa6abcafab1467912 [term 2 FOLLOWER]: Refusing update from remote peer 08c70c9a6a064d40bfa68fd105d86d86: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:58:54.459131 24926 consensus_queue.cc:1035] T d8c90027b99f41b78f3b41c079e77ed7 P 08c70c9a6a064d40bfa68fd105d86d86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:58:54.466094 24767 raft_consensus.cc:1270] T d8c90027b99f41b78f3b41c079e77ed7 P 1a9b5a4afd63410b813b41edbfe69e28 [term 2 FOLLOWER]: Refusing update from remote peer 08c70c9a6a064d40bfa68fd105d86d86: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:58:54.469237 24925 consensus_queue.cc:1035] T d8c90027b99f41b78f3b41c079e77ed7 P 08c70c9a6a064d40bfa68fd105d86d86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:58:54.471671 24694 raft_consensus.cc:1270] T e8d95383d13941a99003d8e54fbed202 P f366ff6c37a24eaaa6abcafab1467912 [term 2 FOLLOWER]: Refusing update from remote peer 08c70c9a6a064d40bfa68fd105d86d86: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:58:54.473147 24925 consensus_queue.cc:1035] T e8d95383d13941a99003d8e54fbed202 P 08c70c9a6a064d40bfa68fd105d86d86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:58:54.481072 24767 raft_consensus.cc:1270] T e8d95383d13941a99003d8e54fbed202 P 1a9b5a4afd63410b813b41edbfe69e28 [term 2 FOLLOWER]: Refusing update from remote peer 08c70c9a6a064d40bfa68fd105d86d86: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:58:54.482393 24926 consensus_queue.cc:1035] T e8d95383d13941a99003d8e54fbed202 P 08c70c9a6a064d40bfa68fd105d86d86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:58:54.537470 24616 raft_consensus.cc:1270] T 53eef9db17b548f081a60a39a43e4f19 P 08c70c9a6a064d40bfa68fd105d86d86 [term 2 FOLLOWER]: Refusing update from remote peer f366ff6c37a24eaaa6abcafab1467912: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:58:54.537629 24769 raft_consensus.cc:1270] T 53eef9db17b548f081a60a39a43e4f19 P 1a9b5a4afd63410b813b41edbfe69e28 [term 2 FOLLOWER]: Refusing update from remote peer f366ff6c37a24eaaa6abcafab1467912: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:58:54.538872 24849 consensus_queue.cc:1035] T 53eef9db17b548f081a60a39a43e4f19 P f366ff6c37a24eaaa6abcafab1467912 [LEADER]: Connected to new peer: Peer: permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:58:54.539856 24903 consensus_queue.cc:1035] T 53eef9db17b548f081a60a39a43e4f19 P f366ff6c37a24eaaa6abcafab1467912 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:58:54.579144 24767 raft_consensus.cc:1270] T ff0a7cf17ff34eedbdda1e5a4fac0dfa P 1a9b5a4afd63410b813b41edbfe69e28 [term 2 FOLLOWER]: Refusing update from remote peer f366ff6c37a24eaaa6abcafab1467912: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:58:54.580425 24903 consensus_queue.cc:1035] T ff0a7cf17ff34eedbdda1e5a4fac0dfa P f366ff6c37a24eaaa6abcafab1467912 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:58:54.587061 24620 raft_consensus.cc:1270] T ff0a7cf17ff34eedbdda1e5a4fac0dfa P 08c70c9a6a064d40bfa68fd105d86d86 [term 2 FOLLOWER]: Refusing update from remote peer f366ff6c37a24eaaa6abcafab1467912: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:58:54.588562 24923 consensus_queue.cc:1035] T ff0a7cf17ff34eedbdda1e5a4fac0dfa P f366ff6c37a24eaaa6abcafab1467912 [LEADER]: Connected to new peer: Peer: permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:58:54.760663 24694 raft_consensus.cc:1270] T 0faa532dc2514e6b9991e3832a029fab P f366ff6c37a24eaaa6abcafab1467912 [term 2 FOLLOWER]: Refusing update from remote peer 1a9b5a4afd63410b813b41edbfe69e28: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:58:54.761775 24975 consensus_queue.cc:1035] T 0faa532dc2514e6b9991e3832a029fab P 1a9b5a4afd63410b813b41edbfe69e28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:58:54.769089 24620 raft_consensus.cc:1270] T 0faa532dc2514e6b9991e3832a029fab P 08c70c9a6a064d40bfa68fd105d86d86 [term 2 FOLLOWER]: Refusing update from remote peer 1a9b5a4afd63410b813b41edbfe69e28: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:58:54.770982 24978 consensus_queue.cc:1035] T 0faa532dc2514e6b9991e3832a029fab P 1a9b5a4afd63410b813b41edbfe69e28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:58:54.800110 24620 raft_consensus.cc:1270] T 2141d5ba289f4dd99a8b0c4c1d11c66a P 08c70c9a6a064d40bfa68fd105d86d86 [term 2 FOLLOWER]: Refusing update from remote peer f366ff6c37a24eaaa6abcafab1467912: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:58:54.801223 24923 consensus_queue.cc:1035] T 2141d5ba289f4dd99a8b0c4c1d11c66a P f366ff6c37a24eaaa6abcafab1467912 [LEADER]: Connected to new peer: Peer: permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:58:54.808172 24767 raft_consensus.cc:1270] T 2141d5ba289f4dd99a8b0c4c1d11c66a P 1a9b5a4afd63410b813b41edbfe69e28 [term 2 FOLLOWER]: Refusing update from remote peer f366ff6c37a24eaaa6abcafab1467912: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:58:54.809619 24923 consensus_queue.cc:1035] T 2141d5ba289f4dd99a8b0c4c1d11c66a P f366ff6c37a24eaaa6abcafab1467912 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20250109 08:58:56.870147 24515 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:56748:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20250109 08:58:56.888758 24515 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:56748:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20250109 08:58:56.905546 20363 auto_leader_rebalancer.cc:140] leader rebalance for table soft_deleted_table
I20250109 08:58:56.966289 25008 tablet.cc:1721] T a3207340610143c7bfb4943c87dc78bb P 08c70c9a6a064d40bfa68fd105d86d86: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
I20250109 08:58:57.137773 25008 tablet.cc:1721] T d8c90027b99f41b78f3b41c079e77ed7 P 08c70c9a6a064d40bfa68fd105d86d86: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
W20250109 08:58:57.179456 24661 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.226.193:43567, user_credentials={real_user=slave}} blocked reactor thread for 60779.8us
I20250109 08:58:57.191195 25027 tablet.cc:1721] T d8c90027b99f41b78f3b41c079e77ed7 P 1a9b5a4afd63410b813b41edbfe69e28: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
I20250109 08:58:57.204339 25039 tablet.cc:1721] T a3207340610143c7bfb4943c87dc78bb P f366ff6c37a24eaaa6abcafab1467912: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
W20250109 08:58:57.228340 24646 tablet_mm_ops.cc:139] Rowset compaction is disabled (check --enable_rowset_compaction and disable_compaction in extra_config for tablet:a3207340610143c7bfb4943c87dc78bb)
W20250109 08:58:57.248984 24646 tablet_mm_ops.cc:304] Major delta compaction is disabled (check --enable_major_delta_compaction and disable_compaction in extra_config for tablet:a3207340610143c7bfb4943c87dc78bb)
W20250109 08:58:57.252806 24646 tablet_mm_ops.cc:220] Minor delta compaction is disabled (check --enable_minor_delta_compaction and disable_compaction in extra_config for tablet:a3207340610143c7bfb4943c87dc78bb)
W20250109 08:58:57.265112 25025 log.cc:926] Time spent T e8d95383d13941a99003d8e54fbed202 P f366ff6c37a24eaaa6abcafab1467912: Append to log took a long time: real 0.078s user 0.003s sys 0.000s
I20250109 08:58:57.264664 25044 tablet.cc:1721] T 0faa532dc2514e6b9991e3832a029fab P 1a9b5a4afd63410b813b41edbfe69e28: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
I20250109 08:58:57.226364 25037 tablet.cc:1721] T ff0a7cf17ff34eedbdda1e5a4fac0dfa P f366ff6c37a24eaaa6abcafab1467912: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
W20250109 08:58:57.250895 24583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.226.194:43355, user_credentials={real_user=slave}} blocked reactor thread for 54718.2us
I20250109 08:58:57.270359 24693 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "53eef9db17b548f081a60a39a43e4f19"
dest_uuid: "f366ff6c37a24eaaa6abcafab1467912"
mode: GRACEFUL
new_leader_uuid: "1a9b5a4afd63410b813b41edbfe69e28"
from {username='slave'} at 127.0.0.1:39938
I20250109 08:58:57.256281 25030 tablet.cc:1721] T d8c90027b99f41b78f3b41c079e77ed7 P f366ff6c37a24eaaa6abcafab1467912: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
I20250109 08:58:57.282625 25031 tablet.cc:1721] T e8d95383d13941a99003d8e54fbed202 P 08c70c9a6a064d40bfa68fd105d86d86: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
I20250109 08:58:57.305227 24693 raft_consensus.cc:604] T 53eef9db17b548f081a60a39a43e4f19 P f366ff6c37a24eaaa6abcafab1467912 [term 2 LEADER]: Received request to transfer leadership to TS 1a9b5a4afd63410b813b41edbfe69e28
W20250109 08:58:57.329128 25017 log.cc:926] Time spent T 9e0af0555b484b748a03f99b45c463ef P f366ff6c37a24eaaa6abcafab1467912: Append to log took a long time: real 0.072s user 0.001s sys 0.000s
I20250109 08:58:57.342252 25045 tablet.cc:1721] T e8d95383d13941a99003d8e54fbed202 P 1a9b5a4afd63410b813b41edbfe69e28: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
I20250109 08:58:57.346834 25048 tablet.cc:1721] T 0364f388e25b466b8b39fdaa50aebf42 P 08c70c9a6a064d40bfa68fd105d86d86: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
I20250109 08:58:57.347622 25039 tablet.cc:1721] T 2141d5ba289f4dd99a8b0c4c1d11c66a P f366ff6c37a24eaaa6abcafab1467912: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
I20250109 08:58:57.357082 25008 tablet.cc:1721] T 0faa532dc2514e6b9991e3832a029fab P 08c70c9a6a064d40bfa68fd105d86d86: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
I20250109 08:58:57.365345 25049 tablet.cc:1721] T ff0a7cf17ff34eedbdda1e5a4fac0dfa P 1a9b5a4afd63410b813b41edbfe69e28: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
I20250109 08:58:57.368579 25031 tablet.cc:1721] T ff0a7cf17ff34eedbdda1e5a4fac0dfa P 08c70c9a6a064d40bfa68fd105d86d86: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
I20250109 08:58:57.372252 25048 tablet.cc:1721] T 2141d5ba289f4dd99a8b0c4c1d11c66a P 08c70c9a6a064d40bfa68fd105d86d86: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
I20250109 08:58:57.373795 25037 tablet.cc:1721] T 0faa532dc2514e6b9991e3832a029fab P f366ff6c37a24eaaa6abcafab1467912: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
I20250109 08:58:57.391364 25050 tablet.cc:1721] T 0364f388e25b466b8b39fdaa50aebf42 P f366ff6c37a24eaaa6abcafab1467912: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
I20250109 08:58:57.392524 24923 raft_consensus.cc:988] T 53eef9db17b548f081a60a39a43e4f19 P f366ff6c37a24eaaa6abcafab1467912: : Instructing follower 1a9b5a4afd63410b813b41edbfe69e28 to start an election
I20250109 08:58:57.393064 24923 raft_consensus.cc:1076] T 53eef9db17b548f081a60a39a43e4f19 P f366ff6c37a24eaaa6abcafab1467912 [term 2 LEADER]: Signalling peer 1a9b5a4afd63410b813b41edbfe69e28 to start an election
I20250109 08:58:57.395603 24767 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "53eef9db17b548f081a60a39a43e4f19"
dest_uuid: "1a9b5a4afd63410b813b41edbfe69e28"
from {username='slave'} at 127.0.0.1:37340
I20250109 08:58:57.396697 24767 raft_consensus.cc:491] T 53eef9db17b548f081a60a39a43e4f19 P 1a9b5a4afd63410b813b41edbfe69e28 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:58:57.397043 24767 raft_consensus.cc:3054] T 53eef9db17b548f081a60a39a43e4f19 P 1a9b5a4afd63410b813b41edbfe69e28 [term 2 FOLLOWER]: Advancing to term 3
I20250109 08:58:57.387109 25027 tablet.cc:1721] T 2141d5ba289f4dd99a8b0c4c1d11c66a P 1a9b5a4afd63410b813b41edbfe69e28: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
I20250109 08:58:57.409782 24767 raft_consensus.cc:513] T 53eef9db17b548f081a60a39a43e4f19 P 1a9b5a4afd63410b813b41edbfe69e28 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:57.412216 24617 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53eef9db17b548f081a60a39a43e4f19" candidate_uuid: "1a9b5a4afd63410b813b41edbfe69e28" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "08c70c9a6a064d40bfa68fd105d86d86"
I20250109 08:58:57.412817 24617 raft_consensus.cc:3054] T 53eef9db17b548f081a60a39a43e4f19 P 08c70c9a6a064d40bfa68fd105d86d86 [term 2 FOLLOWER]: Advancing to term 3
I20250109 08:58:57.419479 25030 tablet.cc:1721] T 53eef9db17b548f081a60a39a43e4f19 P f366ff6c37a24eaaa6abcafab1467912: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
I20250109 08:58:57.422340 24617 raft_consensus.cc:2463] T 53eef9db17b548f081a60a39a43e4f19 P 08c70c9a6a064d40bfa68fd105d86d86 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a9b5a4afd63410b813b41edbfe69e28 in term 3.
I20250109 08:58:57.427064 24767 leader_election.cc:290] T 53eef9db17b548f081a60a39a43e4f19 P 1a9b5a4afd63410b813b41edbfe69e28 [CANDIDATE]: Term 3 election: Requested vote from peers 08c70c9a6a064d40bfa68fd105d86d86 (127.19.226.193:43567), f366ff6c37a24eaaa6abcafab1467912 (127.19.226.194:43355)
I20250109 08:58:57.466852 25044 tablet.cc:1721] T 53eef9db17b548f081a60a39a43e4f19 P 1a9b5a4afd63410b813b41edbfe69e28: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
I20250109 08:58:57.471350 24735 leader_election.cc:304] T 53eef9db17b548f081a60a39a43e4f19 P 1a9b5a4afd63410b813b41edbfe69e28 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08c70c9a6a064d40bfa68fd105d86d86, 1a9b5a4afd63410b813b41edbfe69e28; no voters:
I20250109 08:58:57.470202 24695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53eef9db17b548f081a60a39a43e4f19" candidate_uuid: "1a9b5a4afd63410b813b41edbfe69e28" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "f366ff6c37a24eaaa6abcafab1467912"
I20250109 08:58:57.472412 25033 raft_consensus.cc:2798] T 53eef9db17b548f081a60a39a43e4f19 P 1a9b5a4afd63410b813b41edbfe69e28 [term 3 FOLLOWER]: Leader election won for term 3
I20250109 08:58:57.472894 25033 raft_consensus.cc:695] T 53eef9db17b548f081a60a39a43e4f19 P 1a9b5a4afd63410b813b41edbfe69e28 [term 3 LEADER]: Becoming Leader. State: Replica: 1a9b5a4afd63410b813b41edbfe69e28, State: Running, Role: LEADER
I20250109 08:58:57.473558 25033 consensus_queue.cc:237] T 53eef9db17b548f081a60a39a43e4f19 P 1a9b5a4afd63410b813b41edbfe69e28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:57.474076 24617 raft_consensus.cc:1235] T 53eef9db17b548f081a60a39a43e4f19 P 08c70c9a6a064d40bfa68fd105d86d86 [term 3 FOLLOWER]: Rejecting Update request from peer f366ff6c37a24eaaa6abcafab1467912 for earlier term 2. Current term is 3. Ops: []
I20250109 08:58:57.477567 25031 tablet.cc:1721] T 9e0af0555b484b748a03f99b45c463ef P 08c70c9a6a064d40bfa68fd105d86d86: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
I20250109 08:58:57.477720 25030 tablet.cc:1721] T e8d95383d13941a99003d8e54fbed202 P f366ff6c37a24eaaa6abcafab1467912: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
I20250109 08:58:57.480449 24695 raft_consensus.cc:3049] T 53eef9db17b548f081a60a39a43e4f19 P f366ff6c37a24eaaa6abcafab1467912 [term 2 LEADER]: Stepping down as leader of term 2
I20250109 08:58:57.480804 24695 raft_consensus.cc:738] T 53eef9db17b548f081a60a39a43e4f19 P f366ff6c37a24eaaa6abcafab1467912 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f366ff6c37a24eaaa6abcafab1467912, State: Running, Role: LEADER
I20250109 08:58:57.480798 25034 consensus_queue.cc:1046] T 53eef9db17b548f081a60a39a43e4f19 P f366ff6c37a24eaaa6abcafab1467912 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 }, Status: INVALID_TERM, Last received: 2.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20250109 08:58:57.482270 24695 consensus_queue.cc:260] T 53eef9db17b548f081a60a39a43e4f19 P f366ff6c37a24eaaa6abcafab1467912 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:57.483547 24695 raft_consensus.cc:3054] T 53eef9db17b548f081a60a39a43e4f19 P f366ff6c37a24eaaa6abcafab1467912 [term 2 FOLLOWER]: Advancing to term 3
I20250109 08:58:57.487833 25045 tablet.cc:1721] T a3207340610143c7bfb4943c87dc78bb P 1a9b5a4afd63410b813b41edbfe69e28: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
I20250109 08:58:57.490778 25049 tablet.cc:1721] T 0364f388e25b466b8b39fdaa50aebf42 P 1a9b5a4afd63410b813b41edbfe69e28: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
I20250109 08:58:57.491117 24695 raft_consensus.cc:2463] T 53eef9db17b548f081a60a39a43e4f19 P f366ff6c37a24eaaa6abcafab1467912 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a9b5a4afd63410b813b41edbfe69e28 in term 3.
W20250109 08:58:57.497066 25023 raft_consensus.cc:931] Couldn't advance consensus term.: Illegal state: Can't advance term to: 3 current term: 3 is higher.
I20250109 08:58:57.497313 25050 tablet.cc:1721] T 9e0af0555b484b748a03f99b45c463ef P f366ff6c37a24eaaa6abcafab1467912: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
I20250109 08:58:57.510955 25027 tablet.cc:1721] T 9e0af0555b484b748a03f99b45c463ef P 1a9b5a4afd63410b813b41edbfe69e28: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
I20250109 08:58:57.516356 24517 catalog_manager.cc:5526] T 53eef9db17b548f081a60a39a43e4f19 P 1a9b5a4afd63410b813b41edbfe69e28 reported cstate change: term changed from 2 to 3, leader changed from f366ff6c37a24eaaa6abcafab1467912 (127.19.226.194) to 1a9b5a4afd63410b813b41edbfe69e28 (127.19.226.195). New cstate: current_term: 3 leader_uuid: "1a9b5a4afd63410b813b41edbfe69e28" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } health_report { overall_health: HEALTHY } } }
I20250109 08:58:57.525852 24617 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "a3207340610143c7bfb4943c87dc78bb"
dest_uuid: "08c70c9a6a064d40bfa68fd105d86d86"
mode: GRACEFUL
new_leader_uuid: "1a9b5a4afd63410b813b41edbfe69e28"
from {username='slave'} at 127.0.0.1:35922
I20250109 08:58:57.526589 24617 raft_consensus.cc:604] T a3207340610143c7bfb4943c87dc78bb P 08c70c9a6a064d40bfa68fd105d86d86 [term 1 LEADER]: Received request to transfer leadership to TS 1a9b5a4afd63410b813b41edbfe69e28
I20250109 08:58:57.527938 20363 auto_leader_rebalancer.cc:388] table: soft_deleted_table, leader rebalance finish, leader transfer count: 2
I20250109 08:58:57.528618 20363 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250109 08:58:57.537987 24617 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "970b2f8b130d431cb25d79d33d5f0e5c"
dest_uuid: "08c70c9a6a064d40bfa68fd105d86d86"
mode: GRACEFUL
new_leader_uuid: "1a9b5a4afd63410b813b41edbfe69e28"
from {username='slave'} at 127.0.0.1:35922
I20250109 08:58:57.538609 24617 raft_consensus.cc:604] T 970b2f8b130d431cb25d79d33d5f0e5c P 08c70c9a6a064d40bfa68fd105d86d86 [term 2 LEADER]: Received request to transfer leadership to TS 1a9b5a4afd63410b813b41edbfe69e28
I20250109 08:58:57.541565 24695 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "98383febcb2145208b05b96598aed748"
dest_uuid: "f366ff6c37a24eaaa6abcafab1467912"
mode: GRACEFUL
new_leader_uuid: "1a9b5a4afd63410b813b41edbfe69e28"
from {username='slave'} at 127.0.0.1:39938
I20250109 08:58:57.542145 24695 raft_consensus.cc:604] T 98383febcb2145208b05b96598aed748 P f366ff6c37a24eaaa6abcafab1467912 [term 1 LEADER]: Received request to transfer leadership to TS 1a9b5a4afd63410b813b41edbfe69e28
I20250109 08:58:57.544306 20363 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 2
I20250109 08:58:57.544919 20363 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250109 08:58:57.548885 24518 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 8066e53f72404343aa3a9f0ff5455781: soft_deleted_table [id=f05587e8b44d44c898cd6984d831bfbb] alter complete (version 1)
I20250109 08:58:57.614300 25038 raft_consensus.cc:988] T 98383febcb2145208b05b96598aed748 P f366ff6c37a24eaaa6abcafab1467912: : Instructing follower 1a9b5a4afd63410b813b41edbfe69e28 to start an election
I20250109 08:58:57.614733 25034 raft_consensus.cc:1076] T 98383febcb2145208b05b96598aed748 P f366ff6c37a24eaaa6abcafab1467912 [term 1 LEADER]: Signalling peer 1a9b5a4afd63410b813b41edbfe69e28 to start an election
I20250109 08:58:57.616228 24768 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "98383febcb2145208b05b96598aed748"
dest_uuid: "1a9b5a4afd63410b813b41edbfe69e28"
from {username='slave'} at 127.0.0.1:37340
I20250109 08:58:57.616755 24768 raft_consensus.cc:491] T 98383febcb2145208b05b96598aed748 P 1a9b5a4afd63410b813b41edbfe69e28 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:58:57.617040 24768 raft_consensus.cc:3054] T 98383febcb2145208b05b96598aed748 P 1a9b5a4afd63410b813b41edbfe69e28 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:57.621882 24768 raft_consensus.cc:513] T 98383febcb2145208b05b96598aed748 P 1a9b5a4afd63410b813b41edbfe69e28 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:57.623646 24768 leader_election.cc:290] T 98383febcb2145208b05b96598aed748 P 1a9b5a4afd63410b813b41edbfe69e28 [CANDIDATE]: Term 2 election: Requested vote from peers 08c70c9a6a064d40bfa68fd105d86d86 (127.19.226.193:43567), f366ff6c37a24eaaa6abcafab1467912 (127.19.226.194:43355)
I20250109 08:58:57.624433 24617 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98383febcb2145208b05b96598aed748" candidate_uuid: "1a9b5a4afd63410b813b41edbfe69e28" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "08c70c9a6a064d40bfa68fd105d86d86"
I20250109 08:58:57.624660 24693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98383febcb2145208b05b96598aed748" candidate_uuid: "1a9b5a4afd63410b813b41edbfe69e28" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f366ff6c37a24eaaa6abcafab1467912"
I20250109 08:58:57.625109 24617 raft_consensus.cc:3054] T 98383febcb2145208b05b96598aed748 P 08c70c9a6a064d40bfa68fd105d86d86 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:57.625305 24693 raft_consensus.cc:3049] T 98383febcb2145208b05b96598aed748 P f366ff6c37a24eaaa6abcafab1467912 [term 1 LEADER]: Stepping down as leader of term 1
I20250109 08:58:57.625720 24693 raft_consensus.cc:738] T 98383febcb2145208b05b96598aed748 P f366ff6c37a24eaaa6abcafab1467912 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f366ff6c37a24eaaa6abcafab1467912, State: Running, Role: LEADER
I20250109 08:58:57.626463 24693 consensus_queue.cc:260] T 98383febcb2145208b05b96598aed748 P f366ff6c37a24eaaa6abcafab1467912 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:57.627399 24693 raft_consensus.cc:3054] T 98383febcb2145208b05b96598aed748 P f366ff6c37a24eaaa6abcafab1467912 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:57.630275 24617 raft_consensus.cc:2463] T 98383febcb2145208b05b96598aed748 P 08c70c9a6a064d40bfa68fd105d86d86 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a9b5a4afd63410b813b41edbfe69e28 in term 2.
I20250109 08:58:57.631312 24735 leader_election.cc:304] T 98383febcb2145208b05b96598aed748 P 1a9b5a4afd63410b813b41edbfe69e28 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08c70c9a6a064d40bfa68fd105d86d86, 1a9b5a4afd63410b813b41edbfe69e28; no voters:
I20250109 08:58:57.632097 25040 raft_consensus.cc:2798] T 98383febcb2145208b05b96598aed748 P 1a9b5a4afd63410b813b41edbfe69e28 [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:58:57.632125 24693 raft_consensus.cc:2463] T 98383febcb2145208b05b96598aed748 P f366ff6c37a24eaaa6abcafab1467912 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a9b5a4afd63410b813b41edbfe69e28 in term 2.
I20250109 08:58:57.632601 25040 raft_consensus.cc:695] T 98383febcb2145208b05b96598aed748 P 1a9b5a4afd63410b813b41edbfe69e28 [term 2 LEADER]: Becoming Leader. State: Replica: 1a9b5a4afd63410b813b41edbfe69e28, State: Running, Role: LEADER
I20250109 08:58:57.633358 25040 consensus_queue.cc:237] T 98383febcb2145208b05b96598aed748 P 1a9b5a4afd63410b813b41edbfe69e28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:57.640189 24517 catalog_manager.cc:5526] T 98383febcb2145208b05b96598aed748 P 1a9b5a4afd63410b813b41edbfe69e28 reported cstate change: term changed from 1 to 2, leader changed from f366ff6c37a24eaaa6abcafab1467912 (127.19.226.194) to 1a9b5a4afd63410b813b41edbfe69e28 (127.19.226.195). New cstate: current_term: 2 leader_uuid: "1a9b5a4afd63410b813b41edbfe69e28" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } health_report { overall_health: HEALTHY } } }
I20250109 08:58:57.656834 25053 raft_consensus.cc:988] T 970b2f8b130d431cb25d79d33d5f0e5c P 08c70c9a6a064d40bfa68fd105d86d86: : Instructing follower 1a9b5a4afd63410b813b41edbfe69e28 to start an election
I20250109 08:58:57.657249 24926 raft_consensus.cc:1076] T 970b2f8b130d431cb25d79d33d5f0e5c P 08c70c9a6a064d40bfa68fd105d86d86 [term 2 LEADER]: Signalling peer 1a9b5a4afd63410b813b41edbfe69e28 to start an election
I20250109 08:58:57.658562 24768 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "970b2f8b130d431cb25d79d33d5f0e5c"
dest_uuid: "1a9b5a4afd63410b813b41edbfe69e28"
from {username='slave'} at 127.0.0.1:37354
I20250109 08:58:57.659011 24768 raft_consensus.cc:491] T 970b2f8b130d431cb25d79d33d5f0e5c P 1a9b5a4afd63410b813b41edbfe69e28 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:58:57.659277 24768 raft_consensus.cc:3054] T 970b2f8b130d431cb25d79d33d5f0e5c P 1a9b5a4afd63410b813b41edbfe69e28 [term 2 FOLLOWER]: Advancing to term 3
I20250109 08:58:57.663897 24768 raft_consensus.cc:513] T 970b2f8b130d431cb25d79d33d5f0e5c P 1a9b5a4afd63410b813b41edbfe69e28 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:57.665460 24768 leader_election.cc:290] T 970b2f8b130d431cb25d79d33d5f0e5c P 1a9b5a4afd63410b813b41edbfe69e28 [CANDIDATE]: Term 3 election: Requested vote from peers 08c70c9a6a064d40bfa68fd105d86d86 (127.19.226.193:43567), f366ff6c37a24eaaa6abcafab1467912 (127.19.226.194:43355)
I20250109 08:58:57.666479 24693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "970b2f8b130d431cb25d79d33d5f0e5c" candidate_uuid: "1a9b5a4afd63410b813b41edbfe69e28" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f366ff6c37a24eaaa6abcafab1467912"
I20250109 08:58:57.666815 24617 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "970b2f8b130d431cb25d79d33d5f0e5c" candidate_uuid: "1a9b5a4afd63410b813b41edbfe69e28" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "08c70c9a6a064d40bfa68fd105d86d86"
I20250109 08:58:57.667135 24693 raft_consensus.cc:3054] T 970b2f8b130d431cb25d79d33d5f0e5c P f366ff6c37a24eaaa6abcafab1467912 [term 2 FOLLOWER]: Advancing to term 3
I20250109 08:58:57.667395 24617 raft_consensus.cc:3049] T 970b2f8b130d431cb25d79d33d5f0e5c P 08c70c9a6a064d40bfa68fd105d86d86 [term 2 LEADER]: Stepping down as leader of term 2
I20250109 08:58:57.667722 24617 raft_consensus.cc:738] T 970b2f8b130d431cb25d79d33d5f0e5c P 08c70c9a6a064d40bfa68fd105d86d86 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 08c70c9a6a064d40bfa68fd105d86d86, State: Running, Role: LEADER
I20250109 08:58:57.668445 24617 consensus_queue.cc:260] T 970b2f8b130d431cb25d79d33d5f0e5c P 08c70c9a6a064d40bfa68fd105d86d86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:57.669572 24617 raft_consensus.cc:3054] T 970b2f8b130d431cb25d79d33d5f0e5c P 08c70c9a6a064d40bfa68fd105d86d86 [term 2 FOLLOWER]: Advancing to term 3
I20250109 08:58:57.672053 24693 raft_consensus.cc:2463] T 970b2f8b130d431cb25d79d33d5f0e5c P f366ff6c37a24eaaa6abcafab1467912 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a9b5a4afd63410b813b41edbfe69e28 in term 3.
I20250109 08:58:57.672971 24732 leader_election.cc:304] T 970b2f8b130d431cb25d79d33d5f0e5c P 1a9b5a4afd63410b813b41edbfe69e28 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a9b5a4afd63410b813b41edbfe69e28, f366ff6c37a24eaaa6abcafab1467912; no voters:
I20250109 08:58:57.673589 25040 raft_consensus.cc:2798] T 970b2f8b130d431cb25d79d33d5f0e5c P 1a9b5a4afd63410b813b41edbfe69e28 [term 3 FOLLOWER]: Leader election won for term 3
I20250109 08:58:57.674038 25040 raft_consensus.cc:695] T 970b2f8b130d431cb25d79d33d5f0e5c P 1a9b5a4afd63410b813b41edbfe69e28 [term 3 LEADER]: Becoming Leader. State: Replica: 1a9b5a4afd63410b813b41edbfe69e28, State: Running, Role: LEADER
I20250109 08:58:57.674281 24617 raft_consensus.cc:2463] T 970b2f8b130d431cb25d79d33d5f0e5c P 08c70c9a6a064d40bfa68fd105d86d86 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a9b5a4afd63410b813b41edbfe69e28 in term 3.
I20250109 08:58:57.674836 25040 consensus_queue.cc:237] T 970b2f8b130d431cb25d79d33d5f0e5c P 1a9b5a4afd63410b813b41edbfe69e28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:57.681708 24517 catalog_manager.cc:5526] T 970b2f8b130d431cb25d79d33d5f0e5c P 1a9b5a4afd63410b813b41edbfe69e28 reported cstate change: term changed from 2 to 3, leader changed from 08c70c9a6a064d40bfa68fd105d86d86 (127.19.226.193) to 1a9b5a4afd63410b813b41edbfe69e28 (127.19.226.195). New cstate: current_term: 3 leader_uuid: "1a9b5a4afd63410b813b41edbfe69e28" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } health_report { overall_health: HEALTHY } } }
I20250109 08:58:57.890755 25053 raft_consensus.cc:988] T a3207340610143c7bfb4943c87dc78bb P 08c70c9a6a064d40bfa68fd105d86d86: : Instructing follower 1a9b5a4afd63410b813b41edbfe69e28 to start an election
I20250109 08:58:57.891155 24926 raft_consensus.cc:1076] T a3207340610143c7bfb4943c87dc78bb P 08c70c9a6a064d40bfa68fd105d86d86 [term 1 LEADER]: Signalling peer 1a9b5a4afd63410b813b41edbfe69e28 to start an election
I20250109 08:58:57.892477 24768 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "a3207340610143c7bfb4943c87dc78bb"
dest_uuid: "1a9b5a4afd63410b813b41edbfe69e28"
from {username='slave'} at 127.0.0.1:37354
I20250109 08:58:57.892956 24768 raft_consensus.cc:491] T a3207340610143c7bfb4943c87dc78bb P 1a9b5a4afd63410b813b41edbfe69e28 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:58:57.893240 24768 raft_consensus.cc:3054] T a3207340610143c7bfb4943c87dc78bb P 1a9b5a4afd63410b813b41edbfe69e28 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:57.897372 24693 raft_consensus.cc:1270] T 53eef9db17b548f081a60a39a43e4f19 P f366ff6c37a24eaaa6abcafab1467912 [term 3 FOLLOWER]: Refusing update from remote peer 1a9b5a4afd63410b813b41edbfe69e28: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20250109 08:58:57.898208 24768 raft_consensus.cc:513] T a3207340610143c7bfb4943c87dc78bb P 1a9b5a4afd63410b813b41edbfe69e28 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:57.898689 25040 consensus_queue.cc:1035] T 53eef9db17b548f081a60a39a43e4f19 P 1a9b5a4afd63410b813b41edbfe69e28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250109 08:58:57.899917 24768 leader_election.cc:290] T a3207340610143c7bfb4943c87dc78bb P 1a9b5a4afd63410b813b41edbfe69e28 [CANDIDATE]: Term 2 election: Requested vote from peers 08c70c9a6a064d40bfa68fd105d86d86 (127.19.226.193:43567), f366ff6c37a24eaaa6abcafab1467912 (127.19.226.194:43355)
I20250109 08:58:57.900707 24617 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3207340610143c7bfb4943c87dc78bb" candidate_uuid: "1a9b5a4afd63410b813b41edbfe69e28" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "08c70c9a6a064d40bfa68fd105d86d86"
I20250109 08:58:57.901299 24617 raft_consensus.cc:3049] T a3207340610143c7bfb4943c87dc78bb P 08c70c9a6a064d40bfa68fd105d86d86 [term 1 LEADER]: Stepping down as leader of term 1
I20250109 08:58:57.901300 24695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3207340610143c7bfb4943c87dc78bb" candidate_uuid: "1a9b5a4afd63410b813b41edbfe69e28" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "f366ff6c37a24eaaa6abcafab1467912"
I20250109 08:58:57.901641 24617 raft_consensus.cc:738] T a3207340610143c7bfb4943c87dc78bb P 08c70c9a6a064d40bfa68fd105d86d86 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 08c70c9a6a064d40bfa68fd105d86d86, State: Running, Role: LEADER
I20250109 08:58:57.902065 24695 raft_consensus.cc:3054] T a3207340610143c7bfb4943c87dc78bb P f366ff6c37a24eaaa6abcafab1467912 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:57.902541 24617 consensus_queue.cc:260] T a3207340610143c7bfb4943c87dc78bb P 08c70c9a6a064d40bfa68fd105d86d86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:57.903702 24617 raft_consensus.cc:3054] T a3207340610143c7bfb4943c87dc78bb P 08c70c9a6a064d40bfa68fd105d86d86 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:58:57.908980 24695 raft_consensus.cc:2463] T a3207340610143c7bfb4943c87dc78bb P f366ff6c37a24eaaa6abcafab1467912 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a9b5a4afd63410b813b41edbfe69e28 in term 2.
I20250109 08:58:57.910696 24732 leader_election.cc:304] T a3207340610143c7bfb4943c87dc78bb P 1a9b5a4afd63410b813b41edbfe69e28 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a9b5a4afd63410b813b41edbfe69e28, f366ff6c37a24eaaa6abcafab1467912; no voters:
I20250109 08:58:57.911548 25040 raft_consensus.cc:2798] T a3207340610143c7bfb4943c87dc78bb P 1a9b5a4afd63410b813b41edbfe69e28 [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:58:57.914639 24617 raft_consensus.cc:2463] T a3207340610143c7bfb4943c87dc78bb P 08c70c9a6a064d40bfa68fd105d86d86 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a9b5a4afd63410b813b41edbfe69e28 in term 2.
I20250109 08:58:57.921157 25040 raft_consensus.cc:695] T a3207340610143c7bfb4943c87dc78bb P 1a9b5a4afd63410b813b41edbfe69e28 [term 2 LEADER]: Becoming Leader. State: Replica: 1a9b5a4afd63410b813b41edbfe69e28, State: Running, Role: LEADER
I20250109 08:58:57.922132 25040 consensus_queue.cc:237] T a3207340610143c7bfb4943c87dc78bb P 1a9b5a4afd63410b813b41edbfe69e28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } }
I20250109 08:58:57.922873 24617 raft_consensus.cc:1270] T 53eef9db17b548f081a60a39a43e4f19 P 08c70c9a6a064d40bfa68fd105d86d86 [term 3 FOLLOWER]: Refusing update from remote peer 1a9b5a4afd63410b813b41edbfe69e28: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20250109 08:58:57.924304 25064 consensus_queue.cc:1035] T 53eef9db17b548f081a60a39a43e4f19 P 1a9b5a4afd63410b813b41edbfe69e28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:58:57.927330 25031 tablet.cc:1721] T 53eef9db17b548f081a60a39a43e4f19 P 08c70c9a6a064d40bfa68fd105d86d86: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
I20250109 08:58:57.930631 24517 catalog_manager.cc:5526] T a3207340610143c7bfb4943c87dc78bb P 1a9b5a4afd63410b813b41edbfe69e28 reported cstate change: term changed from 1 to 2, leader changed from 08c70c9a6a064d40bfa68fd105d86d86 (127.19.226.193) to 1a9b5a4afd63410b813b41edbfe69e28 (127.19.226.195). New cstate: current_term: 2 leader_uuid: "1a9b5a4afd63410b813b41edbfe69e28" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a9b5a4afd63410b813b41edbfe69e28" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 42529 } health_report { overall_health: HEALTHY } } }
I20250109 08:58:57.935096 24515 catalog_manager.cc:5618] TS 08c70c9a6a064d40bfa68fd105d86d86 (127.19.226.193:43567) does not have the latest schema for tablet 53eef9db17b548f081a60a39a43e4f19 (table soft_deleted_table [id=f05587e8b44d44c898cd6984d831bfbb]). Expected version 1 got 0
I20250109 08:58:58.036254 24617 raft_consensus.cc:1270] T 98383febcb2145208b05b96598aed748 P 08c70c9a6a064d40bfa68fd105d86d86 [term 2 FOLLOWER]: Refusing update from remote peer 1a9b5a4afd63410b813b41edbfe69e28: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:58:58.037518 25064 consensus_queue.cc:1035] T 98383febcb2145208b05b96598aed748 P 1a9b5a4afd63410b813b41edbfe69e28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:58:58.045455 24695 raft_consensus.cc:1270] T 98383febcb2145208b05b96598aed748 P f366ff6c37a24eaaa6abcafab1467912 [term 2 FOLLOWER]: Refusing update from remote peer 1a9b5a4afd63410b813b41edbfe69e28: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:58:58.046880 25063 consensus_queue.cc:1035] T 98383febcb2145208b05b96598aed748 P 1a9b5a4afd63410b813b41edbfe69e28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:58:58.153550 24695 raft_consensus.cc:1270] T 970b2f8b130d431cb25d79d33d5f0e5c P f366ff6c37a24eaaa6abcafab1467912 [term 3 FOLLOWER]: Refusing update from remote peer 1a9b5a4afd63410b813b41edbfe69e28: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20250109 08:58:58.154649 25064 consensus_queue.cc:1035] T 970b2f8b130d431cb25d79d33d5f0e5c P 1a9b5a4afd63410b813b41edbfe69e28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:58:58.165577 24617 raft_consensus.cc:1270] T 970b2f8b130d431cb25d79d33d5f0e5c P 08c70c9a6a064d40bfa68fd105d86d86 [term 3 FOLLOWER]: Refusing update from remote peer 1a9b5a4afd63410b813b41edbfe69e28: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20250109 08:58:58.167320 25040 consensus_queue.cc:1035] T 970b2f8b130d431cb25d79d33d5f0e5c P 1a9b5a4afd63410b813b41edbfe69e28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:58:58.302986 24695 raft_consensus.cc:1270] T a3207340610143c7bfb4943c87dc78bb P f366ff6c37a24eaaa6abcafab1467912 [term 2 FOLLOWER]: Refusing update from remote peer 1a9b5a4afd63410b813b41edbfe69e28: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20250109 08:58:58.304248 25040 consensus_queue.cc:1035] T a3207340610143c7bfb4943c87dc78bb P 1a9b5a4afd63410b813b41edbfe69e28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f366ff6c37a24eaaa6abcafab1467912" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 43355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:58:58.312484 24617 raft_consensus.cc:1270] T a3207340610143c7bfb4943c87dc78bb P 08c70c9a6a064d40bfa68fd105d86d86 [term 2 FOLLOWER]: Refusing update from remote peer 1a9b5a4afd63410b813b41edbfe69e28: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20250109 08:58:58.313843 25064 consensus_queue.cc:1035] T a3207340610143c7bfb4943c87dc78bb P 1a9b5a4afd63410b813b41edbfe69e28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "08c70c9a6a064d40bfa68fd105d86d86" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 43567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:58:58.545825 20363 auto_leader_rebalancer.cc:140] leader rebalance for table soft_deleted_table
I20250109 08:58:58.550781 20363 auto_leader_rebalancer.cc:388] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20250109 08:58:58.551163 20363 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250109 08:58:58.555706 20363 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250109 08:58:58.556041 20363 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250109 08:58:59.557845 20363 auto_leader_rebalancer.cc:140] leader rebalance for table soft_deleted_table
I20250109 08:58:59.566947 20363 auto_leader_rebalancer.cc:388] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20250109 08:58:59.567456 20363 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250109 08:58:59.574239 20363 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250109 08:58:59.574610 20363 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250109 08:59:00.575488 20363 auto_leader_rebalancer.cc:140] leader rebalance for table soft_deleted_table
I20250109 08:59:00.581228 20363 auto_leader_rebalancer.cc:388] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20250109 08:59:00.581632 20363 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250109 08:59:00.586162 20363 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250109 08:59:00.586519 20363 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250109 08:59:01.587252 20363 auto_leader_rebalancer.cc:140] leader rebalance for table soft_deleted_table
I20250109 08:59:01.592156 20363 auto_leader_rebalancer.cc:388] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20250109 08:59:01.592739 20363 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250109 08:59:01.597361 20363 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250109 08:59:01.597728 20363 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250109 08:59:02.598532 20363 auto_leader_rebalancer.cc:140] leader rebalance for table soft_deleted_table
I20250109 08:59:02.604663 20363 auto_leader_rebalancer.cc:388] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20250109 08:59:02.605093 20363 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250109 08:59:02.609803 20363 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250109 08:59:02.610185 20363 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250109 08:59:03.611131 20363 auto_leader_rebalancer.cc:140] leader rebalance for table soft_deleted_table
I20250109 08:59:03.616804 20363 auto_leader_rebalancer.cc:388] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20250109 08:59:03.617228 20363 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250109 08:59:03.621765 20363 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250109 08:59:03.622131 20363 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250109 08:59:04.622998 20363 auto_leader_rebalancer.cc:140] leader rebalance for table soft_deleted_table
I20250109 08:59:04.629215 20363 auto_leader_rebalancer.cc:388] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20250109 08:59:04.629683 20363 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250109 08:59:04.634899 20363 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250109 08:59:04.635279 20363 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250109 08:59:05.636015 20363 auto_leader_rebalancer.cc:140] leader rebalance for table soft_deleted_table
I20250109 08:59:05.642338 20363 auto_leader_rebalancer.cc:388] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20250109 08:59:05.642877 20363 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250109 08:59:05.647468 20363 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250109 08:59:05.647850 20363 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250109 08:59:06.648795 20363 auto_leader_rebalancer.cc:140] leader rebalance for table soft_deleted_table
I20250109 08:59:06.654587 20363 auto_leader_rebalancer.cc:388] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20250109 08:59:06.655192 20363 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250109 08:59:06.660193 20363 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250109 08:59:06.660612 20363 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250109 08:59:07.716190 20363 tablet_server.cc:178] TabletServer@127.19.226.193:0 shutting down...
I20250109 08:59:07.748931 20363 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250109 08:59:07.749898 20363 tablet_replica.cc:331] T d8c90027b99f41b78f3b41c079e77ed7 P 08c70c9a6a064d40bfa68fd105d86d86: stopping tablet replica
I20250109 08:59:07.750638 20363 raft_consensus.cc:2238] T d8c90027b99f41b78f3b41c079e77ed7 P 08c70c9a6a064d40bfa68fd105d86d86 [term 2 LEADER]: Raft consensus shutting down.
I20250109 08:59:07.751641 20363 raft_consensus.cc:2267] T d8c90027b99f41b78f3b41c079e77ed7 P 08c70c9a6a064d40bfa68fd105d86d86 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:07.753876 20363 tablet_replica.cc:331] T 53eef9db17b548f081a60a39a43e4f19 P 08c70c9a6a064d40bfa68fd105d86d86: stopping tablet replica
I20250109 08:59:07.754495 20363 raft_consensus.cc:2238] T 53eef9db17b548f081a60a39a43e4f19 P 08c70c9a6a064d40bfa68fd105d86d86 [term 3 FOLLOWER]: Raft consensus shutting down.
I20250109 08:59:07.755002 20363 raft_consensus.cc:2267] T 53eef9db17b548f081a60a39a43e4f19 P 08c70c9a6a064d40bfa68fd105d86d86 [term 3 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:07.757885 20363 tablet_replica.cc:331] T 9e0af0555b484b748a03f99b45c463ef P 08c70c9a6a064d40bfa68fd105d86d86: stopping tablet replica
I20250109 08:59:07.758463 20363 raft_consensus.cc:2238] T 9e0af0555b484b748a03f99b45c463ef P 08c70c9a6a064d40bfa68fd105d86d86 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:59:07.759142 20363 raft_consensus.cc:2267] T 9e0af0555b484b748a03f99b45c463ef P 08c70c9a6a064d40bfa68fd105d86d86 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:07.761350 20363 tablet_replica.cc:331] T ff0a7cf17ff34eedbdda1e5a4fac0dfa P 08c70c9a6a064d40bfa68fd105d86d86: stopping tablet replica
I20250109 08:59:07.762107 20363 raft_consensus.cc:2238] T ff0a7cf17ff34eedbdda1e5a4fac0dfa P 08c70c9a6a064d40bfa68fd105d86d86 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:59:07.762616 20363 raft_consensus.cc:2267] T ff0a7cf17ff34eedbdda1e5a4fac0dfa P 08c70c9a6a064d40bfa68fd105d86d86 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:07.764901 20363 tablet_replica.cc:331] T 2141d5ba289f4dd99a8b0c4c1d11c66a P 08c70c9a6a064d40bfa68fd105d86d86: stopping tablet replica
I20250109 08:59:07.765463 20363 raft_consensus.cc:2238] T 2141d5ba289f4dd99a8b0c4c1d11c66a P 08c70c9a6a064d40bfa68fd105d86d86 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:59:07.765965 20363 raft_consensus.cc:2267] T 2141d5ba289f4dd99a8b0c4c1d11c66a P 08c70c9a6a064d40bfa68fd105d86d86 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:07.768059 20363 tablet_replica.cc:331] T 0364f388e25b466b8b39fdaa50aebf42 P 08c70c9a6a064d40bfa68fd105d86d86: stopping tablet replica
I20250109 08:59:07.768641 20363 raft_consensus.cc:2238] T 0364f388e25b466b8b39fdaa50aebf42 P 08c70c9a6a064d40bfa68fd105d86d86 [term 1 LEADER]: Raft consensus shutting down.
I20250109 08:59:07.769495 20363 raft_consensus.cc:2267] T 0364f388e25b466b8b39fdaa50aebf42 P 08c70c9a6a064d40bfa68fd105d86d86 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:07.771842 20363 tablet_replica.cc:331] T 0faa532dc2514e6b9991e3832a029fab P 08c70c9a6a064d40bfa68fd105d86d86: stopping tablet replica
I20250109 08:59:07.772495 20363 raft_consensus.cc:2238] T 0faa532dc2514e6b9991e3832a029fab P 08c70c9a6a064d40bfa68fd105d86d86 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:59:07.773082 20363 raft_consensus.cc:2267] T 0faa532dc2514e6b9991e3832a029fab P 08c70c9a6a064d40bfa68fd105d86d86 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:07.775281 20363 tablet_replica.cc:331] T 4e2136a82d344ae6b0657803f1551906 P 08c70c9a6a064d40bfa68fd105d86d86: stopping tablet replica
I20250109 08:59:07.775800 20363 raft_consensus.cc:2238] T 4e2136a82d344ae6b0657803f1551906 P 08c70c9a6a064d40bfa68fd105d86d86 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:59:07.776262 20363 raft_consensus.cc:2267] T 4e2136a82d344ae6b0657803f1551906 P 08c70c9a6a064d40bfa68fd105d86d86 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:07.778363 20363 tablet_replica.cc:331] T e8d95383d13941a99003d8e54fbed202 P 08c70c9a6a064d40bfa68fd105d86d86: stopping tablet replica
I20250109 08:59:07.778874 20363 raft_consensus.cc:2238] T e8d95383d13941a99003d8e54fbed202 P 08c70c9a6a064d40bfa68fd105d86d86 [term 2 LEADER]: Raft consensus shutting down.
I20250109 08:59:07.779825 20363 raft_consensus.cc:2267] T e8d95383d13941a99003d8e54fbed202 P 08c70c9a6a064d40bfa68fd105d86d86 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:07.782639 20363 tablet_replica.cc:331] T a3207340610143c7bfb4943c87dc78bb P 08c70c9a6a064d40bfa68fd105d86d86: stopping tablet replica
I20250109 08:59:07.783231 20363 raft_consensus.cc:2238] T a3207340610143c7bfb4943c87dc78bb P 08c70c9a6a064d40bfa68fd105d86d86 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:59:07.783696 20363 raft_consensus.cc:2267] T a3207340610143c7bfb4943c87dc78bb P 08c70c9a6a064d40bfa68fd105d86d86 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:07.785795 20363 tablet_replica.cc:331] T c1c586b8d3a64430bf1fe82b27fef388 P 08c70c9a6a064d40bfa68fd105d86d86: stopping tablet replica
I20250109 08:59:07.786319 20363 raft_consensus.cc:2238] T c1c586b8d3a64430bf1fe82b27fef388 P 08c70c9a6a064d40bfa68fd105d86d86 [term 2 LEADER]: Raft consensus shutting down.
I20250109 08:59:07.787163 20363 raft_consensus.cc:2267] T c1c586b8d3a64430bf1fe82b27fef388 P 08c70c9a6a064d40bfa68fd105d86d86 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:07.789448 20363 tablet_replica.cc:331] T 630fe12978194fb9ae9f60597b3ce1c1 P 08c70c9a6a064d40bfa68fd105d86d86: stopping tablet replica
I20250109 08:59:07.790213 20363 raft_consensus.cc:2238] T 630fe12978194fb9ae9f60597b3ce1c1 P 08c70c9a6a064d40bfa68fd105d86d86 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:59:07.790704 20363 raft_consensus.cc:2267] T 630fe12978194fb9ae9f60597b3ce1c1 P 08c70c9a6a064d40bfa68fd105d86d86 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:07.793439 20363 tablet_replica.cc:331] T d8f9f0e755da4a7186c2c9316918ca73 P 08c70c9a6a064d40bfa68fd105d86d86: stopping tablet replica
I20250109 08:59:07.794082 20363 raft_consensus.cc:2238] T d8f9f0e755da4a7186c2c9316918ca73 P 08c70c9a6a064d40bfa68fd105d86d86 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:59:07.794610 20363 raft_consensus.cc:2267] T d8f9f0e755da4a7186c2c9316918ca73 P 08c70c9a6a064d40bfa68fd105d86d86 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:07.796653 20363 tablet_replica.cc:331] T 970b2f8b130d431cb25d79d33d5f0e5c P 08c70c9a6a064d40bfa68fd105d86d86: stopping tablet replica
I20250109 08:59:07.797152 20363 raft_consensus.cc:2238] T 970b2f8b130d431cb25d79d33d5f0e5c P 08c70c9a6a064d40bfa68fd105d86d86 [term 3 FOLLOWER]: Raft consensus shutting down.
I20250109 08:59:07.797603 20363 raft_consensus.cc:2267] T 970b2f8b130d431cb25d79d33d5f0e5c P 08c70c9a6a064d40bfa68fd105d86d86 [term 3 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:07.799536 20363 tablet_replica.cc:331] T 0d92161e8e3f4a9c88b0ca5fa5555a2c P 08c70c9a6a064d40bfa68fd105d86d86: stopping tablet replica
I20250109 08:59:07.800045 20363 raft_consensus.cc:2238] T 0d92161e8e3f4a9c88b0ca5fa5555a2c P 08c70c9a6a064d40bfa68fd105d86d86 [term 1 LEADER]: Raft consensus shutting down.
I20250109 08:59:07.800997 20363 raft_consensus.cc:2267] T 0d92161e8e3f4a9c88b0ca5fa5555a2c P 08c70c9a6a064d40bfa68fd105d86d86 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:07.803154 20363 tablet_replica.cc:331] T 98383febcb2145208b05b96598aed748 P 08c70c9a6a064d40bfa68fd105d86d86: stopping tablet replica
I20250109 08:59:07.803663 20363 raft_consensus.cc:2238] T 98383febcb2145208b05b96598aed748 P 08c70c9a6a064d40bfa68fd105d86d86 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:59:07.804183 20363 raft_consensus.cc:2267] T 98383febcb2145208b05b96598aed748 P 08c70c9a6a064d40bfa68fd105d86d86 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:07.806262 20363 tablet_replica.cc:331] T 6e2f4cbbf3f744e586058426df0287c2 P 08c70c9a6a064d40bfa68fd105d86d86: stopping tablet replica
I20250109 08:59:07.806761 20363 raft_consensus.cc:2238] T 6e2f4cbbf3f744e586058426df0287c2 P 08c70c9a6a064d40bfa68fd105d86d86 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:59:07.807192 20363 raft_consensus.cc:2267] T 6e2f4cbbf3f744e586058426df0287c2 P 08c70c9a6a064d40bfa68fd105d86d86 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:07.809587 20363 tablet_replica.cc:331] T a930173474344b2c93cc8ac295a74729 P 08c70c9a6a064d40bfa68fd105d86d86: stopping tablet replica
I20250109 08:59:07.810050 20363 raft_consensus.cc:2238] T a930173474344b2c93cc8ac295a74729 P 08c70c9a6a064d40bfa68fd105d86d86 [term 1 LEADER]: Raft consensus shutting down.
I20250109 08:59:07.810932 20363 raft_consensus.cc:2267] T a930173474344b2c93cc8ac295a74729 P 08c70c9a6a064d40bfa68fd105d86d86 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:07.858830 20363 tablet_server.cc:195] TabletServer@127.19.226.193:0 shutdown complete.
W20250109 08:59:07.859458 24735 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.226.193:43567: connect: Connection refused (error 111) [suppressed 82 similar messages]
W20250109 08:59:07.863732 24735 consensus_peers.cc:487] T 98383febcb2145208b05b96598aed748 P 1a9b5a4afd63410b813b41edbfe69e28 -> Peer 08c70c9a6a064d40bfa68fd105d86d86 (127.19.226.193:43567): Couldn't send request to peer 08c70c9a6a064d40bfa68fd105d86d86. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.193:43567: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250109 08:59:07.901875 20363 tablet_server.cc:178] TabletServer@127.19.226.194:0 shutting down...
W20250109 08:59:07.910887 24735 consensus_peers.cc:487] T a3207340610143c7bfb4943c87dc78bb P 1a9b5a4afd63410b813b41edbfe69e28 -> Peer 08c70c9a6a064d40bfa68fd105d86d86 (127.19.226.193:43567): Couldn't send request to peer 08c70c9a6a064d40bfa68fd105d86d86. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.193:43567: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20250109 08:59:07.931066 24735 consensus_peers.cc:487] T 53eef9db17b548f081a60a39a43e4f19 P 1a9b5a4afd63410b813b41edbfe69e28 -> Peer 08c70c9a6a064d40bfa68fd105d86d86 (127.19.226.193:43567): Couldn't send request to peer 08c70c9a6a064d40bfa68fd105d86d86. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.193:43567: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20250109 08:59:07.942126 24661 consensus_peers.cc:487] T ff0a7cf17ff34eedbdda1e5a4fac0dfa P f366ff6c37a24eaaa6abcafab1467912 -> Peer 08c70c9a6a064d40bfa68fd105d86d86 (127.19.226.193:43567): Couldn't send request to peer 08c70c9a6a064d40bfa68fd105d86d86. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.193:43567: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250109 08:59:07.947283 20363 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250109 08:59:07.948262 20363 tablet_replica.cc:331] T ff0a7cf17ff34eedbdda1e5a4fac0dfa P f366ff6c37a24eaaa6abcafab1467912: stopping tablet replica
I20250109 08:59:07.949086 20363 raft_consensus.cc:2238] T ff0a7cf17ff34eedbdda1e5a4fac0dfa P f366ff6c37a24eaaa6abcafab1467912 [term 2 LEADER]: Raft consensus shutting down.
I20250109 08:59:07.950011 20363 raft_consensus.cc:2267] T ff0a7cf17ff34eedbdda1e5a4fac0dfa P f366ff6c37a24eaaa6abcafab1467912 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:07.952217 20363 tablet_replica.cc:331] T a3207340610143c7bfb4943c87dc78bb P f366ff6c37a24eaaa6abcafab1467912: stopping tablet replica
I20250109 08:59:07.952858 20363 raft_consensus.cc:2238] T a3207340610143c7bfb4943c87dc78bb P f366ff6c37a24eaaa6abcafab1467912 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:59:07.953362 20363 raft_consensus.cc:2267] T a3207340610143c7bfb4943c87dc78bb P f366ff6c37a24eaaa6abcafab1467912 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:07.955466 20363 tablet_replica.cc:331] T 9e0af0555b484b748a03f99b45c463ef P f366ff6c37a24eaaa6abcafab1467912: stopping tablet replica
I20250109 08:59:07.955973 20363 raft_consensus.cc:2238] T 9e0af0555b484b748a03f99b45c463ef P f366ff6c37a24eaaa6abcafab1467912 [term 1 LEADER]: Raft consensus shutting down.
I20250109 08:59:07.956804 20363 raft_consensus.cc:2267] T 9e0af0555b484b748a03f99b45c463ef P f366ff6c37a24eaaa6abcafab1467912 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:07.958827 20363 tablet_replica.cc:331] T 53eef9db17b548f081a60a39a43e4f19 P f366ff6c37a24eaaa6abcafab1467912: stopping tablet replica
I20250109 08:59:07.959451 20363 raft_consensus.cc:2238] T 53eef9db17b548f081a60a39a43e4f19 P f366ff6c37a24eaaa6abcafab1467912 [term 3 FOLLOWER]: Raft consensus shutting down.
I20250109 08:59:07.959939 20363 raft_consensus.cc:2267] T 53eef9db17b548f081a60a39a43e4f19 P f366ff6c37a24eaaa6abcafab1467912 [term 3 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:07.962292 20363 tablet_replica.cc:331] T 2141d5ba289f4dd99a8b0c4c1d11c66a P f366ff6c37a24eaaa6abcafab1467912: stopping tablet replica
I20250109 08:59:07.963032 20363 raft_consensus.cc:2238] T 2141d5ba289f4dd99a8b0c4c1d11c66a P f366ff6c37a24eaaa6abcafab1467912 [term 2 LEADER]: Raft consensus shutting down.
I20250109 08:59:07.964105 20363 raft_consensus.cc:2267] T 2141d5ba289f4dd99a8b0c4c1d11c66a P f366ff6c37a24eaaa6abcafab1467912 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:07.966328 20363 tablet_replica.cc:331] T 0364f388e25b466b8b39fdaa50aebf42 P f366ff6c37a24eaaa6abcafab1467912: stopping tablet replica
I20250109 08:59:07.966918 20363 raft_consensus.cc:2238] T 0364f388e25b466b8b39fdaa50aebf42 P f366ff6c37a24eaaa6abcafab1467912 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:59:07.967361 20363 raft_consensus.cc:2267] T 0364f388e25b466b8b39fdaa50aebf42 P f366ff6c37a24eaaa6abcafab1467912 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:07.969444 20363 tablet_replica.cc:331] T 0faa532dc2514e6b9991e3832a029fab P f366ff6c37a24eaaa6abcafab1467912: stopping tablet replica
I20250109 08:59:07.969928 20363 raft_consensus.cc:2238] T 0faa532dc2514e6b9991e3832a029fab P f366ff6c37a24eaaa6abcafab1467912 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:59:07.970352 20363 raft_consensus.cc:2267] T 0faa532dc2514e6b9991e3832a029fab P f366ff6c37a24eaaa6abcafab1467912 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:07.972970 20363 tablet_replica.cc:331] T e8d95383d13941a99003d8e54fbed202 P f366ff6c37a24eaaa6abcafab1467912: stopping tablet replica
I20250109 08:59:07.973589 20363 raft_consensus.cc:2238] T e8d95383d13941a99003d8e54fbed202 P f366ff6c37a24eaaa6abcafab1467912 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:59:07.974119 20363 raft_consensus.cc:2267] T e8d95383d13941a99003d8e54fbed202 P f366ff6c37a24eaaa6abcafab1467912 [term 2 FOLLOWER]: Raft consensus is shut down!
W20250109 08:59:07.974946 24735 consensus_peers.cc:487] T 970b2f8b130d431cb25d79d33d5f0e5c P 1a9b5a4afd63410b813b41edbfe69e28 -> Peer 08c70c9a6a064d40bfa68fd105d86d86 (127.19.226.193:43567): Couldn't send request to peer 08c70c9a6a064d40bfa68fd105d86d86. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.193:43567: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250109 08:59:07.976305 20363 tablet_replica.cc:331] T 4e2136a82d344ae6b0657803f1551906 P f366ff6c37a24eaaa6abcafab1467912: stopping tablet replica
I20250109 08:59:07.976904 20363 raft_consensus.cc:2238] T 4e2136a82d344ae6b0657803f1551906 P f366ff6c37a24eaaa6abcafab1467912 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:59:07.977298 20363 raft_consensus.cc:2267] T 4e2136a82d344ae6b0657803f1551906 P f366ff6c37a24eaaa6abcafab1467912 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:07.979454 20363 tablet_replica.cc:331] T d8f9f0e755da4a7186c2c9316918ca73 P f366ff6c37a24eaaa6abcafab1467912: stopping tablet replica
I20250109 08:59:07.979972 20363 raft_consensus.cc:2238] T d8f9f0e755da4a7186c2c9316918ca73 P f366ff6c37a24eaaa6abcafab1467912 [term 1 LEADER]: Raft consensus shutting down.
I20250109 08:59:07.980859 20363 raft_consensus.cc:2267] T d8f9f0e755da4a7186c2c9316918ca73 P f366ff6c37a24eaaa6abcafab1467912 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:07.983101 20363 tablet_replica.cc:331] T 630fe12978194fb9ae9f60597b3ce1c1 P f366ff6c37a24eaaa6abcafab1467912: stopping tablet replica
I20250109 08:59:07.983635 20363 raft_consensus.cc:2238] T 630fe12978194fb9ae9f60597b3ce1c1 P f366ff6c37a24eaaa6abcafab1467912 [term 2 LEADER]: Raft consensus shutting down.
I20250109 08:59:07.985035 20363 raft_consensus.cc:2267] T 630fe12978194fb9ae9f60597b3ce1c1 P f366ff6c37a24eaaa6abcafab1467912 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:07.987704 20363 tablet_replica.cc:331] T 98383febcb2145208b05b96598aed748 P f366ff6c37a24eaaa6abcafab1467912: stopping tablet replica
I20250109 08:59:07.988401 20363 raft_consensus.cc:2238] T 98383febcb2145208b05b96598aed748 P f366ff6c37a24eaaa6abcafab1467912 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:59:07.988891 20363 raft_consensus.cc:2267] T 98383febcb2145208b05b96598aed748 P f366ff6c37a24eaaa6abcafab1467912 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:07.990844 20363 tablet_replica.cc:331] T c1c586b8d3a64430bf1fe82b27fef388 P f366ff6c37a24eaaa6abcafab1467912: stopping tablet replica
I20250109 08:59:07.991323 20363 raft_consensus.cc:2238] T c1c586b8d3a64430bf1fe82b27fef388 P f366ff6c37a24eaaa6abcafab1467912 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:59:07.991753 20363 raft_consensus.cc:2267] T c1c586b8d3a64430bf1fe82b27fef388 P f366ff6c37a24eaaa6abcafab1467912 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:07.994266 20363 tablet_replica.cc:331] T d8c90027b99f41b78f3b41c079e77ed7 P f366ff6c37a24eaaa6abcafab1467912: stopping tablet replica
I20250109 08:59:07.994746 20363 raft_consensus.cc:2238] T d8c90027b99f41b78f3b41c079e77ed7 P f366ff6c37a24eaaa6abcafab1467912 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:59:07.995216 20363 raft_consensus.cc:2267] T d8c90027b99f41b78f3b41c079e77ed7 P f366ff6c37a24eaaa6abcafab1467912 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:07.997213 20363 tablet_replica.cc:331] T 970b2f8b130d431cb25d79d33d5f0e5c P f366ff6c37a24eaaa6abcafab1467912: stopping tablet replica
I20250109 08:59:07.997702 20363 raft_consensus.cc:2238] T 970b2f8b130d431cb25d79d33d5f0e5c P f366ff6c37a24eaaa6abcafab1467912 [term 3 FOLLOWER]: Raft consensus shutting down.
I20250109 08:59:07.998178 20363 raft_consensus.cc:2267] T 970b2f8b130d431cb25d79d33d5f0e5c P f366ff6c37a24eaaa6abcafab1467912 [term 3 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:08.000133 20363 tablet_replica.cc:331] T 0d92161e8e3f4a9c88b0ca5fa5555a2c P f366ff6c37a24eaaa6abcafab1467912: stopping tablet replica
I20250109 08:59:08.000643 20363 raft_consensus.cc:2238] T 0d92161e8e3f4a9c88b0ca5fa5555a2c P f366ff6c37a24eaaa6abcafab1467912 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:59:08.001092 20363 raft_consensus.cc:2267] T 0d92161e8e3f4a9c88b0ca5fa5555a2c P f366ff6c37a24eaaa6abcafab1467912 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:08.003369 20363 tablet_replica.cc:331] T 6e2f4cbbf3f744e586058426df0287c2 P f366ff6c37a24eaaa6abcafab1467912: stopping tablet replica
I20250109 08:59:08.003823 20363 raft_consensus.cc:2238] T 6e2f4cbbf3f744e586058426df0287c2 P f366ff6c37a24eaaa6abcafab1467912 [term 1 LEADER]: Raft consensus shutting down.
I20250109 08:59:08.004642 20363 raft_consensus.cc:2267] T 6e2f4cbbf3f744e586058426df0287c2 P f366ff6c37a24eaaa6abcafab1467912 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:08.006623 20363 tablet_replica.cc:331] T a930173474344b2c93cc8ac295a74729 P f366ff6c37a24eaaa6abcafab1467912: stopping tablet replica
I20250109 08:59:08.007064 20363 raft_consensus.cc:2238] T a930173474344b2c93cc8ac295a74729 P f366ff6c37a24eaaa6abcafab1467912 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:59:08.007603 20363 raft_consensus.cc:2267] T a930173474344b2c93cc8ac295a74729 P f366ff6c37a24eaaa6abcafab1467912 [term 1 FOLLOWER]: Raft consensus is shut down!
W20250109 08:59:08.039798 24735 consensus_peers.cc:487] T 0faa532dc2514e6b9991e3832a029fab P 1a9b5a4afd63410b813b41edbfe69e28 -> Peer 08c70c9a6a064d40bfa68fd105d86d86 (127.19.226.193:43567): Couldn't send request to peer 08c70c9a6a064d40bfa68fd105d86d86. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.193:43567: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250109 08:59:08.049651 20363 tablet_server.cc:195] TabletServer@127.19.226.194:0 shutdown complete.
I20250109 08:59:08.083606 20363 tablet_server.cc:178] TabletServer@127.19.226.195:0 shutting down...
I20250109 08:59:08.109313 20363 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250109 08:59:08.110081 20363 tablet_replica.cc:331] T d8c90027b99f41b78f3b41c079e77ed7 P 1a9b5a4afd63410b813b41edbfe69e28: stopping tablet replica
I20250109 08:59:08.110797 20363 raft_consensus.cc:2238] T d8c90027b99f41b78f3b41c079e77ed7 P 1a9b5a4afd63410b813b41edbfe69e28 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:59:08.111384 20363 raft_consensus.cc:2267] T d8c90027b99f41b78f3b41c079e77ed7 P 1a9b5a4afd63410b813b41edbfe69e28 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:08.113487 20363 tablet_replica.cc:331] T 53eef9db17b548f081a60a39a43e4f19 P 1a9b5a4afd63410b813b41edbfe69e28: stopping tablet replica
I20250109 08:59:08.114035 20363 raft_consensus.cc:2238] T 53eef9db17b548f081a60a39a43e4f19 P 1a9b5a4afd63410b813b41edbfe69e28 [term 3 LEADER]: Raft consensus shutting down.
I20250109 08:59:08.114837 20363 raft_consensus.cc:2267] T 53eef9db17b548f081a60a39a43e4f19 P 1a9b5a4afd63410b813b41edbfe69e28 [term 3 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:08.117033 20363 tablet_replica.cc:331] T 9e0af0555b484b748a03f99b45c463ef P 1a9b5a4afd63410b813b41edbfe69e28: stopping tablet replica
I20250109 08:59:08.117636 20363 raft_consensus.cc:2238] T 9e0af0555b484b748a03f99b45c463ef P 1a9b5a4afd63410b813b41edbfe69e28 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:59:08.118093 20363 raft_consensus.cc:2267] T 9e0af0555b484b748a03f99b45c463ef P 1a9b5a4afd63410b813b41edbfe69e28 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:08.119966 20363 tablet_replica.cc:331] T ff0a7cf17ff34eedbdda1e5a4fac0dfa P 1a9b5a4afd63410b813b41edbfe69e28: stopping tablet replica
I20250109 08:59:08.120556 20363 raft_consensus.cc:2238] T ff0a7cf17ff34eedbdda1e5a4fac0dfa P 1a9b5a4afd63410b813b41edbfe69e28 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:59:08.121009 20363 raft_consensus.cc:2267] T ff0a7cf17ff34eedbdda1e5a4fac0dfa P 1a9b5a4afd63410b813b41edbfe69e28 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:08.122956 20363 tablet_replica.cc:331] T 2141d5ba289f4dd99a8b0c4c1d11c66a P 1a9b5a4afd63410b813b41edbfe69e28: stopping tablet replica
I20250109 08:59:08.123489 20363 raft_consensus.cc:2238] T 2141d5ba289f4dd99a8b0c4c1d11c66a P 1a9b5a4afd63410b813b41edbfe69e28 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:59:08.123939 20363 raft_consensus.cc:2267] T 2141d5ba289f4dd99a8b0c4c1d11c66a P 1a9b5a4afd63410b813b41edbfe69e28 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:08.125813 20363 tablet_replica.cc:331] T 0364f388e25b466b8b39fdaa50aebf42 P 1a9b5a4afd63410b813b41edbfe69e28: stopping tablet replica
I20250109 08:59:08.126379 20363 raft_consensus.cc:2238] T 0364f388e25b466b8b39fdaa50aebf42 P 1a9b5a4afd63410b813b41edbfe69e28 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:59:08.126813 20363 raft_consensus.cc:2267] T 0364f388e25b466b8b39fdaa50aebf42 P 1a9b5a4afd63410b813b41edbfe69e28 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:08.128590 20363 tablet_replica.cc:331] T a3207340610143c7bfb4943c87dc78bb P 1a9b5a4afd63410b813b41edbfe69e28: stopping tablet replica
I20250109 08:59:08.129068 20363 raft_consensus.cc:2238] T a3207340610143c7bfb4943c87dc78bb P 1a9b5a4afd63410b813b41edbfe69e28 [term 2 LEADER]: Raft consensus shutting down.
I20250109 08:59:08.129839 20363 raft_consensus.cc:2267] T a3207340610143c7bfb4943c87dc78bb P 1a9b5a4afd63410b813b41edbfe69e28 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:08.131695 20363 tablet_replica.cc:331] T 630fe12978194fb9ae9f60597b3ce1c1 P 1a9b5a4afd63410b813b41edbfe69e28: stopping tablet replica
I20250109 08:59:08.132186 20363 raft_consensus.cc:2238] T 630fe12978194fb9ae9f60597b3ce1c1 P 1a9b5a4afd63410b813b41edbfe69e28 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:59:08.132644 20363 raft_consensus.cc:2267] T 630fe12978194fb9ae9f60597b3ce1c1 P 1a9b5a4afd63410b813b41edbfe69e28 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:08.134451 20363 tablet_replica.cc:331] T d8f9f0e755da4a7186c2c9316918ca73 P 1a9b5a4afd63410b813b41edbfe69e28: stopping tablet replica
I20250109 08:59:08.134955 20363 raft_consensus.cc:2238] T d8f9f0e755da4a7186c2c9316918ca73 P 1a9b5a4afd63410b813b41edbfe69e28 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:59:08.135346 20363 raft_consensus.cc:2267] T d8f9f0e755da4a7186c2c9316918ca73 P 1a9b5a4afd63410b813b41edbfe69e28 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:08.137217 20363 tablet_replica.cc:331] T 6e2f4cbbf3f744e586058426df0287c2 P 1a9b5a4afd63410b813b41edbfe69e28: stopping tablet replica
I20250109 08:59:08.137712 20363 raft_consensus.cc:2238] T 6e2f4cbbf3f744e586058426df0287c2 P 1a9b5a4afd63410b813b41edbfe69e28 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:59:08.138103 20363 raft_consensus.cc:2267] T 6e2f4cbbf3f744e586058426df0287c2 P 1a9b5a4afd63410b813b41edbfe69e28 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:08.140050 20363 tablet_replica.cc:331] T 98383febcb2145208b05b96598aed748 P 1a9b5a4afd63410b813b41edbfe69e28: stopping tablet replica
I20250109 08:59:08.140539 20363 raft_consensus.cc:2238] T 98383febcb2145208b05b96598aed748 P 1a9b5a4afd63410b813b41edbfe69e28 [term 2 LEADER]: Raft consensus shutting down.
I20250109 08:59:08.141283 20363 raft_consensus.cc:2267] T 98383febcb2145208b05b96598aed748 P 1a9b5a4afd63410b813b41edbfe69e28 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:08.143255 20363 tablet_replica.cc:331] T c1c586b8d3a64430bf1fe82b27fef388 P 1a9b5a4afd63410b813b41edbfe69e28: stopping tablet replica
I20250109 08:59:08.143769 20363 raft_consensus.cc:2238] T c1c586b8d3a64430bf1fe82b27fef388 P 1a9b5a4afd63410b813b41edbfe69e28 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:59:08.144295 20363 raft_consensus.cc:2267] T c1c586b8d3a64430bf1fe82b27fef388 P 1a9b5a4afd63410b813b41edbfe69e28 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:08.146541 20363 tablet_replica.cc:331] T a930173474344b2c93cc8ac295a74729 P 1a9b5a4afd63410b813b41edbfe69e28: stopping tablet replica
W20250109 08:59:08.146802 24735 consensus_peers.cc:487] T 4e2136a82d344ae6b0657803f1551906 P 1a9b5a4afd63410b813b41edbfe69e28 -> Peer 08c70c9a6a064d40bfa68fd105d86d86 (127.19.226.193:43567): Couldn't send request to peer 08c70c9a6a064d40bfa68fd105d86d86. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.193:43567: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250109 08:59:08.147114 20363 raft_consensus.cc:2238] T a930173474344b2c93cc8ac295a74729 P 1a9b5a4afd63410b813b41edbfe69e28 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:59:08.147641 20363 raft_consensus.cc:2267] T a930173474344b2c93cc8ac295a74729 P 1a9b5a4afd63410b813b41edbfe69e28 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:08.149538 20363 tablet_replica.cc:331] T 970b2f8b130d431cb25d79d33d5f0e5c P 1a9b5a4afd63410b813b41edbfe69e28: stopping tablet replica
I20250109 08:59:08.150034 20363 raft_consensus.cc:2238] T 970b2f8b130d431cb25d79d33d5f0e5c P 1a9b5a4afd63410b813b41edbfe69e28 [term 3 LEADER]: Raft consensus shutting down.
I20250109 08:59:08.150758 20363 raft_consensus.cc:2267] T 970b2f8b130d431cb25d79d33d5f0e5c P 1a9b5a4afd63410b813b41edbfe69e28 [term 3 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:08.152621 20363 tablet_replica.cc:331] T 0d92161e8e3f4a9c88b0ca5fa5555a2c P 1a9b5a4afd63410b813b41edbfe69e28: stopping tablet replica
I20250109 08:59:08.153157 20363 raft_consensus.cc:2238] T 0d92161e8e3f4a9c88b0ca5fa5555a2c P 1a9b5a4afd63410b813b41edbfe69e28 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250109 08:59:08.153656 20363 raft_consensus.cc:2267] T 0d92161e8e3f4a9c88b0ca5fa5555a2c P 1a9b5a4afd63410b813b41edbfe69e28 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:08.155580 20363 tablet_replica.cc:331] T e8d95383d13941a99003d8e54fbed202 P 1a9b5a4afd63410b813b41edbfe69e28: stopping tablet replica
I20250109 08:59:08.156033 20363 raft_consensus.cc:2238] T e8d95383d13941a99003d8e54fbed202 P 1a9b5a4afd63410b813b41edbfe69e28 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:59:08.156514 20363 raft_consensus.cc:2267] T e8d95383d13941a99003d8e54fbed202 P 1a9b5a4afd63410b813b41edbfe69e28 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:08.158217 20363 tablet_replica.cc:331] T 0faa532dc2514e6b9991e3832a029fab P 1a9b5a4afd63410b813b41edbfe69e28: stopping tablet replica
I20250109 08:59:08.158635 20363 raft_consensus.cc:2238] T 0faa532dc2514e6b9991e3832a029fab P 1a9b5a4afd63410b813b41edbfe69e28 [term 2 LEADER]: Raft consensus shutting down.
I20250109 08:59:08.159385 20363 raft_consensus.cc:2267] T 0faa532dc2514e6b9991e3832a029fab P 1a9b5a4afd63410b813b41edbfe69e28 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:08.161134 20363 tablet_replica.cc:331] T 4e2136a82d344ae6b0657803f1551906 P 1a9b5a4afd63410b813b41edbfe69e28: stopping tablet replica
I20250109 08:59:08.161552 20363 raft_consensus.cc:2238] T 4e2136a82d344ae6b0657803f1551906 P 1a9b5a4afd63410b813b41edbfe69e28 [term 1 LEADER]: Raft consensus shutting down.
I20250109 08:59:08.162283 20363 raft_consensus.cc:2267] T 4e2136a82d344ae6b0657803f1551906 P 1a9b5a4afd63410b813b41edbfe69e28 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:08.200098 20363 tablet_server.cc:195] TabletServer@127.19.226.195:0 shutdown complete.
I20250109 08:59:08.233258 20363 master.cc:537] Master@127.19.226.254:45577 shutting down...
I20250109 08:59:08.254664 20363 raft_consensus.cc:2238] T 00000000000000000000000000000000 P 8066e53f72404343aa3a9f0ff5455781 [term 1 LEADER]: Raft consensus shutting down.
I20250109 08:59:08.255291 20363 raft_consensus.cc:2267] T 00000000000000000000000000000000 P 8066e53f72404343aa3a9f0ff5455781 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:08.255616 20363 tablet_replica.cc:331] T 00000000000000000000000000000000 P 8066e53f72404343aa3a9f0ff5455781: stopping tablet replica
I20250109 08:59:08.276090 20363 master.cc:559] Master@127.19.226.254:45577 shutdown complete.
[ OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0 (20575 ms)
[ RUN ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1
I20250109 08:59:08.324059 20363 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.19.226.254:43781
I20250109 08:59:08.325198 20363 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250109 08:59:08.330665 25095 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250109 08:59:08.330787 25096 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250109 08:59:08.331604 25098 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250109 08:59:08.332289 20363 server_base.cc:1034] running on GCE node
I20250109 08:59:08.333123 20363 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250109 08:59:08.333302 20363 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250109 08:59:08.333420 20363 hybrid_clock.cc:648] HybridClock initialized: now 1736413148333407 us; error 0 us; skew 500 ppm
I20250109 08:59:08.333834 20363 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250109 08:59:08.336092 20363 webserver.cc:458] Webserver started at http://127.19.226.254:43533/ using document root <none> and password file <none>
I20250109 08:59:08.336585 20363 fs_manager.cc:362] Metadata directory not provided
I20250109 08:59:08.336737 20363 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250109 08:59:08.336985 20363 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250109 08:59:08.338128 20363 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1736412987538469-20363-0/minicluster-data/master-0-root/instance:
uuid: "7eb8c6a9d79149bc93b8c4660b491282"
format_stamp: "Formatted at 2025-01-09 08:59:08 on dist-test-slave-g8ds"
I20250109 08:59:08.342520 20363 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20250109 08:59:08.345736 25103 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250109 08:59:08.346534 20363 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20250109 08:59:08.346789 20363 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1736412987538469-20363-0/minicluster-data/master-0-root
uuid: "7eb8c6a9d79149bc93b8c4660b491282"
format_stamp: "Formatted at 2025-01-09 08:59:08 on dist-test-slave-g8ds"
I20250109 08:59:08.347039 20363 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1736412987538469-20363-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1736412987538469-20363-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1736412987538469-20363-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250109 08:59:08.369678 20363 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250109 08:59:08.370857 20363 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250109 08:59:08.406646 20363 rpc_server.cc:307] RPC server started. Bound to: 127.19.226.254:43781
I20250109 08:59:08.406752 25154 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.226.254:43781 every 8 connection(s)
I20250109 08:59:08.410342 25155 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:59:08.421041 25155 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7eb8c6a9d79149bc93b8c4660b491282: Bootstrap starting.
I20250109 08:59:08.425696 25155 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7eb8c6a9d79149bc93b8c4660b491282: Neither blocks nor log segments found. Creating new log.
I20250109 08:59:08.429834 25155 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7eb8c6a9d79149bc93b8c4660b491282: No bootstrap required, opened a new log
I20250109 08:59:08.431905 25155 raft_consensus.cc:357] T 00000000000000000000000000000000 P 7eb8c6a9d79149bc93b8c4660b491282 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7eb8c6a9d79149bc93b8c4660b491282" member_type: VOTER }
I20250109 08:59:08.432392 25155 raft_consensus.cc:383] T 00000000000000000000000000000000 P 7eb8c6a9d79149bc93b8c4660b491282 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:59:08.432610 25155 raft_consensus.cc:738] T 00000000000000000000000000000000 P 7eb8c6a9d79149bc93b8c4660b491282 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7eb8c6a9d79149bc93b8c4660b491282, State: Initialized, Role: FOLLOWER
I20250109 08:59:08.433161 25155 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7eb8c6a9d79149bc93b8c4660b491282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7eb8c6a9d79149bc93b8c4660b491282" member_type: VOTER }
I20250109 08:59:08.433593 25155 raft_consensus.cc:397] T 00000000000000000000000000000000 P 7eb8c6a9d79149bc93b8c4660b491282 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250109 08:59:08.433802 25155 raft_consensus.cc:491] T 00000000000000000000000000000000 P 7eb8c6a9d79149bc93b8c4660b491282 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250109 08:59:08.434048 25155 raft_consensus.cc:3054] T 00000000000000000000000000000000 P 7eb8c6a9d79149bc93b8c4660b491282 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:59:08.438627 25155 raft_consensus.cc:513] T 00000000000000000000000000000000 P 7eb8c6a9d79149bc93b8c4660b491282 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7eb8c6a9d79149bc93b8c4660b491282" member_type: VOTER }
I20250109 08:59:08.439193 25155 leader_election.cc:304] T 00000000000000000000000000000000 P 7eb8c6a9d79149bc93b8c4660b491282 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7eb8c6a9d79149bc93b8c4660b491282; no voters:
I20250109 08:59:08.440258 25155 leader_election.cc:290] T 00000000000000000000000000000000 P 7eb8c6a9d79149bc93b8c4660b491282 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250109 08:59:08.440683 25158 raft_consensus.cc:2798] T 00000000000000000000000000000000 P 7eb8c6a9d79149bc93b8c4660b491282 [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:59:08.441923 25158 raft_consensus.cc:695] T 00000000000000000000000000000000 P 7eb8c6a9d79149bc93b8c4660b491282 [term 1 LEADER]: Becoming Leader. State: Replica: 7eb8c6a9d79149bc93b8c4660b491282, State: Running, Role: LEADER
I20250109 08:59:08.442590 25158 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7eb8c6a9d79149bc93b8c4660b491282 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7eb8c6a9d79149bc93b8c4660b491282" member_type: VOTER }
I20250109 08:59:08.443104 25155 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7eb8c6a9d79149bc93b8c4660b491282 [sys.catalog]: configured and running, proceeding with master startup.
I20250109 08:59:08.447428 25160 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7eb8c6a9d79149bc93b8c4660b491282 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7eb8c6a9d79149bc93b8c4660b491282. Latest consensus state: current_term: 1 leader_uuid: "7eb8c6a9d79149bc93b8c4660b491282" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7eb8c6a9d79149bc93b8c4660b491282" member_type: VOTER } }
I20250109 08:59:08.448067 25160 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7eb8c6a9d79149bc93b8c4660b491282 [sys.catalog]: This master's current role is: LEADER
I20250109 08:59:08.448271 25159 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7eb8c6a9d79149bc93b8c4660b491282 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7eb8c6a9d79149bc93b8c4660b491282" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7eb8c6a9d79149bc93b8c4660b491282" member_type: VOTER } }
I20250109 08:59:08.448864 25159 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7eb8c6a9d79149bc93b8c4660b491282 [sys.catalog]: This master's current role is: LEADER
I20250109 08:59:08.450309 25165 catalog_manager.cc:1476] Loading table and tablet metadata into memory...
I20250109 08:59:08.454892 25165 catalog_manager.cc:1485] Initializing Kudu cluster ID...
I20250109 08:59:08.458011 20363 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20250109 08:59:08.463187 25165 catalog_manager.cc:1348] Generated new cluster ID: b227a47509ff4470b5195c28c825c765
I20250109 08:59:08.463479 25165 catalog_manager.cc:1496] Initializing Kudu internal certificate authority...
I20250109 08:59:08.522500 25165 catalog_manager.cc:1371] Generated new certificate authority record
I20250109 08:59:08.524004 25165 catalog_manager.cc:1505] Loading token signing keys...
I20250109 08:59:08.545258 25165 catalog_manager.cc:5899] T 00000000000000000000000000000000 P 7eb8c6a9d79149bc93b8c4660b491282: Generated new TSK 0
I20250109 08:59:08.545883 25165 catalog_manager.cc:1515] Initializing in-progress tserver states...
I20250109 08:59:08.589138 20363 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250109 08:59:08.595075 25176 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250109 08:59:08.595695 25177 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250109 08:59:08.597823 25179 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250109 08:59:08.598251 20363 server_base.cc:1034] running on GCE node
I20250109 08:59:08.599098 20363 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250109 08:59:08.599292 20363 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250109 08:59:08.599442 20363 hybrid_clock.cc:648] HybridClock initialized: now 1736413148599425 us; error 0 us; skew 500 ppm
I20250109 08:59:08.599908 20363 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250109 08:59:08.602108 20363 webserver.cc:458] Webserver started at http://127.19.226.193:39153/ using document root <none> and password file <none>
I20250109 08:59:08.602573 20363 fs_manager.cc:362] Metadata directory not provided
I20250109 08:59:08.602744 20363 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250109 08:59:08.602974 20363 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250109 08:59:08.604009 20363 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1736412987538469-20363-0/minicluster-data/ts-0-root/instance:
uuid: "2ff0106754a84b39989b9e042b4af3d6"
format_stamp: "Formatted at 2025-01-09 08:59:08 on dist-test-slave-g8ds"
I20250109 08:59:08.608302 20363 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.001s sys 0.002s
I20250109 08:59:08.611626 25184 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250109 08:59:08.612342 20363 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20250109 08:59:08.612641 20363 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1736412987538469-20363-0/minicluster-data/ts-0-root
uuid: "2ff0106754a84b39989b9e042b4af3d6"
format_stamp: "Formatted at 2025-01-09 08:59:08 on dist-test-slave-g8ds"
I20250109 08:59:08.612896 20363 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1736412987538469-20363-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1736412987538469-20363-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1736412987538469-20363-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250109 08:59:08.661522 20363 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250109 08:59:08.662735 20363 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250109 08:59:08.664207 20363 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250109 08:59:08.666514 20363 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250109 08:59:08.666700 20363 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250109 08:59:08.666919 20363 ts_tablet_manager.cc:610] Registered 0 tablets
I20250109 08:59:08.667061 20363 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250109 08:59:08.705680 20363 rpc_server.cc:307] RPC server started. Bound to: 127.19.226.193:34345
I20250109 08:59:08.705777 25246 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.226.193:34345 every 8 connection(s)
I20250109 08:59:08.710372 20363 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250109 08:59:08.718181 25251 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250109 08:59:08.718775 25252 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250109 08:59:08.724135 25254 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250109 08:59:08.724179 20363 server_base.cc:1034] running on GCE node
I20250109 08:59:08.725481 25247 heartbeater.cc:346] Connected to a master server at 127.19.226.254:43781
I20250109 08:59:08.725929 25247 heartbeater.cc:463] Registering TS with master...
I20250109 08:59:08.726169 20363 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250109 08:59:08.726470 20363 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250109 08:59:08.726658 20363 hybrid_clock.cc:648] HybridClock initialized: now 1736413148726642 us; error 0 us; skew 500 ppm
I20250109 08:59:08.726833 25247 heartbeater.cc:510] Master 127.19.226.254:43781 requested a full tablet report, sending...
I20250109 08:59:08.727411 20363 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250109 08:59:08.729391 25120 ts_manager.cc:194] Registered new tserver with Master: 2ff0106754a84b39989b9e042b4af3d6 (127.19.226.193:34345)
I20250109 08:59:08.730455 20363 webserver.cc:458] Webserver started at http://127.19.226.194:35997/ using document root <none> and password file <none>
I20250109 08:59:08.731145 20363 fs_manager.cc:362] Metadata directory not provided
I20250109 08:59:08.731376 20363 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250109 08:59:08.731674 20363 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250109 08:59:08.731877 25120 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:48076
I20250109 08:59:08.733284 20363 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1736412987538469-20363-0/minicluster-data/ts-1-root/instance:
uuid: "9d26e8b39691431590fd77e182046c66"
format_stamp: "Formatted at 2025-01-09 08:59:08 on dist-test-slave-g8ds"
I20250109 08:59:08.738214 20363 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20250109 08:59:08.741587 25259 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250109 08:59:08.742339 20363 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20250109 08:59:08.742620 20363 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1736412987538469-20363-0/minicluster-data/ts-1-root
uuid: "9d26e8b39691431590fd77e182046c66"
format_stamp: "Formatted at 2025-01-09 08:59:08 on dist-test-slave-g8ds"
I20250109 08:59:08.742897 20363 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1736412987538469-20363-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1736412987538469-20363-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1736412987538469-20363-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250109 08:59:08.762094 20363 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250109 08:59:08.763285 20363 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250109 08:59:08.764711 20363 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250109 08:59:08.766896 20363 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250109 08:59:08.767079 20363 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250109 08:59:08.767325 20363 ts_tablet_manager.cc:610] Registered 0 tablets
I20250109 08:59:08.767475 20363 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250109 08:59:08.805124 20363 rpc_server.cc:307] RPC server started. Bound to: 127.19.226.194:33163
I20250109 08:59:08.805219 25321 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.226.194:33163 every 8 connection(s)
I20250109 08:59:08.809638 20363 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250109 08:59:08.817723 25325 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250109 08:59:08.818666 25326 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250109 08:59:08.821962 25322 heartbeater.cc:346] Connected to a master server at 127.19.226.254:43781
I20250109 08:59:08.822717 25322 heartbeater.cc:463] Registering TS with master...
I20250109 08:59:08.823525 25322 heartbeater.cc:510] Master 127.19.226.254:43781 requested a full tablet report, sending...
W20250109 08:59:08.823832 25328 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250109 08:59:08.824229 20363 server_base.cc:1034] running on GCE node
I20250109 08:59:08.826248 25120 ts_manager.cc:194] Registered new tserver with Master: 9d26e8b39691431590fd77e182046c66 (127.19.226.194:33163)
I20250109 08:59:08.826587 20363 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250109 08:59:08.826843 20363 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250109 08:59:08.826994 20363 hybrid_clock.cc:648] HybridClock initialized: now 1736413148826980 us; error 0 us; skew 500 ppm
I20250109 08:59:08.827565 20363 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250109 08:59:08.827620 25120 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:48088
I20250109 08:59:08.830024 20363 webserver.cc:458] Webserver started at http://127.19.226.195:38131/ using document root <none> and password file <none>
I20250109 08:59:08.830539 20363 fs_manager.cc:362] Metadata directory not provided
I20250109 08:59:08.830811 20363 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250109 08:59:08.831104 20363 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250109 08:59:08.832242 20363 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1736412987538469-20363-0/minicluster-data/ts-2-root/instance:
uuid: "e6d654389db14a49a1f1bf3dbe538a27"
format_stamp: "Formatted at 2025-01-09 08:59:08 on dist-test-slave-g8ds"
I20250109 08:59:08.836751 20363 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20250109 08:59:08.840113 25333 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250109 08:59:08.840986 20363 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20250109 08:59:08.841277 20363 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1736412987538469-20363-0/minicluster-data/ts-2-root
uuid: "e6d654389db14a49a1f1bf3dbe538a27"
format_stamp: "Formatted at 2025-01-09 08:59:08 on dist-test-slave-g8ds"
I20250109 08:59:08.841573 20363 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1736412987538469-20363-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1736412987538469-20363-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1736412987538469-20363-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250109 08:59:08.856665 20363 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250109 08:59:08.857849 20363 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250109 08:59:08.859058 20363 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250109 08:59:08.861241 20363 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250109 08:59:08.861452 20363 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250109 08:59:08.861670 20363 ts_tablet_manager.cc:610] Registered 0 tablets
I20250109 08:59:08.861814 20363 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250109 08:59:08.900485 20363 rpc_server.cc:307] RPC server started. Bound to: 127.19.226.195:39911
I20250109 08:59:08.900580 25395 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.226.195:39911 every 8 connection(s)
I20250109 08:59:08.913282 25396 heartbeater.cc:346] Connected to a master server at 127.19.226.254:43781
I20250109 08:59:08.913717 25396 heartbeater.cc:463] Registering TS with master...
I20250109 08:59:08.914410 25396 heartbeater.cc:510] Master 127.19.226.254:43781 requested a full tablet report, sending...
I20250109 08:59:08.916317 25120 ts_manager.cc:194] Registered new tserver with Master: e6d654389db14a49a1f1bf3dbe538a27 (127.19.226.195:39911)
I20250109 08:59:08.916903 20363 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.013425911s
I20250109 08:59:08.918094 25120 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:48104
I20250109 08:59:08.919842 20363 test_util.cc:274] Using random seed: 820819985
I20250109 08:59:08.942049 25120 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48120:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250109 08:59:08.944964 25120 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250109 08:59:09.023561 25361 tablet_service.cc:1467] Processing CreateTablet for tablet 1d01a6ffe04e4c7bae266bbc80ee5d54 (DEFAULT_TABLE table=test-workload [id=bd2048aae15748a9b47ad16bbe9fd9c1]), partition=RANGE (key) PARTITION VALUES < 238609294
I20250109 08:59:09.024945 25360 tablet_service.cc:1467] Processing CreateTablet for tablet 9f829cc37f3b447a800c8ee3c51bd346 (DEFAULT_TABLE table=test-workload [id=bd2048aae15748a9b47ad16bbe9fd9c1]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20250109 08:59:09.027184 25359 tablet_service.cc:1467] Processing CreateTablet for tablet c16639dc193644c8bbcf05ed9f02295e (DEFAULT_TABLE table=test-workload [id=bd2048aae15748a9b47ad16bbe9fd9c1]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20250109 08:59:09.029505 25358 tablet_service.cc:1467] Processing CreateTablet for tablet da8d4429a2364a3781155ecc9a3b18b3 (DEFAULT_TABLE table=test-workload [id=bd2048aae15748a9b47ad16bbe9fd9c1]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20250109 08:59:09.029835 25287 tablet_service.cc:1467] Processing CreateTablet for tablet 1d01a6ffe04e4c7bae266bbc80ee5d54 (DEFAULT_TABLE table=test-workload [id=bd2048aae15748a9b47ad16bbe9fd9c1]), partition=RANGE (key) PARTITION VALUES < 238609294
I20250109 08:59:09.031212 25287 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d01a6ffe04e4c7bae266bbc80ee5d54. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:59:09.031366 25286 tablet_service.cc:1467] Processing CreateTablet for tablet 9f829cc37f3b447a800c8ee3c51bd346 (DEFAULT_TABLE table=test-workload [id=bd2048aae15748a9b47ad16bbe9fd9c1]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20250109 08:59:09.031752 25357 tablet_service.cc:1467] Processing CreateTablet for tablet 075f0030bd964a1687ccee531080fd1c (DEFAULT_TABLE table=test-workload [id=bd2048aae15748a9b47ad16bbe9fd9c1]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20250109 08:59:09.025007 25361 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d01a6ffe04e4c7bae266bbc80ee5d54. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:59:09.033556 25285 tablet_service.cc:1467] Processing CreateTablet for tablet c16639dc193644c8bbcf05ed9f02295e (DEFAULT_TABLE table=test-workload [id=bd2048aae15748a9b47ad16bbe9fd9c1]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20250109 08:59:09.033984 25356 tablet_service.cc:1467] Processing CreateTablet for tablet 7622fd22d50942618088f5e65422e713 (DEFAULT_TABLE table=test-workload [id=bd2048aae15748a9b47ad16bbe9fd9c1]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20250109 08:59:09.035355 25356 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7622fd22d50942618088f5e65422e713. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:59:09.035754 25284 tablet_service.cc:1467] Processing CreateTablet for tablet da8d4429a2364a3781155ecc9a3b18b3 (DEFAULT_TABLE table=test-workload [id=bd2048aae15748a9b47ad16bbe9fd9c1]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20250109 08:59:09.036700 25357 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 075f0030bd964a1687ccee531080fd1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:59:09.037076 25284 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet da8d4429a2364a3781155ecc9a3b18b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:59:09.039291 25283 tablet_service.cc:1467] Processing CreateTablet for tablet 075f0030bd964a1687ccee531080fd1c (DEFAULT_TABLE table=test-workload [id=bd2048aae15748a9b47ad16bbe9fd9c1]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20250109 08:59:09.040743 25360 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f829cc37f3b447a800c8ee3c51bd346. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:59:09.041599 25282 tablet_service.cc:1467] Processing CreateTablet for tablet 7622fd22d50942618088f5e65422e713 (DEFAULT_TABLE table=test-workload [id=bd2048aae15748a9b47ad16bbe9fd9c1]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20250109 08:59:09.040638 25355 tablet_service.cc:1467] Processing CreateTablet for tablet 486c1bbdc3e247cdbbf13e1f7456e729 (DEFAULT_TABLE table=test-workload [id=bd2048aae15748a9b47ad16bbe9fd9c1]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20250109 08:59:09.043429 25355 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 486c1bbdc3e247cdbbf13e1f7456e729. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:59:09.044018 25285 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c16639dc193644c8bbcf05ed9f02295e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:59:09.044720 25359 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c16639dc193644c8bbcf05ed9f02295e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:59:09.045348 25286 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f829cc37f3b447a800c8ee3c51bd346. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:59:09.046816 25353 tablet_service.cc:1467] Processing CreateTablet for tablet 422f918bde184ce1ae2532c9dee609a7 (DEFAULT_TABLE table=test-workload [id=bd2048aae15748a9b47ad16bbe9fd9c1]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20250109 08:59:09.042024 25354 tablet_service.cc:1467] Processing CreateTablet for tablet 709470e8d22046dd87f2a20e8a8ce745 (DEFAULT_TABLE table=test-workload [id=bd2048aae15748a9b47ad16bbe9fd9c1]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20250109 08:59:09.050184 25281 tablet_service.cc:1467] Processing CreateTablet for tablet 486c1bbdc3e247cdbbf13e1f7456e729 (DEFAULT_TABLE table=test-workload [id=bd2048aae15748a9b47ad16bbe9fd9c1]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20250109 08:59:09.051514 25281 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 486c1bbdc3e247cdbbf13e1f7456e729. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:59:09.053447 25280 tablet_service.cc:1467] Processing CreateTablet for tablet 709470e8d22046dd87f2a20e8a8ce745 (DEFAULT_TABLE table=test-workload [id=bd2048aae15748a9b47ad16bbe9fd9c1]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20250109 08:59:09.054730 25280 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 709470e8d22046dd87f2a20e8a8ce745. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:59:09.057128 25279 tablet_service.cc:1467] Processing CreateTablet for tablet 422f918bde184ce1ae2532c9dee609a7 (DEFAULT_TABLE table=test-workload [id=bd2048aae15748a9b47ad16bbe9fd9c1]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20250109 08:59:09.058526 25279 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 422f918bde184ce1ae2532c9dee609a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:59:09.047399 25358 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet da8d4429a2364a3781155ecc9a3b18b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:59:09.063757 25354 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 709470e8d22046dd87f2a20e8a8ce745. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:59:09.064512 25353 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 422f918bde184ce1ae2532c9dee609a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:59:09.063675 25411 tablet_bootstrap.cc:492] T 1d01a6ffe04e4c7bae266bbc80ee5d54 P 9d26e8b39691431590fd77e182046c66: Bootstrap starting.
I20250109 08:59:09.076761 25282 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7622fd22d50942618088f5e65422e713. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:59:09.078403 25283 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 075f0030bd964a1687ccee531080fd1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:59:09.083328 25412 tablet_bootstrap.cc:492] T c16639dc193644c8bbcf05ed9f02295e P e6d654389db14a49a1f1bf3dbe538a27: Bootstrap starting.
I20250109 08:59:09.087504 25411 tablet_bootstrap.cc:654] T 1d01a6ffe04e4c7bae266bbc80ee5d54 P 9d26e8b39691431590fd77e182046c66: Neither blocks nor log segments found. Creating new log.
I20250109 08:59:09.093564 25412 tablet_bootstrap.cc:654] T c16639dc193644c8bbcf05ed9f02295e P e6d654389db14a49a1f1bf3dbe538a27: Neither blocks nor log segments found. Creating new log.
I20250109 08:59:09.099308 25411 tablet_bootstrap.cc:492] T 1d01a6ffe04e4c7bae266bbc80ee5d54 P 9d26e8b39691431590fd77e182046c66: No bootstrap required, opened a new log
I20250109 08:59:09.099867 25411 ts_tablet_manager.cc:1397] T 1d01a6ffe04e4c7bae266bbc80ee5d54 P 9d26e8b39691431590fd77e182046c66: Time spent bootstrapping tablet: real 0.037s user 0.013s sys 0.003s
I20250109 08:59:09.103046 25411 raft_consensus.cc:357] T 1d01a6ffe04e4c7bae266bbc80ee5d54 P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } } peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } }
I20250109 08:59:09.103881 25411 raft_consensus.cc:383] T 1d01a6ffe04e4c7bae266bbc80ee5d54 P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:59:09.104209 25411 raft_consensus.cc:738] T 1d01a6ffe04e4c7bae266bbc80ee5d54 P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d26e8b39691431590fd77e182046c66, State: Initialized, Role: FOLLOWER
I20250109 08:59:09.105252 25411 consensus_queue.cc:260] T 1d01a6ffe04e4c7bae266bbc80ee5d54 P 9d26e8b39691431590fd77e182046c66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } } peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } }
I20250109 08:59:09.109696 25412 tablet_bootstrap.cc:492] T c16639dc193644c8bbcf05ed9f02295e P e6d654389db14a49a1f1bf3dbe538a27: No bootstrap required, opened a new log
I20250109 08:59:09.110456 25412 ts_tablet_manager.cc:1397] T c16639dc193644c8bbcf05ed9f02295e P e6d654389db14a49a1f1bf3dbe538a27: Time spent bootstrapping tablet: real 0.027s user 0.015s sys 0.003s
I20250109 08:59:09.114651 25412 raft_consensus.cc:357] T c16639dc193644c8bbcf05ed9f02295e P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:09.116447 25412 raft_consensus.cc:383] T c16639dc193644c8bbcf05ed9f02295e P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:59:09.116808 25412 raft_consensus.cc:738] T c16639dc193644c8bbcf05ed9f02295e P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6d654389db14a49a1f1bf3dbe538a27, State: Initialized, Role: FOLLOWER
I20250109 08:59:09.117612 25412 consensus_queue.cc:260] T c16639dc193644c8bbcf05ed9f02295e P e6d654389db14a49a1f1bf3dbe538a27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:09.129778 25396 heartbeater.cc:502] Master 127.19.226.254:43781 was elected leader, sending a full tablet report...
I20250109 08:59:09.148912 25412 ts_tablet_manager.cc:1428] T c16639dc193644c8bbcf05ed9f02295e P e6d654389db14a49a1f1bf3dbe538a27: Time spent starting tablet: real 0.038s user 0.007s sys 0.010s
I20250109 08:59:09.155182 25412 tablet_bootstrap.cc:492] T 486c1bbdc3e247cdbbf13e1f7456e729 P e6d654389db14a49a1f1bf3dbe538a27: Bootstrap starting.
I20250109 08:59:09.158205 25322 heartbeater.cc:502] Master 127.19.226.254:43781 was elected leader, sending a full tablet report...
I20250109 08:59:09.164276 25412 tablet_bootstrap.cc:654] T 486c1bbdc3e247cdbbf13e1f7456e729 P e6d654389db14a49a1f1bf3dbe538a27: Neither blocks nor log segments found. Creating new log.
I20250109 08:59:09.168064 25411 ts_tablet_manager.cc:1428] T 1d01a6ffe04e4c7bae266bbc80ee5d54 P 9d26e8b39691431590fd77e182046c66: Time spent starting tablet: real 0.068s user 0.016s sys 0.030s
I20250109 08:59:09.168660 25205 tablet_service.cc:1467] Processing CreateTablet for tablet 709470e8d22046dd87f2a20e8a8ce745 (DEFAULT_TABLE table=test-workload [id=bd2048aae15748a9b47ad16bbe9fd9c1]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20250109 08:59:09.169245 25411 tablet_bootstrap.cc:492] T 075f0030bd964a1687ccee531080fd1c P 9d26e8b39691431590fd77e182046c66: Bootstrap starting.
I20250109 08:59:09.170047 25205 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 709470e8d22046dd87f2a20e8a8ce745. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:59:09.173544 25212 tablet_service.cc:1467] Processing CreateTablet for tablet 1d01a6ffe04e4c7bae266bbc80ee5d54 (DEFAULT_TABLE table=test-workload [id=bd2048aae15748a9b47ad16bbe9fd9c1]), partition=RANGE (key) PARTITION VALUES < 238609294
I20250109 08:59:09.174889 25212 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d01a6ffe04e4c7bae266bbc80ee5d54. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:59:09.175614 25206 tablet_service.cc:1467] Processing CreateTablet for tablet 486c1bbdc3e247cdbbf13e1f7456e729 (DEFAULT_TABLE table=test-workload [id=bd2048aae15748a9b47ad16bbe9fd9c1]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20250109 08:59:09.176887 25206 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 486c1bbdc3e247cdbbf13e1f7456e729. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:59:09.177963 25411 tablet_bootstrap.cc:654] T 075f0030bd964a1687ccee531080fd1c P 9d26e8b39691431590fd77e182046c66: Neither blocks nor log segments found. Creating new log.
I20250109 08:59:09.180604 25207 tablet_service.cc:1467] Processing CreateTablet for tablet 7622fd22d50942618088f5e65422e713 (DEFAULT_TABLE table=test-workload [id=bd2048aae15748a9b47ad16bbe9fd9c1]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20250109 08:59:09.181465 25209 tablet_service.cc:1467] Processing CreateTablet for tablet da8d4429a2364a3781155ecc9a3b18b3 (DEFAULT_TABLE table=test-workload [id=bd2048aae15748a9b47ad16bbe9fd9c1]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20250109 08:59:09.182189 25207 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7622fd22d50942618088f5e65422e713. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:59:09.183326 25210 tablet_service.cc:1467] Processing CreateTablet for tablet c16639dc193644c8bbcf05ed9f02295e (DEFAULT_TABLE table=test-workload [id=bd2048aae15748a9b47ad16bbe9fd9c1]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20250109 08:59:09.184291 25211 tablet_service.cc:1467] Processing CreateTablet for tablet 9f829cc37f3b447a800c8ee3c51bd346 (DEFAULT_TABLE table=test-workload [id=bd2048aae15748a9b47ad16bbe9fd9c1]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20250109 08:59:09.185410 25208 tablet_service.cc:1467] Processing CreateTablet for tablet 075f0030bd964a1687ccee531080fd1c (DEFAULT_TABLE table=test-workload [id=bd2048aae15748a9b47ad16bbe9fd9c1]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20250109 08:59:09.185495 25204 tablet_service.cc:1467] Processing CreateTablet for tablet 422f918bde184ce1ae2532c9dee609a7 (DEFAULT_TABLE table=test-workload [id=bd2048aae15748a9b47ad16bbe9fd9c1]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20250109 08:59:09.187316 25210 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c16639dc193644c8bbcf05ed9f02295e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:59:09.189427 25204 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 422f918bde184ce1ae2532c9dee609a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:59:09.192737 25209 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet da8d4429a2364a3781155ecc9a3b18b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:59:09.193324 25211 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f829cc37f3b447a800c8ee3c51bd346. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:59:09.194443 25208 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 075f0030bd964a1687ccee531080fd1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:59:09.215618 25411 tablet_bootstrap.cc:492] T 075f0030bd964a1687ccee531080fd1c P 9d26e8b39691431590fd77e182046c66: No bootstrap required, opened a new log
I20250109 08:59:09.216187 25411 ts_tablet_manager.cc:1397] T 075f0030bd964a1687ccee531080fd1c P 9d26e8b39691431590fd77e182046c66: Time spent bootstrapping tablet: real 0.047s user 0.009s sys 0.005s
I20250109 08:59:09.218829 25412 tablet_bootstrap.cc:492] T 486c1bbdc3e247cdbbf13e1f7456e729 P e6d654389db14a49a1f1bf3dbe538a27: No bootstrap required, opened a new log
I20250109 08:59:09.219458 25412 ts_tablet_manager.cc:1397] T 486c1bbdc3e247cdbbf13e1f7456e729 P e6d654389db14a49a1f1bf3dbe538a27: Time spent bootstrapping tablet: real 0.064s user 0.012s sys 0.001s
I20250109 08:59:09.219106 25411 raft_consensus.cc:357] T 075f0030bd964a1687ccee531080fd1c P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:09.221474 25411 raft_consensus.cc:383] T 075f0030bd964a1687ccee531080fd1c P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:59:09.221747 25411 raft_consensus.cc:738] T 075f0030bd964a1687ccee531080fd1c P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d26e8b39691431590fd77e182046c66, State: Initialized, Role: FOLLOWER
I20250109 08:59:09.239226 25411 consensus_queue.cc:260] T 075f0030bd964a1687ccee531080fd1c P 9d26e8b39691431590fd77e182046c66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:09.240265 25419 tablet_bootstrap.cc:492] T 1d01a6ffe04e4c7bae266bbc80ee5d54 P 2ff0106754a84b39989b9e042b4af3d6: Bootstrap starting.
I20250109 08:59:09.244215 25411 ts_tablet_manager.cc:1428] T 075f0030bd964a1687ccee531080fd1c P 9d26e8b39691431590fd77e182046c66: Time spent starting tablet: real 0.028s user 0.007s sys 0.008s
I20250109 08:59:09.245960 25411 tablet_bootstrap.cc:492] T 7622fd22d50942618088f5e65422e713 P 9d26e8b39691431590fd77e182046c66: Bootstrap starting.
I20250109 08:59:09.246495 25419 tablet_bootstrap.cc:654] T 1d01a6ffe04e4c7bae266bbc80ee5d54 P 2ff0106754a84b39989b9e042b4af3d6: Neither blocks nor log segments found. Creating new log.
I20250109 08:59:09.245780 25412 raft_consensus.cc:357] T 486c1bbdc3e247cdbbf13e1f7456e729 P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:09.249145 25412 raft_consensus.cc:383] T 486c1bbdc3e247cdbbf13e1f7456e729 P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:59:09.249598 25412 raft_consensus.cc:738] T 486c1bbdc3e247cdbbf13e1f7456e729 P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6d654389db14a49a1f1bf3dbe538a27, State: Initialized, Role: FOLLOWER
I20250109 08:59:09.250432 25412 consensus_queue.cc:260] T 486c1bbdc3e247cdbbf13e1f7456e729 P e6d654389db14a49a1f1bf3dbe538a27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:09.252676 25411 tablet_bootstrap.cc:654] T 7622fd22d50942618088f5e65422e713 P 9d26e8b39691431590fd77e182046c66: Neither blocks nor log segments found. Creating new log.
I20250109 08:59:09.256574 25416 raft_consensus.cc:491] T 486c1bbdc3e247cdbbf13e1f7456e729 P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:59:09.257021 25416 raft_consensus.cc:513] T 486c1bbdc3e247cdbbf13e1f7456e729 P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:09.259434 25416 leader_election.cc:290] T 486c1bbdc3e247cdbbf13e1f7456e729 P e6d654389db14a49a1f1bf3dbe538a27 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2ff0106754a84b39989b9e042b4af3d6 (127.19.226.193:34345), 9d26e8b39691431590fd77e182046c66 (127.19.226.194:33163)
I20250109 08:59:09.262698 25411 tablet_bootstrap.cc:492] T 7622fd22d50942618088f5e65422e713 P 9d26e8b39691431590fd77e182046c66: No bootstrap required, opened a new log
I20250109 08:59:09.266189 25411 ts_tablet_manager.cc:1397] T 7622fd22d50942618088f5e65422e713 P 9d26e8b39691431590fd77e182046c66: Time spent bootstrapping tablet: real 0.020s user 0.012s sys 0.000s
I20250109 08:59:09.268954 25411 raft_consensus.cc:357] T 7622fd22d50942618088f5e65422e713 P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:09.269737 25411 raft_consensus.cc:383] T 7622fd22d50942618088f5e65422e713 P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:59:09.270036 25411 raft_consensus.cc:738] T 7622fd22d50942618088f5e65422e713 P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d26e8b39691431590fd77e182046c66, State: Initialized, Role: FOLLOWER
I20250109 08:59:09.271404 25411 consensus_queue.cc:260] T 7622fd22d50942618088f5e65422e713 P 9d26e8b39691431590fd77e182046c66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:09.272449 25222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "486c1bbdc3e247cdbbf13e1f7456e729" candidate_uuid: "e6d654389db14a49a1f1bf3dbe538a27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ff0106754a84b39989b9e042b4af3d6" is_pre_election: true
I20250109 08:59:09.273236 25412 ts_tablet_manager.cc:1428] T 486c1bbdc3e247cdbbf13e1f7456e729 P e6d654389db14a49a1f1bf3dbe538a27: Time spent starting tablet: real 0.053s user 0.005s sys 0.009s
I20250109 08:59:09.274191 25412 tablet_bootstrap.cc:492] T 7622fd22d50942618088f5e65422e713 P e6d654389db14a49a1f1bf3dbe538a27: Bootstrap starting.
W20250109 08:59:09.274415 25335 leader_election.cc:343] T 486c1bbdc3e247cdbbf13e1f7456e729 P e6d654389db14a49a1f1bf3dbe538a27 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2ff0106754a84b39989b9e042b4af3d6 (127.19.226.193:34345): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:59:09.278025 25411 ts_tablet_manager.cc:1428] T 7622fd22d50942618088f5e65422e713 P 9d26e8b39691431590fd77e182046c66: Time spent starting tablet: real 0.011s user 0.006s sys 0.000s
I20250109 08:59:09.279104 25411 tablet_bootstrap.cc:492] T 709470e8d22046dd87f2a20e8a8ce745 P 9d26e8b39691431590fd77e182046c66: Bootstrap starting.
I20250109 08:59:09.280315 25412 tablet_bootstrap.cc:654] T 7622fd22d50942618088f5e65422e713 P e6d654389db14a49a1f1bf3dbe538a27: Neither blocks nor log segments found. Creating new log.
I20250109 08:59:09.282872 25419 tablet_bootstrap.cc:492] T 1d01a6ffe04e4c7bae266bbc80ee5d54 P 2ff0106754a84b39989b9e042b4af3d6: No bootstrap required, opened a new log
I20250109 08:59:09.283407 25419 ts_tablet_manager.cc:1397] T 1d01a6ffe04e4c7bae266bbc80ee5d54 P 2ff0106754a84b39989b9e042b4af3d6: Time spent bootstrapping tablet: real 0.043s user 0.006s sys 0.011s
I20250109 08:59:09.286540 25419 raft_consensus.cc:357] T 1d01a6ffe04e4c7bae266bbc80ee5d54 P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } } peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } }
I20250109 08:59:09.287353 25419 raft_consensus.cc:383] T 1d01a6ffe04e4c7bae266bbc80ee5d54 P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:59:09.287633 25419 raft_consensus.cc:738] T 1d01a6ffe04e4c7bae266bbc80ee5d54 P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ff0106754a84b39989b9e042b4af3d6, State: Initialized, Role: FOLLOWER
I20250109 08:59:09.288326 25419 consensus_queue.cc:260] T 1d01a6ffe04e4c7bae266bbc80ee5d54 P 2ff0106754a84b39989b9e042b4af3d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } } peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } }
I20250109 08:59:09.291329 25411 tablet_bootstrap.cc:654] T 709470e8d22046dd87f2a20e8a8ce745 P 9d26e8b39691431590fd77e182046c66: Neither blocks nor log segments found. Creating new log.
I20250109 08:59:09.291574 25297 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "486c1bbdc3e247cdbbf13e1f7456e729" candidate_uuid: "e6d654389db14a49a1f1bf3dbe538a27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d26e8b39691431590fd77e182046c66" is_pre_election: true
I20250109 08:59:09.292352 25412 tablet_bootstrap.cc:492] T 7622fd22d50942618088f5e65422e713 P e6d654389db14a49a1f1bf3dbe538a27: No bootstrap required, opened a new log
I20250109 08:59:09.292981 25412 ts_tablet_manager.cc:1397] T 7622fd22d50942618088f5e65422e713 P e6d654389db14a49a1f1bf3dbe538a27: Time spent bootstrapping tablet: real 0.019s user 0.014s sys 0.004s
W20250109 08:59:09.293534 25336 leader_election.cc:343] T 486c1bbdc3e247cdbbf13e1f7456e729 P e6d654389db14a49a1f1bf3dbe538a27 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9d26e8b39691431590fd77e182046c66 (127.19.226.194:33163): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:59:09.294124 25336 leader_election.cc:304] T 486c1bbdc3e247cdbbf13e1f7456e729 P e6d654389db14a49a1f1bf3dbe538a27 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e6d654389db14a49a1f1bf3dbe538a27; no voters: 2ff0106754a84b39989b9e042b4af3d6, 9d26e8b39691431590fd77e182046c66
I20250109 08:59:09.295109 25416 raft_consensus.cc:2743] T 486c1bbdc3e247cdbbf13e1f7456e729 P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:59:09.297111 25412 raft_consensus.cc:357] T 7622fd22d50942618088f5e65422e713 P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:09.297940 25412 raft_consensus.cc:383] T 7622fd22d50942618088f5e65422e713 P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:59:09.298242 25412 raft_consensus.cc:738] T 7622fd22d50942618088f5e65422e713 P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6d654389db14a49a1f1bf3dbe538a27, State: Initialized, Role: FOLLOWER
I20250109 08:59:09.299072 25412 consensus_queue.cc:260] T 7622fd22d50942618088f5e65422e713 P e6d654389db14a49a1f1bf3dbe538a27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:09.300196 25247 heartbeater.cc:502] Master 127.19.226.254:43781 was elected leader, sending a full tablet report...
I20250109 08:59:09.301767 25412 ts_tablet_manager.cc:1428] T 7622fd22d50942618088f5e65422e713 P e6d654389db14a49a1f1bf3dbe538a27: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20250109 08:59:09.301906 25411 tablet_bootstrap.cc:492] T 709470e8d22046dd87f2a20e8a8ce745 P 9d26e8b39691431590fd77e182046c66: No bootstrap required, opened a new log
I20250109 08:59:09.300179 25419 ts_tablet_manager.cc:1428] T 1d01a6ffe04e4c7bae266bbc80ee5d54 P 2ff0106754a84b39989b9e042b4af3d6: Time spent starting tablet: real 0.016s user 0.007s sys 0.007s
I20250109 08:59:09.307206 25411 ts_tablet_manager.cc:1397] T 709470e8d22046dd87f2a20e8a8ce745 P 9d26e8b39691431590fd77e182046c66: Time spent bootstrapping tablet: real 0.028s user 0.015s sys 0.000s
I20250109 08:59:09.308007 25419 tablet_bootstrap.cc:492] T 486c1bbdc3e247cdbbf13e1f7456e729 P 2ff0106754a84b39989b9e042b4af3d6: Bootstrap starting.
I20250109 08:59:09.309831 25412 tablet_bootstrap.cc:492] T 422f918bde184ce1ae2532c9dee609a7 P e6d654389db14a49a1f1bf3dbe538a27: Bootstrap starting.
I20250109 08:59:09.311244 25411 raft_consensus.cc:357] T 709470e8d22046dd87f2a20e8a8ce745 P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:09.312069 25411 raft_consensus.cc:383] T 709470e8d22046dd87f2a20e8a8ce745 P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:59:09.312353 25416 raft_consensus.cc:491] T c16639dc193644c8bbcf05ed9f02295e P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:59:09.312914 25416 raft_consensus.cc:513] T c16639dc193644c8bbcf05ed9f02295e P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:09.314922 25416 leader_election.cc:290] T c16639dc193644c8bbcf05ed9f02295e P e6d654389db14a49a1f1bf3dbe538a27 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2ff0106754a84b39989b9e042b4af3d6 (127.19.226.193:34345), 9d26e8b39691431590fd77e182046c66 (127.19.226.194:33163)
I20250109 08:59:09.315742 25419 tablet_bootstrap.cc:654] T 486c1bbdc3e247cdbbf13e1f7456e729 P 2ff0106754a84b39989b9e042b4af3d6: Neither blocks nor log segments found. Creating new log.
I20250109 08:59:09.317245 25222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c16639dc193644c8bbcf05ed9f02295e" candidate_uuid: "e6d654389db14a49a1f1bf3dbe538a27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ff0106754a84b39989b9e042b4af3d6" is_pre_election: true
I20250109 08:59:09.318384 25412 tablet_bootstrap.cc:654] T 422f918bde184ce1ae2532c9dee609a7 P e6d654389db14a49a1f1bf3dbe538a27: Neither blocks nor log segments found. Creating new log.
W20250109 08:59:09.319298 25335 leader_election.cc:343] T c16639dc193644c8bbcf05ed9f02295e P e6d654389db14a49a1f1bf3dbe538a27 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2ff0106754a84b39989b9e042b4af3d6 (127.19.226.193:34345): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:59:09.318528 25297 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c16639dc193644c8bbcf05ed9f02295e" candidate_uuid: "e6d654389db14a49a1f1bf3dbe538a27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d26e8b39691431590fd77e182046c66" is_pre_election: true
W20250109 08:59:09.321115 25336 leader_election.cc:343] T c16639dc193644c8bbcf05ed9f02295e P e6d654389db14a49a1f1bf3dbe538a27 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9d26e8b39691431590fd77e182046c66 (127.19.226.194:33163): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:59:09.321568 25336 leader_election.cc:304] T c16639dc193644c8bbcf05ed9f02295e P e6d654389db14a49a1f1bf3dbe538a27 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e6d654389db14a49a1f1bf3dbe538a27; no voters: 2ff0106754a84b39989b9e042b4af3d6, 9d26e8b39691431590fd77e182046c66
I20250109 08:59:09.322721 25416 raft_consensus.cc:2743] T c16639dc193644c8bbcf05ed9f02295e P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:59:09.312402 25411 raft_consensus.cc:738] T 709470e8d22046dd87f2a20e8a8ce745 P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d26e8b39691431590fd77e182046c66, State: Initialized, Role: FOLLOWER
I20250109 08:59:09.324146 25411 consensus_queue.cc:260] T 709470e8d22046dd87f2a20e8a8ce745 P 9d26e8b39691431590fd77e182046c66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:09.329056 25411 ts_tablet_manager.cc:1428] T 709470e8d22046dd87f2a20e8a8ce745 P 9d26e8b39691431590fd77e182046c66: Time spent starting tablet: real 0.022s user 0.006s sys 0.001s
I20250109 08:59:09.330053 25411 tablet_bootstrap.cc:492] T 422f918bde184ce1ae2532c9dee609a7 P 9d26e8b39691431590fd77e182046c66: Bootstrap starting.
I20250109 08:59:09.336257 25411 tablet_bootstrap.cc:654] T 422f918bde184ce1ae2532c9dee609a7 P 9d26e8b39691431590fd77e182046c66: Neither blocks nor log segments found. Creating new log.
I20250109 08:59:09.338804 25412 tablet_bootstrap.cc:492] T 422f918bde184ce1ae2532c9dee609a7 P e6d654389db14a49a1f1bf3dbe538a27: No bootstrap required, opened a new log
I20250109 08:59:09.339306 25419 tablet_bootstrap.cc:492] T 486c1bbdc3e247cdbbf13e1f7456e729 P 2ff0106754a84b39989b9e042b4af3d6: No bootstrap required, opened a new log
I20250109 08:59:09.339844 25419 ts_tablet_manager.cc:1397] T 486c1bbdc3e247cdbbf13e1f7456e729 P 2ff0106754a84b39989b9e042b4af3d6: Time spent bootstrapping tablet: real 0.032s user 0.009s sys 0.018s
I20250109 08:59:09.339356 25412 ts_tablet_manager.cc:1397] T 422f918bde184ce1ae2532c9dee609a7 P e6d654389db14a49a1f1bf3dbe538a27: Time spent bootstrapping tablet: real 0.030s user 0.017s sys 0.008s
I20250109 08:59:09.342022 25419 raft_consensus.cc:357] T 486c1bbdc3e247cdbbf13e1f7456e729 P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:09.342340 25411 tablet_bootstrap.cc:492] T 422f918bde184ce1ae2532c9dee609a7 P 9d26e8b39691431590fd77e182046c66: No bootstrap required, opened a new log
I20250109 08:59:09.342674 25419 raft_consensus.cc:383] T 486c1bbdc3e247cdbbf13e1f7456e729 P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:59:09.342963 25419 raft_consensus.cc:738] T 486c1bbdc3e247cdbbf13e1f7456e729 P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ff0106754a84b39989b9e042b4af3d6, State: Initialized, Role: FOLLOWER
I20250109 08:59:09.342952 25411 ts_tablet_manager.cc:1397] T 422f918bde184ce1ae2532c9dee609a7 P 9d26e8b39691431590fd77e182046c66: Time spent bootstrapping tablet: real 0.013s user 0.006s sys 0.006s
I20250109 08:59:09.343257 25412 raft_consensus.cc:357] T 422f918bde184ce1ae2532c9dee609a7 P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:09.343768 25419 consensus_queue.cc:260] T 486c1bbdc3e247cdbbf13e1f7456e729 P 2ff0106754a84b39989b9e042b4af3d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:09.344249 25412 raft_consensus.cc:383] T 422f918bde184ce1ae2532c9dee609a7 P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:59:09.344763 25412 raft_consensus.cc:738] T 422f918bde184ce1ae2532c9dee609a7 P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6d654389db14a49a1f1bf3dbe538a27, State: Initialized, Role: FOLLOWER
I20250109 08:59:09.345583 25412 consensus_queue.cc:260] T 422f918bde184ce1ae2532c9dee609a7 P e6d654389db14a49a1f1bf3dbe538a27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:09.346230 25411 raft_consensus.cc:357] T 422f918bde184ce1ae2532c9dee609a7 P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:09.347090 25411 raft_consensus.cc:383] T 422f918bde184ce1ae2532c9dee609a7 P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:59:09.347374 25411 raft_consensus.cc:738] T 422f918bde184ce1ae2532c9dee609a7 P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d26e8b39691431590fd77e182046c66, State: Initialized, Role: FOLLOWER
I20250109 08:59:09.347951 25412 ts_tablet_manager.cc:1428] T 422f918bde184ce1ae2532c9dee609a7 P e6d654389db14a49a1f1bf3dbe538a27: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20250109 08:59:09.346156 25419 ts_tablet_manager.cc:1428] T 486c1bbdc3e247cdbbf13e1f7456e729 P 2ff0106754a84b39989b9e042b4af3d6: Time spent starting tablet: real 0.006s user 0.003s sys 0.002s
I20250109 08:59:09.349112 25412 tablet_bootstrap.cc:492] T 709470e8d22046dd87f2a20e8a8ce745 P e6d654389db14a49a1f1bf3dbe538a27: Bootstrap starting.
I20250109 08:59:09.348143 25411 consensus_queue.cc:260] T 422f918bde184ce1ae2532c9dee609a7 P 9d26e8b39691431590fd77e182046c66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:09.349159 25419 tablet_bootstrap.cc:492] T 709470e8d22046dd87f2a20e8a8ce745 P 2ff0106754a84b39989b9e042b4af3d6: Bootstrap starting.
I20250109 08:59:09.355845 25412 tablet_bootstrap.cc:654] T 709470e8d22046dd87f2a20e8a8ce745 P e6d654389db14a49a1f1bf3dbe538a27: Neither blocks nor log segments found. Creating new log.
I20250109 08:59:09.358801 25419 tablet_bootstrap.cc:654] T 709470e8d22046dd87f2a20e8a8ce745 P 2ff0106754a84b39989b9e042b4af3d6: Neither blocks nor log segments found. Creating new log.
I20250109 08:59:09.352013 25411 ts_tablet_manager.cc:1428] T 422f918bde184ce1ae2532c9dee609a7 P 9d26e8b39691431590fd77e182046c66: Time spent starting tablet: real 0.008s user 0.005s sys 0.001s
I20250109 08:59:09.361419 25411 tablet_bootstrap.cc:492] T 9f829cc37f3b447a800c8ee3c51bd346 P 9d26e8b39691431590fd77e182046c66: Bootstrap starting.
I20250109 08:59:09.362722 25412 tablet_bootstrap.cc:492] T 709470e8d22046dd87f2a20e8a8ce745 P e6d654389db14a49a1f1bf3dbe538a27: No bootstrap required, opened a new log
I20250109 08:59:09.363317 25412 ts_tablet_manager.cc:1397] T 709470e8d22046dd87f2a20e8a8ce745 P e6d654389db14a49a1f1bf3dbe538a27: Time spent bootstrapping tablet: real 0.014s user 0.012s sys 0.001s
I20250109 08:59:09.366261 25412 raft_consensus.cc:357] T 709470e8d22046dd87f2a20e8a8ce745 P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:09.367205 25412 raft_consensus.cc:383] T 709470e8d22046dd87f2a20e8a8ce745 P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:59:09.367551 25412 raft_consensus.cc:738] T 709470e8d22046dd87f2a20e8a8ce745 P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6d654389db14a49a1f1bf3dbe538a27, State: Initialized, Role: FOLLOWER
I20250109 08:59:09.367627 25411 tablet_bootstrap.cc:654] T 9f829cc37f3b447a800c8ee3c51bd346 P 9d26e8b39691431590fd77e182046c66: Neither blocks nor log segments found. Creating new log.
I20250109 08:59:09.368541 25412 consensus_queue.cc:260] T 709470e8d22046dd87f2a20e8a8ce745 P e6d654389db14a49a1f1bf3dbe538a27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:09.373983 25412 ts_tablet_manager.cc:1428] T 709470e8d22046dd87f2a20e8a8ce745 P e6d654389db14a49a1f1bf3dbe538a27: Time spent starting tablet: real 0.010s user 0.006s sys 0.000s
I20250109 08:59:09.375221 25412 tablet_bootstrap.cc:492] T 9f829cc37f3b447a800c8ee3c51bd346 P e6d654389db14a49a1f1bf3dbe538a27: Bootstrap starting.
I20250109 08:59:09.382675 25412 tablet_bootstrap.cc:654] T 9f829cc37f3b447a800c8ee3c51bd346 P e6d654389db14a49a1f1bf3dbe538a27: Neither blocks nor log segments found. Creating new log.
I20250109 08:59:09.387878 25419 tablet_bootstrap.cc:492] T 709470e8d22046dd87f2a20e8a8ce745 P 2ff0106754a84b39989b9e042b4af3d6: No bootstrap required, opened a new log
I20250109 08:59:09.388303 25419 ts_tablet_manager.cc:1397] T 709470e8d22046dd87f2a20e8a8ce745 P 2ff0106754a84b39989b9e042b4af3d6: Time spent bootstrapping tablet: real 0.039s user 0.015s sys 0.020s
I20250109 08:59:09.388770 25411 tablet_bootstrap.cc:492] T 9f829cc37f3b447a800c8ee3c51bd346 P 9d26e8b39691431590fd77e182046c66: No bootstrap required, opened a new log
I20250109 08:59:09.389323 25411 ts_tablet_manager.cc:1397] T 9f829cc37f3b447a800c8ee3c51bd346 P 9d26e8b39691431590fd77e182046c66: Time spent bootstrapping tablet: real 0.028s user 0.022s sys 0.004s
I20250109 08:59:09.390349 25412 tablet_bootstrap.cc:492] T 9f829cc37f3b447a800c8ee3c51bd346 P e6d654389db14a49a1f1bf3dbe538a27: No bootstrap required, opened a new log
I20250109 08:59:09.390476 25419 raft_consensus.cc:357] T 709470e8d22046dd87f2a20e8a8ce745 P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:09.390951 25412 ts_tablet_manager.cc:1397] T 9f829cc37f3b447a800c8ee3c51bd346 P e6d654389db14a49a1f1bf3dbe538a27: Time spent bootstrapping tablet: real 0.016s user 0.007s sys 0.004s
I20250109 08:59:09.391187 25419 raft_consensus.cc:383] T 709470e8d22046dd87f2a20e8a8ce745 P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:59:09.391480 25419 raft_consensus.cc:738] T 709470e8d22046dd87f2a20e8a8ce745 P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ff0106754a84b39989b9e042b4af3d6, State: Initialized, Role: FOLLOWER
I20250109 08:59:09.392033 25419 consensus_queue.cc:260] T 709470e8d22046dd87f2a20e8a8ce745 P 2ff0106754a84b39989b9e042b4af3d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:09.392167 25411 raft_consensus.cc:357] T 9f829cc37f3b447a800c8ee3c51bd346 P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:09.392992 25411 raft_consensus.cc:383] T 9f829cc37f3b447a800c8ee3c51bd346 P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:59:09.393275 25411 raft_consensus.cc:738] T 9f829cc37f3b447a800c8ee3c51bd346 P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d26e8b39691431590fd77e182046c66, State: Initialized, Role: FOLLOWER
I20250109 08:59:09.394146 25419 ts_tablet_manager.cc:1428] T 709470e8d22046dd87f2a20e8a8ce745 P 2ff0106754a84b39989b9e042b4af3d6: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20250109 08:59:09.393747 25412 raft_consensus.cc:357] T 9f829cc37f3b447a800c8ee3c51bd346 P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:09.393950 25411 consensus_queue.cc:260] T 9f829cc37f3b447a800c8ee3c51bd346 P 9d26e8b39691431590fd77e182046c66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:09.394558 25412 raft_consensus.cc:383] T 9f829cc37f3b447a800c8ee3c51bd346 P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:59:09.394846 25412 raft_consensus.cc:738] T 9f829cc37f3b447a800c8ee3c51bd346 P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6d654389db14a49a1f1bf3dbe538a27, State: Initialized, Role: FOLLOWER
I20250109 08:59:09.395509 25412 consensus_queue.cc:260] T 9f829cc37f3b447a800c8ee3c51bd346 P e6d654389db14a49a1f1bf3dbe538a27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:09.396507 25411 ts_tablet_manager.cc:1428] T 9f829cc37f3b447a800c8ee3c51bd346 P 9d26e8b39691431590fd77e182046c66: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20250109 08:59:09.397433 25411 tablet_bootstrap.cc:492] T da8d4429a2364a3781155ecc9a3b18b3 P 9d26e8b39691431590fd77e182046c66: Bootstrap starting.
I20250109 08:59:09.398087 25412 ts_tablet_manager.cc:1428] T 9f829cc37f3b447a800c8ee3c51bd346 P e6d654389db14a49a1f1bf3dbe538a27: Time spent starting tablet: real 0.007s user 0.004s sys 0.004s
I20250109 08:59:09.399556 25415 raft_consensus.cc:491] T 075f0030bd964a1687ccee531080fd1c P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:59:09.400044 25415 raft_consensus.cc:513] T 075f0030bd964a1687ccee531080fd1c P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:09.402575 25415 leader_election.cc:290] T 075f0030bd964a1687ccee531080fd1c P 9d26e8b39691431590fd77e182046c66 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2ff0106754a84b39989b9e042b4af3d6 (127.19.226.193:34345), e6d654389db14a49a1f1bf3dbe538a27 (127.19.226.195:39911)
I20250109 08:59:09.403275 25419 tablet_bootstrap.cc:492] T 422f918bde184ce1ae2532c9dee609a7 P 2ff0106754a84b39989b9e042b4af3d6: Bootstrap starting.
I20250109 08:59:09.403553 25411 tablet_bootstrap.cc:654] T da8d4429a2364a3781155ecc9a3b18b3 P 9d26e8b39691431590fd77e182046c66: Neither blocks nor log segments found. Creating new log.
I20250109 08:59:09.418046 25419 tablet_bootstrap.cc:654] T 422f918bde184ce1ae2532c9dee609a7 P 2ff0106754a84b39989b9e042b4af3d6: Neither blocks nor log segments found. Creating new log.
I20250109 08:59:09.422374 25412 tablet_bootstrap.cc:492] T 075f0030bd964a1687ccee531080fd1c P e6d654389db14a49a1f1bf3dbe538a27: Bootstrap starting.
I20250109 08:59:09.425657 25222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "075f0030bd964a1687ccee531080fd1c" candidate_uuid: "9d26e8b39691431590fd77e182046c66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ff0106754a84b39989b9e042b4af3d6" is_pre_election: true
W20250109 08:59:09.427210 25261 leader_election.cc:343] T 075f0030bd964a1687ccee531080fd1c P 9d26e8b39691431590fd77e182046c66 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2ff0106754a84b39989b9e042b4af3d6 (127.19.226.193:34345): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:59:09.427892 25371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "075f0030bd964a1687ccee531080fd1c" candidate_uuid: "9d26e8b39691431590fd77e182046c66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6d654389db14a49a1f1bf3dbe538a27" is_pre_election: true
W20250109 08:59:09.429669 25263 leader_election.cc:343] T 075f0030bd964a1687ccee531080fd1c P 9d26e8b39691431590fd77e182046c66 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e6d654389db14a49a1f1bf3dbe538a27 (127.19.226.195:39911): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:59:09.430109 25263 leader_election.cc:304] T 075f0030bd964a1687ccee531080fd1c P 9d26e8b39691431590fd77e182046c66 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9d26e8b39691431590fd77e182046c66; no voters: 2ff0106754a84b39989b9e042b4af3d6, e6d654389db14a49a1f1bf3dbe538a27
I20250109 08:59:09.430322 25412 tablet_bootstrap.cc:654] T 075f0030bd964a1687ccee531080fd1c P e6d654389db14a49a1f1bf3dbe538a27: Neither blocks nor log segments found. Creating new log.
I20250109 08:59:09.432251 25411 tablet_bootstrap.cc:492] T da8d4429a2364a3781155ecc9a3b18b3 P 9d26e8b39691431590fd77e182046c66: No bootstrap required, opened a new log
I20250109 08:59:09.432801 25415 raft_consensus.cc:2743] T 075f0030bd964a1687ccee531080fd1c P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:59:09.432924 25411 ts_tablet_manager.cc:1397] T da8d4429a2364a3781155ecc9a3b18b3 P 9d26e8b39691431590fd77e182046c66: Time spent bootstrapping tablet: real 0.036s user 0.004s sys 0.008s
I20250109 08:59:09.436807 25419 tablet_bootstrap.cc:492] T 422f918bde184ce1ae2532c9dee609a7 P 2ff0106754a84b39989b9e042b4af3d6: No bootstrap required, opened a new log
I20250109 08:59:09.436668 25411 raft_consensus.cc:357] T da8d4429a2364a3781155ecc9a3b18b3 P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:09.437355 25419 ts_tablet_manager.cc:1397] T 422f918bde184ce1ae2532c9dee609a7 P 2ff0106754a84b39989b9e042b4af3d6: Time spent bootstrapping tablet: real 0.034s user 0.009s sys 0.006s
I20250109 08:59:09.437448 25411 raft_consensus.cc:383] T da8d4429a2364a3781155ecc9a3b18b3 P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:59:09.437836 25411 raft_consensus.cc:738] T da8d4429a2364a3781155ecc9a3b18b3 P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d26e8b39691431590fd77e182046c66, State: Initialized, Role: FOLLOWER
I20250109 08:59:09.438560 25411 consensus_queue.cc:260] T da8d4429a2364a3781155ecc9a3b18b3 P 9d26e8b39691431590fd77e182046c66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:09.440047 25419 raft_consensus.cc:357] T 422f918bde184ce1ae2532c9dee609a7 P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:09.440886 25419 raft_consensus.cc:383] T 422f918bde184ce1ae2532c9dee609a7 P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:59:09.441185 25419 raft_consensus.cc:738] T 422f918bde184ce1ae2532c9dee609a7 P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ff0106754a84b39989b9e042b4af3d6, State: Initialized, Role: FOLLOWER
I20250109 08:59:09.442070 25411 ts_tablet_manager.cc:1428] T da8d4429a2364a3781155ecc9a3b18b3 P 9d26e8b39691431590fd77e182046c66: Time spent starting tablet: real 0.009s user 0.006s sys 0.000s
I20250109 08:59:09.441954 25419 consensus_queue.cc:260] T 422f918bde184ce1ae2532c9dee609a7 P 2ff0106754a84b39989b9e042b4af3d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:09.443562 25411 tablet_bootstrap.cc:492] T 486c1bbdc3e247cdbbf13e1f7456e729 P 9d26e8b39691431590fd77e182046c66: Bootstrap starting.
I20250109 08:59:09.446702 25419 ts_tablet_manager.cc:1428] T 422f918bde184ce1ae2532c9dee609a7 P 2ff0106754a84b39989b9e042b4af3d6: Time spent starting tablet: real 0.009s user 0.005s sys 0.000s
I20250109 08:59:09.447763 25419 tablet_bootstrap.cc:492] T da8d4429a2364a3781155ecc9a3b18b3 P 2ff0106754a84b39989b9e042b4af3d6: Bootstrap starting.
I20250109 08:59:09.451319 25411 tablet_bootstrap.cc:654] T 486c1bbdc3e247cdbbf13e1f7456e729 P 9d26e8b39691431590fd77e182046c66: Neither blocks nor log segments found. Creating new log.
I20250109 08:59:09.451547 25412 tablet_bootstrap.cc:492] T 075f0030bd964a1687ccee531080fd1c P e6d654389db14a49a1f1bf3dbe538a27: No bootstrap required, opened a new log
I20250109 08:59:09.452028 25412 ts_tablet_manager.cc:1397] T 075f0030bd964a1687ccee531080fd1c P e6d654389db14a49a1f1bf3dbe538a27: Time spent bootstrapping tablet: real 0.030s user 0.014s sys 0.007s
W20250109 08:59:09.453467 25168 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20250109 08:59:09.454895 25412 raft_consensus.cc:357] T 075f0030bd964a1687ccee531080fd1c P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:09.455783 25412 raft_consensus.cc:383] T 075f0030bd964a1687ccee531080fd1c P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:59:09.456080 25412 raft_consensus.cc:738] T 075f0030bd964a1687ccee531080fd1c P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6d654389db14a49a1f1bf3dbe538a27, State: Initialized, Role: FOLLOWER
I20250109 08:59:09.456640 25419 tablet_bootstrap.cc:654] T da8d4429a2364a3781155ecc9a3b18b3 P 2ff0106754a84b39989b9e042b4af3d6: Neither blocks nor log segments found. Creating new log.
I20250109 08:59:09.456976 25412 consensus_queue.cc:260] T 075f0030bd964a1687ccee531080fd1c P e6d654389db14a49a1f1bf3dbe538a27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:09.462508 25411 tablet_bootstrap.cc:492] T 486c1bbdc3e247cdbbf13e1f7456e729 P 9d26e8b39691431590fd77e182046c66: No bootstrap required, opened a new log
I20250109 08:59:09.463095 25411 ts_tablet_manager.cc:1397] T 486c1bbdc3e247cdbbf13e1f7456e729 P 9d26e8b39691431590fd77e182046c66: Time spent bootstrapping tablet: real 0.020s user 0.008s sys 0.008s
I20250109 08:59:09.463575 25412 ts_tablet_manager.cc:1428] T 075f0030bd964a1687ccee531080fd1c P e6d654389db14a49a1f1bf3dbe538a27: Time spent starting tablet: real 0.011s user 0.004s sys 0.004s
I20250109 08:59:09.464691 25412 tablet_bootstrap.cc:492] T 1d01a6ffe04e4c7bae266bbc80ee5d54 P e6d654389db14a49a1f1bf3dbe538a27: Bootstrap starting.
I20250109 08:59:09.465929 25411 raft_consensus.cc:357] T 486c1bbdc3e247cdbbf13e1f7456e729 P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:09.466553 25411 raft_consensus.cc:383] T 486c1bbdc3e247cdbbf13e1f7456e729 P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:59:09.466778 25411 raft_consensus.cc:738] T 486c1bbdc3e247cdbbf13e1f7456e729 P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d26e8b39691431590fd77e182046c66, State: Initialized, Role: FOLLOWER
I20250109 08:59:09.467312 25411 consensus_queue.cc:260] T 486c1bbdc3e247cdbbf13e1f7456e729 P 9d26e8b39691431590fd77e182046c66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:09.469291 25411 ts_tablet_manager.cc:1428] T 486c1bbdc3e247cdbbf13e1f7456e729 P 9d26e8b39691431590fd77e182046c66: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20250109 08:59:09.469540 25415 raft_consensus.cc:491] T 1d01a6ffe04e4c7bae266bbc80ee5d54 P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:59:09.470224 25411 tablet_bootstrap.cc:492] T c16639dc193644c8bbcf05ed9f02295e P 9d26e8b39691431590fd77e182046c66: Bootstrap starting.
I20250109 08:59:09.470019 25415 raft_consensus.cc:513] T 1d01a6ffe04e4c7bae266bbc80ee5d54 P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } } peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } }
I20250109 08:59:09.471941 25415 leader_election.cc:290] T 1d01a6ffe04e4c7bae266bbc80ee5d54 P 9d26e8b39691431590fd77e182046c66 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e6d654389db14a49a1f1bf3dbe538a27 (127.19.226.195:39911), 2ff0106754a84b39989b9e042b4af3d6 (127.19.226.193:34345)
I20250109 08:59:09.472916 25412 tablet_bootstrap.cc:654] T 1d01a6ffe04e4c7bae266bbc80ee5d54 P e6d654389db14a49a1f1bf3dbe538a27: Neither blocks nor log segments found. Creating new log.
I20250109 08:59:09.473503 25371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d01a6ffe04e4c7bae266bbc80ee5d54" candidate_uuid: "9d26e8b39691431590fd77e182046c66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6d654389db14a49a1f1bf3dbe538a27" is_pre_election: true
I20250109 08:59:09.474058 25222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d01a6ffe04e4c7bae266bbc80ee5d54" candidate_uuid: "9d26e8b39691431590fd77e182046c66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ff0106754a84b39989b9e042b4af3d6" is_pre_election: true
I20250109 08:59:09.474720 25222 raft_consensus.cc:2463] T 1d01a6ffe04e4c7bae266bbc80ee5d54 P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d26e8b39691431590fd77e182046c66 in term 0.
W20250109 08:59:09.474828 25263 leader_election.cc:343] T 1d01a6ffe04e4c7bae266bbc80ee5d54 P 9d26e8b39691431590fd77e182046c66 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e6d654389db14a49a1f1bf3dbe538a27 (127.19.226.195:39911): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:59:09.475915 25411 tablet_bootstrap.cc:654] T c16639dc193644c8bbcf05ed9f02295e P 9d26e8b39691431590fd77e182046c66: Neither blocks nor log segments found. Creating new log.
I20250109 08:59:09.475875 25261 leader_election.cc:304] T 1d01a6ffe04e4c7bae266bbc80ee5d54 P 9d26e8b39691431590fd77e182046c66 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2ff0106754a84b39989b9e042b4af3d6, 9d26e8b39691431590fd77e182046c66; no voters: e6d654389db14a49a1f1bf3dbe538a27
I20250109 08:59:09.476979 25415 raft_consensus.cc:2798] T 1d01a6ffe04e4c7bae266bbc80ee5d54 P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:59:09.477347 25415 raft_consensus.cc:491] T 1d01a6ffe04e4c7bae266bbc80ee5d54 P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:59:09.477674 25419 tablet_bootstrap.cc:492] T da8d4429a2364a3781155ecc9a3b18b3 P 2ff0106754a84b39989b9e042b4af3d6: No bootstrap required, opened a new log
I20250109 08:59:09.478181 25419 ts_tablet_manager.cc:1397] T da8d4429a2364a3781155ecc9a3b18b3 P 2ff0106754a84b39989b9e042b4af3d6: Time spent bootstrapping tablet: real 0.031s user 0.012s sys 0.005s
I20250109 08:59:09.477674 25415 raft_consensus.cc:3054] T 1d01a6ffe04e4c7bae266bbc80ee5d54 P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:59:09.485787 25419 raft_consensus.cc:357] T da8d4429a2364a3781155ecc9a3b18b3 P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:09.486923 25419 raft_consensus.cc:383] T da8d4429a2364a3781155ecc9a3b18b3 P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:59:09.487226 25419 raft_consensus.cc:738] T da8d4429a2364a3781155ecc9a3b18b3 P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ff0106754a84b39989b9e042b4af3d6, State: Initialized, Role: FOLLOWER
I20250109 08:59:09.488051 25419 consensus_queue.cc:260] T da8d4429a2364a3781155ecc9a3b18b3 P 2ff0106754a84b39989b9e042b4af3d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:09.482441 25412 tablet_bootstrap.cc:492] T 1d01a6ffe04e4c7bae266bbc80ee5d54 P e6d654389db14a49a1f1bf3dbe538a27: No bootstrap required, opened a new log
I20250109 08:59:09.490074 25412 ts_tablet_manager.cc:1397] T 1d01a6ffe04e4c7bae266bbc80ee5d54 P e6d654389db14a49a1f1bf3dbe538a27: Time spent bootstrapping tablet: real 0.026s user 0.011s sys 0.000s
I20250109 08:59:09.491618 25419 ts_tablet_manager.cc:1428] T da8d4429a2364a3781155ecc9a3b18b3 P 2ff0106754a84b39989b9e042b4af3d6: Time spent starting tablet: real 0.013s user 0.007s sys 0.000s
I20250109 08:59:09.492861 25419 tablet_bootstrap.cc:492] T 7622fd22d50942618088f5e65422e713 P 2ff0106754a84b39989b9e042b4af3d6: Bootstrap starting.
I20250109 08:59:09.492530 25412 raft_consensus.cc:357] T 1d01a6ffe04e4c7bae266bbc80ee5d54 P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } } peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } }
I20250109 08:59:09.493602 25412 raft_consensus.cc:383] T 1d01a6ffe04e4c7bae266bbc80ee5d54 P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:59:09.493867 25412 raft_consensus.cc:738] T 1d01a6ffe04e4c7bae266bbc80ee5d54 P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6d654389db14a49a1f1bf3dbe538a27, State: Initialized, Role: FOLLOWER
I20250109 08:59:09.494626 25412 consensus_queue.cc:260] T 1d01a6ffe04e4c7bae266bbc80ee5d54 P e6d654389db14a49a1f1bf3dbe538a27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } } peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } }
I20250109 08:59:09.497103 25412 ts_tablet_manager.cc:1428] T 1d01a6ffe04e4c7bae266bbc80ee5d54 P e6d654389db14a49a1f1bf3dbe538a27: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20250109 08:59:09.492892 25415 raft_consensus.cc:513] T 1d01a6ffe04e4c7bae266bbc80ee5d54 P 9d26e8b39691431590fd77e182046c66 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } } peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } }
I20250109 08:59:09.497988 25412 tablet_bootstrap.cc:492] T da8d4429a2364a3781155ecc9a3b18b3 P e6d654389db14a49a1f1bf3dbe538a27: Bootstrap starting.
I20250109 08:59:09.498847 25411 tablet_bootstrap.cc:492] T c16639dc193644c8bbcf05ed9f02295e P 9d26e8b39691431590fd77e182046c66: No bootstrap required, opened a new log
I20250109 08:59:09.499423 25411 ts_tablet_manager.cc:1397] T c16639dc193644c8bbcf05ed9f02295e P 9d26e8b39691431590fd77e182046c66: Time spent bootstrapping tablet: real 0.029s user 0.011s sys 0.016s
I20250109 08:59:09.499640 25415 leader_election.cc:290] T 1d01a6ffe04e4c7bae266bbc80ee5d54 P 9d26e8b39691431590fd77e182046c66 [CANDIDATE]: Term 1 election: Requested vote from peers e6d654389db14a49a1f1bf3dbe538a27 (127.19.226.195:39911), 2ff0106754a84b39989b9e042b4af3d6 (127.19.226.193:34345)
I20250109 08:59:09.500089 25371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d01a6ffe04e4c7bae266bbc80ee5d54" candidate_uuid: "9d26e8b39691431590fd77e182046c66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6d654389db14a49a1f1bf3dbe538a27"
I20250109 08:59:09.500768 25371 raft_consensus.cc:3054] T 1d01a6ffe04e4c7bae266bbc80ee5d54 P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:59:09.502686 25411 raft_consensus.cc:357] T c16639dc193644c8bbcf05ed9f02295e P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:09.503479 25411 raft_consensus.cc:383] T c16639dc193644c8bbcf05ed9f02295e P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:59:09.503808 25419 tablet_bootstrap.cc:654] T 7622fd22d50942618088f5e65422e713 P 2ff0106754a84b39989b9e042b4af3d6: Neither blocks nor log segments found. Creating new log.
I20250109 08:59:09.503779 25411 raft_consensus.cc:738] T c16639dc193644c8bbcf05ed9f02295e P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d26e8b39691431590fd77e182046c66, State: Initialized, Role: FOLLOWER
I20250109 08:59:09.505071 25412 tablet_bootstrap.cc:654] T da8d4429a2364a3781155ecc9a3b18b3 P e6d654389db14a49a1f1bf3dbe538a27: Neither blocks nor log segments found. Creating new log.
I20250109 08:59:09.504662 25411 consensus_queue.cc:260] T c16639dc193644c8bbcf05ed9f02295e P 9d26e8b39691431590fd77e182046c66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:09.506986 25411 ts_tablet_manager.cc:1428] T c16639dc193644c8bbcf05ed9f02295e P 9d26e8b39691431590fd77e182046c66: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20250109 08:59:09.507949 25371 raft_consensus.cc:2463] T 1d01a6ffe04e4c7bae266bbc80ee5d54 P e6d654389db14a49a1f1bf3dbe538a27 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d26e8b39691431590fd77e182046c66 in term 1.
I20250109 08:59:09.542387 25263 leader_election.cc:304] T 1d01a6ffe04e4c7bae266bbc80ee5d54 P 9d26e8b39691431590fd77e182046c66 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d26e8b39691431590fd77e182046c66, e6d654389db14a49a1f1bf3dbe538a27; no voters:
I20250109 08:59:09.561554 25447 raft_consensus.cc:491] T 7622fd22d50942618088f5e65422e713 P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:59:09.562750 25447 raft_consensus.cc:513] T 7622fd22d50942618088f5e65422e713 P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:09.564904 25445 raft_consensus.cc:491] T 422f918bde184ce1ae2532c9dee609a7 P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:59:09.565868 25445 raft_consensus.cc:513] T 422f918bde184ce1ae2532c9dee609a7 P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:09.568861 25297 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7622fd22d50942618088f5e65422e713" candidate_uuid: "e6d654389db14a49a1f1bf3dbe538a27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d26e8b39691431590fd77e182046c66" is_pre_election: true
I20250109 08:59:09.570250 25297 raft_consensus.cc:2463] T 7622fd22d50942618088f5e65422e713 P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6d654389db14a49a1f1bf3dbe538a27 in term 0.
I20250109 08:59:09.571991 25336 leader_election.cc:304] T 7622fd22d50942618088f5e65422e713 P e6d654389db14a49a1f1bf3dbe538a27 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d26e8b39691431590fd77e182046c66, e6d654389db14a49a1f1bf3dbe538a27; no voters:
I20250109 08:59:09.576875 25446 raft_consensus.cc:491] T 9f829cc37f3b447a800c8ee3c51bd346 P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:59:09.579847 25446 raft_consensus.cc:513] T 9f829cc37f3b447a800c8ee3c51bd346 P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:09.580308 25222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d01a6ffe04e4c7bae266bbc80ee5d54" candidate_uuid: "9d26e8b39691431590fd77e182046c66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ff0106754a84b39989b9e042b4af3d6"
I20250109 08:59:09.581265 25222 raft_consensus.cc:3054] T 1d01a6ffe04e4c7bae266bbc80ee5d54 P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:59:09.582590 25446 leader_election.cc:290] T 9f829cc37f3b447a800c8ee3c51bd346 P e6d654389db14a49a1f1bf3dbe538a27 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2ff0106754a84b39989b9e042b4af3d6 (127.19.226.193:34345), 9d26e8b39691431590fd77e182046c66 (127.19.226.194:33163)
I20250109 08:59:09.583237 25446 raft_consensus.cc:491] T 709470e8d22046dd87f2a20e8a8ce745 P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:59:09.588318 25445 leader_election.cc:290] T 422f918bde184ce1ae2532c9dee609a7 P 2ff0106754a84b39989b9e042b4af3d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9d26e8b39691431590fd77e182046c66 (127.19.226.194:33163), e6d654389db14a49a1f1bf3dbe538a27 (127.19.226.195:39911)
I20250109 08:59:09.589018 25445 raft_consensus.cc:491] T 709470e8d22046dd87f2a20e8a8ce745 P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:59:09.589630 25445 raft_consensus.cc:513] T 709470e8d22046dd87f2a20e8a8ce745 P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:09.591966 25445 leader_election.cc:290] T 709470e8d22046dd87f2a20e8a8ce745 P 2ff0106754a84b39989b9e042b4af3d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9d26e8b39691431590fd77e182046c66 (127.19.226.194:33163), e6d654389db14a49a1f1bf3dbe538a27 (127.19.226.195:39911)
I20250109 08:59:09.583961 25446 raft_consensus.cc:513] T 709470e8d22046dd87f2a20e8a8ce745 P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:09.596630 25450 raft_consensus.cc:491] T 075f0030bd964a1687ccee531080fd1c P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:59:09.597442 25450 raft_consensus.cc:513] T 075f0030bd964a1687ccee531080fd1c P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:09.598774 25446 leader_election.cc:290] T 709470e8d22046dd87f2a20e8a8ce745 P e6d654389db14a49a1f1bf3dbe538a27 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2ff0106754a84b39989b9e042b4af3d6 (127.19.226.193:34345), 9d26e8b39691431590fd77e182046c66 (127.19.226.194:33163)
I20250109 08:59:09.599535 25450 leader_election.cc:290] T 075f0030bd964a1687ccee531080fd1c P e6d654389db14a49a1f1bf3dbe538a27 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2ff0106754a84b39989b9e042b4af3d6 (127.19.226.193:34345), 9d26e8b39691431590fd77e182046c66 (127.19.226.194:33163)
I20250109 08:59:09.601467 25297 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f829cc37f3b447a800c8ee3c51bd346" candidate_uuid: "e6d654389db14a49a1f1bf3dbe538a27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d26e8b39691431590fd77e182046c66" is_pre_election: true
I20250109 08:59:09.602475 25297 raft_consensus.cc:2463] T 9f829cc37f3b447a800c8ee3c51bd346 P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6d654389db14a49a1f1bf3dbe538a27 in term 0.
I20250109 08:59:09.604914 25297 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "709470e8d22046dd87f2a20e8a8ce745" candidate_uuid: "e6d654389db14a49a1f1bf3dbe538a27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d26e8b39691431590fd77e182046c66" is_pre_election: true
I20250109 08:59:09.605847 25297 raft_consensus.cc:2463] T 709470e8d22046dd87f2a20e8a8ce745 P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6d654389db14a49a1f1bf3dbe538a27 in term 0.
I20250109 08:59:09.607965 25297 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "075f0030bd964a1687ccee531080fd1c" candidate_uuid: "e6d654389db14a49a1f1bf3dbe538a27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d26e8b39691431590fd77e182046c66" is_pre_election: true
I20250109 08:59:09.608928 25297 raft_consensus.cc:2463] T 075f0030bd964a1687ccee531080fd1c P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6d654389db14a49a1f1bf3dbe538a27 in term 0.
I20250109 08:59:09.609141 25222 raft_consensus.cc:2463] T 1d01a6ffe04e4c7bae266bbc80ee5d54 P 2ff0106754a84b39989b9e042b4af3d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d26e8b39691431590fd77e182046c66 in term 1.
I20250109 08:59:09.611383 25415 raft_consensus.cc:2798] T 1d01a6ffe04e4c7bae266bbc80ee5d54 P 9d26e8b39691431590fd77e182046c66 [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:59:09.616209 25222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7622fd22d50942618088f5e65422e713" candidate_uuid: "e6d654389db14a49a1f1bf3dbe538a27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ff0106754a84b39989b9e042b4af3d6" is_pre_election: true
I20250109 08:59:09.617861 25221 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f829cc37f3b447a800c8ee3c51bd346" candidate_uuid: "e6d654389db14a49a1f1bf3dbe538a27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ff0106754a84b39989b9e042b4af3d6" is_pre_election: true
I20250109 08:59:09.619619 25221 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "709470e8d22046dd87f2a20e8a8ce745" candidate_uuid: "e6d654389db14a49a1f1bf3dbe538a27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ff0106754a84b39989b9e042b4af3d6" is_pre_election: true
I20250109 08:59:09.620187 25221 raft_consensus.cc:2463] T 709470e8d22046dd87f2a20e8a8ce745 P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6d654389db14a49a1f1bf3dbe538a27 in term 0.
I20250109 08:59:09.621488 25221 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "075f0030bd964a1687ccee531080fd1c" candidate_uuid: "e6d654389db14a49a1f1bf3dbe538a27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ff0106754a84b39989b9e042b4af3d6" is_pre_election: true
I20250109 08:59:09.610267 25336 leader_election.cc:304] T 9f829cc37f3b447a800c8ee3c51bd346 P e6d654389db14a49a1f1bf3dbe538a27 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d26e8b39691431590fd77e182046c66, e6d654389db14a49a1f1bf3dbe538a27; no voters:
W20250109 08:59:09.622961 25335 leader_election.cc:343] T 7622fd22d50942618088f5e65422e713 P e6d654389db14a49a1f1bf3dbe538a27 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2ff0106754a84b39989b9e042b4af3d6 (127.19.226.193:34345): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:59:09.623553 25416 raft_consensus.cc:2798] T 9f829cc37f3b447a800c8ee3c51bd346 P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:59:09.624032 25416 raft_consensus.cc:491] T 9f829cc37f3b447a800c8ee3c51bd346 P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20250109 08:59:09.624181 25335 leader_election.cc:343] T 9f829cc37f3b447a800c8ee3c51bd346 P e6d654389db14a49a1f1bf3dbe538a27 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2ff0106754a84b39989b9e042b4af3d6 (127.19.226.193:34345): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:59:09.624405 25416 raft_consensus.cc:3054] T 9f829cc37f3b447a800c8ee3c51bd346 P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:59:09.625226 25335 leader_election.cc:304] T 709470e8d22046dd87f2a20e8a8ce745 P e6d654389db14a49a1f1bf3dbe538a27 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ff0106754a84b39989b9e042b4af3d6, e6d654389db14a49a1f1bf3dbe538a27; no voters:
W20250109 08:59:09.626689 25335 leader_election.cc:343] T 075f0030bd964a1687ccee531080fd1c P e6d654389db14a49a1f1bf3dbe538a27 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2ff0106754a84b39989b9e042b4af3d6 (127.19.226.193:34345): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:59:09.628592 25297 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "422f918bde184ce1ae2532c9dee609a7" candidate_uuid: "2ff0106754a84b39989b9e042b4af3d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d26e8b39691431590fd77e182046c66" is_pre_election: true
I20250109 08:59:09.629475 25297 raft_consensus.cc:2463] T 422f918bde184ce1ae2532c9dee609a7 P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ff0106754a84b39989b9e042b4af3d6 in term 0.
I20250109 08:59:09.631357 25296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "709470e8d22046dd87f2a20e8a8ce745" candidate_uuid: "2ff0106754a84b39989b9e042b4af3d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d26e8b39691431590fd77e182046c66" is_pre_election: true
I20250109 08:59:09.632037 25296 raft_consensus.cc:2463] T 709470e8d22046dd87f2a20e8a8ce745 P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ff0106754a84b39989b9e042b4af3d6 in term 0.
I20250109 08:59:09.633913 25424 raft_consensus.cc:491] T da8d4429a2364a3781155ecc9a3b18b3 P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:59:09.634514 25424 raft_consensus.cc:513] T da8d4429a2364a3781155ecc9a3b18b3 P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:09.635356 25416 raft_consensus.cc:513] T 9f829cc37f3b447a800c8ee3c51bd346 P e6d654389db14a49a1f1bf3dbe538a27 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:09.636766 25424 leader_election.cc:290] T da8d4429a2364a3781155ecc9a3b18b3 P 2ff0106754a84b39989b9e042b4af3d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9d26e8b39691431590fd77e182046c66 (127.19.226.194:33163), e6d654389db14a49a1f1bf3dbe538a27 (127.19.226.195:39911)
I20250109 08:59:09.633338 25187 leader_election.cc:304] T 422f918bde184ce1ae2532c9dee609a7 P 2ff0106754a84b39989b9e042b4af3d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ff0106754a84b39989b9e042b4af3d6, 9d26e8b39691431590fd77e182046c66; no voters:
I20250109 08:59:09.637554 25336 leader_election.cc:304] T 075f0030bd964a1687ccee531080fd1c P e6d654389db14a49a1f1bf3dbe538a27 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9d26e8b39691431590fd77e182046c66, e6d654389db14a49a1f1bf3dbe538a27; no voters: 2ff0106754a84b39989b9e042b4af3d6
I20250109 08:59:09.614557 25419 tablet_bootstrap.cc:492] T 7622fd22d50942618088f5e65422e713 P 2ff0106754a84b39989b9e042b4af3d6: No bootstrap required, opened a new log
I20250109 08:59:09.638509 25446 raft_consensus.cc:2798] T 709470e8d22046dd87f2a20e8a8ce745 P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:59:09.639272 25446 raft_consensus.cc:491] T 709470e8d22046dd87f2a20e8a8ce745 P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:59:09.594583 25451 raft_consensus.cc:2798] T 7622fd22d50942618088f5e65422e713 P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:59:09.638384 25424 raft_consensus.cc:2798] T 422f918bde184ce1ae2532c9dee609a7 P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:59:09.639696 25446 raft_consensus.cc:3054] T 709470e8d22046dd87f2a20e8a8ce745 P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:59:09.614948 25412 tablet_bootstrap.cc:492] T da8d4429a2364a3781155ecc9a3b18b3 P e6d654389db14a49a1f1bf3dbe538a27: No bootstrap required, opened a new log
I20250109 08:59:09.640220 25451 raft_consensus.cc:491] T 7622fd22d50942618088f5e65422e713 P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:59:09.640323 25424 raft_consensus.cc:491] T 422f918bde184ce1ae2532c9dee609a7 P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:59:09.639604 25419 ts_tablet_manager.cc:1397] T 7622fd22d50942618088f5e65422e713 P 2ff0106754a84b39989b9e042b4af3d6: Time spent bootstrapping tablet: real 0.147s user 0.014s sys 0.005s
I20250109 08:59:09.640723 25451 raft_consensus.cc:3054] T 7622fd22d50942618088f5e65422e713 P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:59:09.640903 25412 ts_tablet_manager.cc:1397] T da8d4429a2364a3781155ecc9a3b18b3 P e6d654389db14a49a1f1bf3dbe538a27: Time spent bootstrapping tablet: real 0.143s user 0.011s sys 0.007s
I20250109 08:59:09.642138 25371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "422f918bde184ce1ae2532c9dee609a7" candidate_uuid: "2ff0106754a84b39989b9e042b4af3d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6d654389db14a49a1f1bf3dbe538a27" is_pre_election: true
I20250109 08:59:09.642570 25415 raft_consensus.cc:695] T 1d01a6ffe04e4c7bae266bbc80ee5d54 P 9d26e8b39691431590fd77e182046c66 [term 1 LEADER]: Becoming Leader. State: Replica: 9d26e8b39691431590fd77e182046c66, State: Running, Role: LEADER
I20250109 08:59:09.643245 25371 raft_consensus.cc:2463] T 422f918bde184ce1ae2532c9dee609a7 P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ff0106754a84b39989b9e042b4af3d6 in term 0.
I20250109 08:59:09.644217 25424 raft_consensus.cc:3054] T 422f918bde184ce1ae2532c9dee609a7 P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:59:09.645166 25415 consensus_queue.cc:237] T 1d01a6ffe04e4c7bae266bbc80ee5d54 P 9d26e8b39691431590fd77e182046c66 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } } peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } }
I20250109 08:59:09.645624 25412 raft_consensus.cc:357] T da8d4429a2364a3781155ecc9a3b18b3 P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:09.646870 25412 raft_consensus.cc:383] T da8d4429a2364a3781155ecc9a3b18b3 P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:59:09.647190 25412 raft_consensus.cc:738] T da8d4429a2364a3781155ecc9a3b18b3 P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6d654389db14a49a1f1bf3dbe538a27, State: Initialized, Role: FOLLOWER
I20250109 08:59:09.645584 25371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "709470e8d22046dd87f2a20e8a8ce745" candidate_uuid: "2ff0106754a84b39989b9e042b4af3d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6d654389db14a49a1f1bf3dbe538a27" is_pre_election: true
I20250109 08:59:09.646570 25419 raft_consensus.cc:357] T 7622fd22d50942618088f5e65422e713 P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:09.648499 25412 consensus_queue.cc:260] T da8d4429a2364a3781155ecc9a3b18b3 P e6d654389db14a49a1f1bf3dbe538a27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:09.648352 25419 raft_consensus.cc:383] T 7622fd22d50942618088f5e65422e713 P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:59:09.650279 25187 leader_election.cc:304] T 709470e8d22046dd87f2a20e8a8ce745 P 2ff0106754a84b39989b9e042b4af3d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ff0106754a84b39989b9e042b4af3d6, 9d26e8b39691431590fd77e182046c66; no voters:
I20250109 08:59:09.661497 25445 raft_consensus.cc:2798] T 709470e8d22046dd87f2a20e8a8ce745 P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:59:09.661959 25445 raft_consensus.cc:491] T 709470e8d22046dd87f2a20e8a8ce745 P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:59:09.655007 25450 raft_consensus.cc:2798] T 075f0030bd964a1687ccee531080fd1c P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:59:09.662977 25450 raft_consensus.cc:491] T 075f0030bd964a1687ccee531080fd1c P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:59:09.663344 25450 raft_consensus.cc:3054] T 075f0030bd964a1687ccee531080fd1c P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:59:09.663583 25221 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f829cc37f3b447a800c8ee3c51bd346" candidate_uuid: "e6d654389db14a49a1f1bf3dbe538a27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ff0106754a84b39989b9e042b4af3d6"
I20250109 08:59:09.650856 25370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da8d4429a2364a3781155ecc9a3b18b3" candidate_uuid: "2ff0106754a84b39989b9e042b4af3d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6d654389db14a49a1f1bf3dbe538a27" is_pre_election: true
W20250109 08:59:09.665202 25335 leader_election.cc:343] T 9f829cc37f3b447a800c8ee3c51bd346 P e6d654389db14a49a1f1bf3dbe538a27 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2ff0106754a84b39989b9e042b4af3d6 (127.19.226.193:34345): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:59:09.665752 25370 raft_consensus.cc:2463] T da8d4429a2364a3781155ecc9a3b18b3 P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ff0106754a84b39989b9e042b4af3d6 in term 0.
I20250109 08:59:09.666360 25296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da8d4429a2364a3781155ecc9a3b18b3" candidate_uuid: "2ff0106754a84b39989b9e042b4af3d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d26e8b39691431590fd77e182046c66" is_pre_election: true
I20250109 08:59:09.660213 25297 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f829cc37f3b447a800c8ee3c51bd346" candidate_uuid: "e6d654389db14a49a1f1bf3dbe538a27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d26e8b39691431590fd77e182046c66"
I20250109 08:59:09.667163 25188 leader_election.cc:304] T da8d4429a2364a3781155ecc9a3b18b3 P 2ff0106754a84b39989b9e042b4af3d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ff0106754a84b39989b9e042b4af3d6, e6d654389db14a49a1f1bf3dbe538a27; no voters:
I20250109 08:59:09.667199 25296 raft_consensus.cc:2463] T da8d4429a2364a3781155ecc9a3b18b3 P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ff0106754a84b39989b9e042b4af3d6 in term 0.
I20250109 08:59:09.667938 25297 raft_consensus.cc:3054] T 9f829cc37f3b447a800c8ee3c51bd346 P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:59:09.652555 25416 leader_election.cc:290] T 9f829cc37f3b447a800c8ee3c51bd346 P e6d654389db14a49a1f1bf3dbe538a27 [CANDIDATE]: Term 1 election: Requested vote from peers 2ff0106754a84b39989b9e042b4af3d6 (127.19.226.193:34345), 9d26e8b39691431590fd77e182046c66 (127.19.226.194:33163)
I20250109 08:59:09.662379 25445 raft_consensus.cc:3054] T 709470e8d22046dd87f2a20e8a8ce745 P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:59:09.654805 25424 raft_consensus.cc:513] T 422f918bde184ce1ae2532c9dee609a7 P 2ff0106754a84b39989b9e042b4af3d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:09.656571 25419 raft_consensus.cc:738] T 7622fd22d50942618088f5e65422e713 P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ff0106754a84b39989b9e042b4af3d6, State: Initialized, Role: FOLLOWER
I20250109 08:59:09.672925 25419 consensus_queue.cc:260] T 7622fd22d50942618088f5e65422e713 P 2ff0106754a84b39989b9e042b4af3d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:09.678748 25297 raft_consensus.cc:2463] T 9f829cc37f3b447a800c8ee3c51bd346 P 9d26e8b39691431590fd77e182046c66 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6d654389db14a49a1f1bf3dbe538a27 in term 1.
I20250109 08:59:09.614475 25447 leader_election.cc:290] T 7622fd22d50942618088f5e65422e713 P e6d654389db14a49a1f1bf3dbe538a27 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2ff0106754a84b39989b9e042b4af3d6 (127.19.226.193:34345), 9d26e8b39691431590fd77e182046c66 (127.19.226.194:33163)
I20250109 08:59:09.680158 25445 raft_consensus.cc:513] T 709470e8d22046dd87f2a20e8a8ce745 P 2ff0106754a84b39989b9e042b4af3d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:09.681228 25296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "422f918bde184ce1ae2532c9dee609a7" candidate_uuid: "2ff0106754a84b39989b9e042b4af3d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d26e8b39691431590fd77e182046c66"
I20250109 08:59:09.682004 25296 raft_consensus.cc:3054] T 422f918bde184ce1ae2532c9dee609a7 P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:59:09.682514 25445 leader_election.cc:290] T 709470e8d22046dd87f2a20e8a8ce745 P 2ff0106754a84b39989b9e042b4af3d6 [CANDIDATE]: Term 1 election: Requested vote from peers 9d26e8b39691431590fd77e182046c66 (127.19.226.194:33163), e6d654389db14a49a1f1bf3dbe538a27 (127.19.226.195:39911)
I20250109 08:59:09.672847 25424 leader_election.cc:290] T 422f918bde184ce1ae2532c9dee609a7 P 2ff0106754a84b39989b9e042b4af3d6 [CANDIDATE]: Term 1 election: Requested vote from peers 9d26e8b39691431590fd77e182046c66 (127.19.226.194:33163), e6d654389db14a49a1f1bf3dbe538a27 (127.19.226.195:39911)
I20250109 08:59:09.685184 25295 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "709470e8d22046dd87f2a20e8a8ce745" candidate_uuid: "2ff0106754a84b39989b9e042b4af3d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d26e8b39691431590fd77e182046c66"
I20250109 08:59:09.685973 25295 raft_consensus.cc:3054] T 709470e8d22046dd87f2a20e8a8ce745 P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:59:09.687224 25451 raft_consensus.cc:513] T 7622fd22d50942618088f5e65422e713 P e6d654389db14a49a1f1bf3dbe538a27 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:09.683326 25445 raft_consensus.cc:2798] T da8d4429a2364a3781155ecc9a3b18b3 P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:59:09.690176 25445 raft_consensus.cc:491] T da8d4429a2364a3781155ecc9a3b18b3 P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:59:09.690559 25445 raft_consensus.cc:3054] T da8d4429a2364a3781155ecc9a3b18b3 P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:59:09.690411 25221 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7622fd22d50942618088f5e65422e713" candidate_uuid: "e6d654389db14a49a1f1bf3dbe538a27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ff0106754a84b39989b9e042b4af3d6"
I20250109 08:59:09.691155 25221 raft_consensus.cc:3054] T 7622fd22d50942618088f5e65422e713 P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:59:09.691350 25451 leader_election.cc:290] T 7622fd22d50942618088f5e65422e713 P e6d654389db14a49a1f1bf3dbe538a27 [CANDIDATE]: Term 1 election: Requested vote from peers 2ff0106754a84b39989b9e042b4af3d6 (127.19.226.193:34345), 9d26e8b39691431590fd77e182046c66 (127.19.226.194:33163)
I20250109 08:59:09.695032 25450 raft_consensus.cc:513] T 075f0030bd964a1687ccee531080fd1c P e6d654389db14a49a1f1bf3dbe538a27 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:09.698551 25294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7622fd22d50942618088f5e65422e713" candidate_uuid: "e6d654389db14a49a1f1bf3dbe538a27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d26e8b39691431590fd77e182046c66"
I20250109 08:59:09.699288 25294 raft_consensus.cc:3054] T 7622fd22d50942618088f5e65422e713 P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:59:09.702930 25295 raft_consensus.cc:2463] T 709470e8d22046dd87f2a20e8a8ce745 P 9d26e8b39691431590fd77e182046c66 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ff0106754a84b39989b9e042b4af3d6 in term 1.
I20250109 08:59:09.704241 25187 leader_election.cc:304] T 709470e8d22046dd87f2a20e8a8ce745 P 2ff0106754a84b39989b9e042b4af3d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ff0106754a84b39989b9e042b4af3d6, 9d26e8b39691431590fd77e182046c66; no voters:
I20250109 08:59:09.705374 25450 leader_election.cc:290] T 075f0030bd964a1687ccee531080fd1c P e6d654389db14a49a1f1bf3dbe538a27 [CANDIDATE]: Term 1 election: Requested vote from peers 2ff0106754a84b39989b9e042b4af3d6 (127.19.226.193:34345), 9d26e8b39691431590fd77e182046c66 (127.19.226.194:33163)
I20250109 08:59:09.705734 25222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "075f0030bd964a1687ccee531080fd1c" candidate_uuid: "e6d654389db14a49a1f1bf3dbe538a27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ff0106754a84b39989b9e042b4af3d6"
I20250109 08:59:09.707141 25221 raft_consensus.cc:2463] T 7622fd22d50942618088f5e65422e713 P 2ff0106754a84b39989b9e042b4af3d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6d654389db14a49a1f1bf3dbe538a27 in term 1.
W20250109 08:59:09.707373 25335 leader_election.cc:343] T 075f0030bd964a1687ccee531080fd1c P e6d654389db14a49a1f1bf3dbe538a27 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2ff0106754a84b39989b9e042b4af3d6 (127.19.226.193:34345): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:59:09.709738 25369 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "709470e8d22046dd87f2a20e8a8ce745" candidate_uuid: "2ff0106754a84b39989b9e042b4af3d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6d654389db14a49a1f1bf3dbe538a27"
I20250109 08:59:09.711175 25335 leader_election.cc:304] T 7622fd22d50942618088f5e65422e713 P e6d654389db14a49a1f1bf3dbe538a27 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ff0106754a84b39989b9e042b4af3d6, e6d654389db14a49a1f1bf3dbe538a27; no voters:
I20250109 08:59:09.712178 25370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "422f918bde184ce1ae2532c9dee609a7" candidate_uuid: "2ff0106754a84b39989b9e042b4af3d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6d654389db14a49a1f1bf3dbe538a27"
I20250109 08:59:09.712951 25370 raft_consensus.cc:3054] T 422f918bde184ce1ae2532c9dee609a7 P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:59:09.717749 25450 raft_consensus.cc:2798] T 7622fd22d50942618088f5e65422e713 P e6d654389db14a49a1f1bf3dbe538a27 [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:59:09.718362 25450 raft_consensus.cc:695] T 7622fd22d50942618088f5e65422e713 P e6d654389db14a49a1f1bf3dbe538a27 [term 1 LEADER]: Becoming Leader. State: Replica: e6d654389db14a49a1f1bf3dbe538a27, State: Running, Role: LEADER
I20250109 08:59:09.717314 25455 raft_consensus.cc:2798] T 709470e8d22046dd87f2a20e8a8ce745 P 2ff0106754a84b39989b9e042b4af3d6 [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:59:09.719650 25452 raft_consensus.cc:491] T c16639dc193644c8bbcf05ed9f02295e P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:59:09.720336 25455 raft_consensus.cc:695] T 709470e8d22046dd87f2a20e8a8ce745 P 2ff0106754a84b39989b9e042b4af3d6 [term 1 LEADER]: Becoming Leader. State: Replica: 2ff0106754a84b39989b9e042b4af3d6, State: Running, Role: LEADER
I20250109 08:59:09.720273 25452 raft_consensus.cc:513] T c16639dc193644c8bbcf05ed9f02295e P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:09.721647 25455 consensus_queue.cc:237] T 709470e8d22046dd87f2a20e8a8ce745 P 2ff0106754a84b39989b9e042b4af3d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:09.723457 25452 leader_election.cc:290] T c16639dc193644c8bbcf05ed9f02295e P 9d26e8b39691431590fd77e182046c66 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2ff0106754a84b39989b9e042b4af3d6 (127.19.226.193:34345), e6d654389db14a49a1f1bf3dbe538a27 (127.19.226.195:39911)
I20250109 08:59:09.724854 25445 raft_consensus.cc:513] T da8d4429a2364a3781155ecc9a3b18b3 P 2ff0106754a84b39989b9e042b4af3d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:09.658933 25446 raft_consensus.cc:513] T 709470e8d22046dd87f2a20e8a8ce745 P e6d654389db14a49a1f1bf3dbe538a27 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:09.728263 25369 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c16639dc193644c8bbcf05ed9f02295e" candidate_uuid: "9d26e8b39691431590fd77e182046c66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6d654389db14a49a1f1bf3dbe538a27" is_pre_election: true
I20250109 08:59:09.729178 25369 raft_consensus.cc:2463] T c16639dc193644c8bbcf05ed9f02295e P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d26e8b39691431590fd77e182046c66 in term 0.
I20250109 08:59:09.730227 25221 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c16639dc193644c8bbcf05ed9f02295e" candidate_uuid: "9d26e8b39691431590fd77e182046c66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ff0106754a84b39989b9e042b4af3d6" is_pre_election: true
I20250109 08:59:09.730724 25263 leader_election.cc:304] T c16639dc193644c8bbcf05ed9f02295e P 9d26e8b39691431590fd77e182046c66 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d26e8b39691431590fd77e182046c66, e6d654389db14a49a1f1bf3dbe538a27; no voters:
I20250109 08:59:09.729382 25371 raft_consensus.cc:2388] T 709470e8d22046dd87f2a20e8a8ce745 P e6d654389db14a49a1f1bf3dbe538a27 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2ff0106754a84b39989b9e042b4af3d6 in current term 1: Already voted for candidate e6d654389db14a49a1f1bf3dbe538a27 in this term.
I20250109 08:59:09.731544 25221 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "709470e8d22046dd87f2a20e8a8ce745" candidate_uuid: "e6d654389db14a49a1f1bf3dbe538a27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ff0106754a84b39989b9e042b4af3d6"
W20250109 08:59:09.731720 25261 leader_election.cc:343] T c16639dc193644c8bbcf05ed9f02295e P 9d26e8b39691431590fd77e182046c66 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2ff0106754a84b39989b9e042b4af3d6 (127.19.226.193:34345): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:59:09.734051 25446 leader_election.cc:290] T 709470e8d22046dd87f2a20e8a8ce745 P e6d654389db14a49a1f1bf3dbe538a27 [CANDIDATE]: Term 1 election: Requested vote from peers 2ff0106754a84b39989b9e042b4af3d6 (127.19.226.193:34345), 9d26e8b39691431590fd77e182046c66 (127.19.226.194:33163)
I20250109 08:59:09.729115 25450 consensus_queue.cc:237] T 7622fd22d50942618088f5e65422e713 P e6d654389db14a49a1f1bf3dbe538a27 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:09.734587 25452 raft_consensus.cc:2798] T c16639dc193644c8bbcf05ed9f02295e P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:59:09.738867 25452 raft_consensus.cc:491] T c16639dc193644c8bbcf05ed9f02295e P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:59:09.739436 25452 raft_consensus.cc:3054] T c16639dc193644c8bbcf05ed9f02295e P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:59:09.741077 25412 ts_tablet_manager.cc:1428] T da8d4429a2364a3781155ecc9a3b18b3 P e6d654389db14a49a1f1bf3dbe538a27: Time spent starting tablet: real 0.100s user 0.019s sys 0.025s
I20250109 08:59:09.750026 25294 raft_consensus.cc:2463] T 7622fd22d50942618088f5e65422e713 P 9d26e8b39691431590fd77e182046c66 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6d654389db14a49a1f1bf3dbe538a27 in term 1.
I20250109 08:59:09.757715 25445 leader_election.cc:290] T da8d4429a2364a3781155ecc9a3b18b3 P 2ff0106754a84b39989b9e042b4af3d6 [CANDIDATE]: Term 1 election: Requested vote from peers 9d26e8b39691431590fd77e182046c66 (127.19.226.194:33163), e6d654389db14a49a1f1bf3dbe538a27 (127.19.226.195:39911)
I20250109 08:59:09.758642 25297 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da8d4429a2364a3781155ecc9a3b18b3" candidate_uuid: "2ff0106754a84b39989b9e042b4af3d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d26e8b39691431590fd77e182046c66"
I20250109 08:59:09.759382 25297 raft_consensus.cc:3054] T da8d4429a2364a3781155ecc9a3b18b3 P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:59:09.761611 25336 leader_election.cc:304] T 9f829cc37f3b447a800c8ee3c51bd346 P e6d654389db14a49a1f1bf3dbe538a27 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9d26e8b39691431590fd77e182046c66, e6d654389db14a49a1f1bf3dbe538a27; no voters: 2ff0106754a84b39989b9e042b4af3d6
I20250109 08:59:09.763433 25453 raft_consensus.cc:2798] T 9f829cc37f3b447a800c8ee3c51bd346 P e6d654389db14a49a1f1bf3dbe538a27 [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:59:09.763947 25453 raft_consensus.cc:695] T 9f829cc37f3b447a800c8ee3c51bd346 P e6d654389db14a49a1f1bf3dbe538a27 [term 1 LEADER]: Becoming Leader. State: Replica: e6d654389db14a49a1f1bf3dbe538a27, State: Running, Role: LEADER
I20250109 08:59:09.764273 25371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da8d4429a2364a3781155ecc9a3b18b3" candidate_uuid: "2ff0106754a84b39989b9e042b4af3d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6d654389db14a49a1f1bf3dbe538a27"
I20250109 08:59:09.765254 25371 raft_consensus.cc:3054] T da8d4429a2364a3781155ecc9a3b18b3 P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:59:09.764935 25453 consensus_queue.cc:237] T 9f829cc37f3b447a800c8ee3c51bd346 P e6d654389db14a49a1f1bf3dbe538a27 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:09.767477 25295 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "709470e8d22046dd87f2a20e8a8ce745" candidate_uuid: "e6d654389db14a49a1f1bf3dbe538a27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d26e8b39691431590fd77e182046c66"
I20250109 08:59:09.768258 25295 raft_consensus.cc:2388] T 709470e8d22046dd87f2a20e8a8ce745 P 9d26e8b39691431590fd77e182046c66 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate e6d654389db14a49a1f1bf3dbe538a27 in current term 1: Already voted for candidate 2ff0106754a84b39989b9e042b4af3d6 in this term.
I20250109 08:59:09.775694 25370 raft_consensus.cc:2463] T 422f918bde184ce1ae2532c9dee609a7 P e6d654389db14a49a1f1bf3dbe538a27 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ff0106754a84b39989b9e042b4af3d6 in term 1.
I20250109 08:59:09.776139 25297 raft_consensus.cc:2463] T da8d4429a2364a3781155ecc9a3b18b3 P 9d26e8b39691431590fd77e182046c66 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ff0106754a84b39989b9e042b4af3d6 in term 1.
I20250109 08:59:09.776707 25336 leader_election.cc:304] T 709470e8d22046dd87f2a20e8a8ce745 P e6d654389db14a49a1f1bf3dbe538a27 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e6d654389db14a49a1f1bf3dbe538a27; no voters: 2ff0106754a84b39989b9e042b4af3d6, 9d26e8b39691431590fd77e182046c66
I20250109 08:59:09.777308 25187 leader_election.cc:304] T da8d4429a2364a3781155ecc9a3b18b3 P 2ff0106754a84b39989b9e042b4af3d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ff0106754a84b39989b9e042b4af3d6, 9d26e8b39691431590fd77e182046c66; no voters:
I20250109 08:59:09.778246 25447 raft_consensus.cc:2743] T 709470e8d22046dd87f2a20e8a8ce745 P e6d654389db14a49a1f1bf3dbe538a27 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20250109 08:59:09.779103 25188 leader_election.cc:304] T 422f918bde184ce1ae2532c9dee609a7 P 2ff0106754a84b39989b9e042b4af3d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ff0106754a84b39989b9e042b4af3d6, e6d654389db14a49a1f1bf3dbe538a27; no voters:
I20250109 08:59:09.779613 25445 raft_consensus.cc:2798] T da8d4429a2364a3781155ecc9a3b18b3 P 2ff0106754a84b39989b9e042b4af3d6 [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:59:09.780264 25455 raft_consensus.cc:2798] T 422f918bde184ce1ae2532c9dee609a7 P 2ff0106754a84b39989b9e042b4af3d6 [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:59:09.780202 25445 raft_consensus.cc:695] T da8d4429a2364a3781155ecc9a3b18b3 P 2ff0106754a84b39989b9e042b4af3d6 [term 1 LEADER]: Becoming Leader. State: Replica: 2ff0106754a84b39989b9e042b4af3d6, State: Running, Role: LEADER
I20250109 08:59:09.780853 25455 raft_consensus.cc:695] T 422f918bde184ce1ae2532c9dee609a7 P 2ff0106754a84b39989b9e042b4af3d6 [term 1 LEADER]: Becoming Leader. State: Replica: 2ff0106754a84b39989b9e042b4af3d6, State: Running, Role: LEADER
I20250109 08:59:09.781548 25445 consensus_queue.cc:237] T da8d4429a2364a3781155ecc9a3b18b3 P 2ff0106754a84b39989b9e042b4af3d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:09.781199 25118 catalog_manager.cc:5526] T 709470e8d22046dd87f2a20e8a8ce745 P 2ff0106754a84b39989b9e042b4af3d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2ff0106754a84b39989b9e042b4af3d6 (127.19.226.193). New cstate: current_term: 1 leader_uuid: "2ff0106754a84b39989b9e042b4af3d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } health_report { overall_health: UNKNOWN } } }
I20250109 08:59:09.782038 25455 consensus_queue.cc:237] T 422f918bde184ce1ae2532c9dee609a7 P 2ff0106754a84b39989b9e042b4af3d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:09.787539 25294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "075f0030bd964a1687ccee531080fd1c" candidate_uuid: "e6d654389db14a49a1f1bf3dbe538a27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d26e8b39691431590fd77e182046c66"
I20250109 08:59:09.788447 25294 raft_consensus.cc:3054] T 075f0030bd964a1687ccee531080fd1c P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:59:09.778344 25452 raft_consensus.cc:513] T c16639dc193644c8bbcf05ed9f02295e P 9d26e8b39691431590fd77e182046c66 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:09.793401 25452 leader_election.cc:290] T c16639dc193644c8bbcf05ed9f02295e P 9d26e8b39691431590fd77e182046c66 [CANDIDATE]: Term 1 election: Requested vote from peers 2ff0106754a84b39989b9e042b4af3d6 (127.19.226.193:34345), e6d654389db14a49a1f1bf3dbe538a27 (127.19.226.195:39911)
I20250109 08:59:09.794656 25221 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c16639dc193644c8bbcf05ed9f02295e" candidate_uuid: "9d26e8b39691431590fd77e182046c66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ff0106754a84b39989b9e042b4af3d6"
W20250109 08:59:09.804735 25261 leader_election.cc:343] T c16639dc193644c8bbcf05ed9f02295e P 9d26e8b39691431590fd77e182046c66 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2ff0106754a84b39989b9e042b4af3d6 (127.19.226.193:34345): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:59:09.809346 25370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c16639dc193644c8bbcf05ed9f02295e" candidate_uuid: "9d26e8b39691431590fd77e182046c66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6d654389db14a49a1f1bf3dbe538a27"
I20250109 08:59:09.810164 25370 raft_consensus.cc:3054] T c16639dc193644c8bbcf05ed9f02295e P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:59:09.796357 25296 raft_consensus.cc:2463] T 422f918bde184ce1ae2532c9dee609a7 P 9d26e8b39691431590fd77e182046c66 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ff0106754a84b39989b9e042b4af3d6 in term 1.
I20250109 08:59:09.811998 25294 raft_consensus.cc:2463] T 075f0030bd964a1687ccee531080fd1c P 9d26e8b39691431590fd77e182046c66 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6d654389db14a49a1f1bf3dbe538a27 in term 1.
I20250109 08:59:09.814935 25445 raft_consensus.cc:491] T 486c1bbdc3e247cdbbf13e1f7456e729 P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:59:09.815135 25336 leader_election.cc:304] T 075f0030bd964a1687ccee531080fd1c P e6d654389db14a49a1f1bf3dbe538a27 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9d26e8b39691431590fd77e182046c66, e6d654389db14a49a1f1bf3dbe538a27; no voters: 2ff0106754a84b39989b9e042b4af3d6
I20250109 08:59:09.815584 25445 raft_consensus.cc:513] T 486c1bbdc3e247cdbbf13e1f7456e729 P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:09.816619 25453 raft_consensus.cc:2798] T 075f0030bd964a1687ccee531080fd1c P e6d654389db14a49a1f1bf3dbe538a27 [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:59:09.817137 25453 raft_consensus.cc:695] T 075f0030bd964a1687ccee531080fd1c P e6d654389db14a49a1f1bf3dbe538a27 [term 1 LEADER]: Becoming Leader. State: Replica: e6d654389db14a49a1f1bf3dbe538a27, State: Running, Role: LEADER
I20250109 08:59:09.817759 25445 leader_election.cc:290] T 486c1bbdc3e247cdbbf13e1f7456e729 P 2ff0106754a84b39989b9e042b4af3d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9d26e8b39691431590fd77e182046c66 (127.19.226.194:33163), e6d654389db14a49a1f1bf3dbe538a27 (127.19.226.195:39911)
I20250109 08:59:09.818111 25370 raft_consensus.cc:2463] T c16639dc193644c8bbcf05ed9f02295e P e6d654389db14a49a1f1bf3dbe538a27 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d26e8b39691431590fd77e182046c66 in term 1.
I20250109 08:59:09.818135 25453 consensus_queue.cc:237] T 075f0030bd964a1687ccee531080fd1c P e6d654389db14a49a1f1bf3dbe538a27 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:09.819509 25369 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "486c1bbdc3e247cdbbf13e1f7456e729" candidate_uuid: "2ff0106754a84b39989b9e042b4af3d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6d654389db14a49a1f1bf3dbe538a27" is_pre_election: true
I20250109 08:59:09.820252 25369 raft_consensus.cc:2463] T 486c1bbdc3e247cdbbf13e1f7456e729 P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ff0106754a84b39989b9e042b4af3d6 in term 0.
I20250109 08:59:09.820931 25371 raft_consensus.cc:2463] T da8d4429a2364a3781155ecc9a3b18b3 P e6d654389db14a49a1f1bf3dbe538a27 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ff0106754a84b39989b9e042b4af3d6 in term 1.
I20250109 08:59:09.821765 25188 leader_election.cc:304] T 486c1bbdc3e247cdbbf13e1f7456e729 P 2ff0106754a84b39989b9e042b4af3d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ff0106754a84b39989b9e042b4af3d6, e6d654389db14a49a1f1bf3dbe538a27; no voters:
I20250109 08:59:09.825057 25296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "486c1bbdc3e247cdbbf13e1f7456e729" candidate_uuid: "2ff0106754a84b39989b9e042b4af3d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d26e8b39691431590fd77e182046c66" is_pre_election: true
I20250109 08:59:09.825911 25296 raft_consensus.cc:2463] T 486c1bbdc3e247cdbbf13e1f7456e729 P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ff0106754a84b39989b9e042b4af3d6 in term 0.
I20250109 08:59:09.827986 25263 leader_election.cc:304] T c16639dc193644c8bbcf05ed9f02295e P 9d26e8b39691431590fd77e182046c66 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9d26e8b39691431590fd77e182046c66, e6d654389db14a49a1f1bf3dbe538a27; no voters: 2ff0106754a84b39989b9e042b4af3d6
I20250109 08:59:09.828944 25415 raft_consensus.cc:2798] T c16639dc193644c8bbcf05ed9f02295e P 9d26e8b39691431590fd77e182046c66 [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:59:09.830698 25415 raft_consensus.cc:695] T c16639dc193644c8bbcf05ed9f02295e P 9d26e8b39691431590fd77e182046c66 [term 1 LEADER]: Becoming Leader. State: Replica: 9d26e8b39691431590fd77e182046c66, State: Running, Role: LEADER
I20250109 08:59:09.823200 25445 raft_consensus.cc:2798] T 486c1bbdc3e247cdbbf13e1f7456e729 P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:59:09.831210 25445 raft_consensus.cc:491] T 486c1bbdc3e247cdbbf13e1f7456e729 P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:59:09.831650 25445 raft_consensus.cc:3054] T 486c1bbdc3e247cdbbf13e1f7456e729 P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:59:09.831750 25415 consensus_queue.cc:237] T c16639dc193644c8bbcf05ed9f02295e P 9d26e8b39691431590fd77e182046c66 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:09.845839 25419 ts_tablet_manager.cc:1428] T 7622fd22d50942618088f5e65422e713 P 2ff0106754a84b39989b9e042b4af3d6: Time spent starting tablet: real 0.205s user 0.022s sys 0.029s
I20250109 08:59:09.847436 25419 tablet_bootstrap.cc:492] T 075f0030bd964a1687ccee531080fd1c P 2ff0106754a84b39989b9e042b4af3d6: Bootstrap starting.
I20250109 08:59:09.848804 25117 catalog_manager.cc:5526] T 7622fd22d50942618088f5e65422e713 P e6d654389db14a49a1f1bf3dbe538a27 reported cstate change: term changed from 0 to 1, leader changed from <none> to e6d654389db14a49a1f1bf3dbe538a27 (127.19.226.195). New cstate: current_term: 1 leader_uuid: "e6d654389db14a49a1f1bf3dbe538a27" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } health_report { overall_health: HEALTHY } } }
I20250109 08:59:09.847052 25119 catalog_manager.cc:5526] T 1d01a6ffe04e4c7bae266bbc80ee5d54 P 9d26e8b39691431590fd77e182046c66 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9d26e8b39691431590fd77e182046c66 (127.19.226.194). New cstate: current_term: 1 leader_uuid: "9d26e8b39691431590fd77e182046c66" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } health_report { overall_health: HEALTHY } } }
I20250109 08:59:09.860168 25419 tablet_bootstrap.cc:654] T 075f0030bd964a1687ccee531080fd1c P 2ff0106754a84b39989b9e042b4af3d6: Neither blocks nor log segments found. Creating new log.
I20250109 08:59:09.859688 25445 raft_consensus.cc:513] T 486c1bbdc3e247cdbbf13e1f7456e729 P 2ff0106754a84b39989b9e042b4af3d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:09.862807 25296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "486c1bbdc3e247cdbbf13e1f7456e729" candidate_uuid: "2ff0106754a84b39989b9e042b4af3d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d26e8b39691431590fd77e182046c66"
I20250109 08:59:09.863504 25296 raft_consensus.cc:3054] T 486c1bbdc3e247cdbbf13e1f7456e729 P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:59:09.871330 25296 raft_consensus.cc:2463] T 486c1bbdc3e247cdbbf13e1f7456e729 P 9d26e8b39691431590fd77e182046c66 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ff0106754a84b39989b9e042b4af3d6 in term 1.
I20250109 08:59:09.872531 25187 leader_election.cc:304] T 486c1bbdc3e247cdbbf13e1f7456e729 P 2ff0106754a84b39989b9e042b4af3d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ff0106754a84b39989b9e042b4af3d6, 9d26e8b39691431590fd77e182046c66; no voters:
I20250109 08:59:09.874871 25456 raft_consensus.cc:2798] T 486c1bbdc3e247cdbbf13e1f7456e729 P 2ff0106754a84b39989b9e042b4af3d6 [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:59:09.875406 25456 raft_consensus.cc:695] T 486c1bbdc3e247cdbbf13e1f7456e729 P 2ff0106754a84b39989b9e042b4af3d6 [term 1 LEADER]: Becoming Leader. State: Replica: 2ff0106754a84b39989b9e042b4af3d6, State: Running, Role: LEADER
I20250109 08:59:09.876480 25456 consensus_queue.cc:237] T 486c1bbdc3e247cdbbf13e1f7456e729 P 2ff0106754a84b39989b9e042b4af3d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:09.880131 25419 tablet_bootstrap.cc:492] T 075f0030bd964a1687ccee531080fd1c P 2ff0106754a84b39989b9e042b4af3d6: No bootstrap required, opened a new log
I20250109 08:59:09.880759 25419 ts_tablet_manager.cc:1397] T 075f0030bd964a1687ccee531080fd1c P 2ff0106754a84b39989b9e042b4af3d6: Time spent bootstrapping tablet: real 0.034s user 0.019s sys 0.008s
I20250109 08:59:09.883833 25419 raft_consensus.cc:357] T 075f0030bd964a1687ccee531080fd1c P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:09.884738 25419 raft_consensus.cc:383] T 075f0030bd964a1687ccee531080fd1c P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:59:09.885061 25419 raft_consensus.cc:738] T 075f0030bd964a1687ccee531080fd1c P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ff0106754a84b39989b9e042b4af3d6, State: Initialized, Role: FOLLOWER
I20250109 08:59:09.885749 25419 consensus_queue.cc:260] T 075f0030bd964a1687ccee531080fd1c P 2ff0106754a84b39989b9e042b4af3d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:09.888263 25419 ts_tablet_manager.cc:1428] T 075f0030bd964a1687ccee531080fd1c P 2ff0106754a84b39989b9e042b4af3d6: Time spent starting tablet: real 0.007s user 0.004s sys 0.000s
I20250109 08:59:09.889612 25419 tablet_bootstrap.cc:492] T c16639dc193644c8bbcf05ed9f02295e P 2ff0106754a84b39989b9e042b4af3d6: Bootstrap starting.
I20250109 08:59:09.892931 25445 leader_election.cc:290] T 486c1bbdc3e247cdbbf13e1f7456e729 P 2ff0106754a84b39989b9e042b4af3d6 [CANDIDATE]: Term 1 election: Requested vote from peers 9d26e8b39691431590fd77e182046c66 (127.19.226.194:33163), e6d654389db14a49a1f1bf3dbe538a27 (127.19.226.195:39911)
I20250109 08:59:09.894196 25370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "486c1bbdc3e247cdbbf13e1f7456e729" candidate_uuid: "2ff0106754a84b39989b9e042b4af3d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6d654389db14a49a1f1bf3dbe538a27"
I20250109 08:59:09.894892 25370 raft_consensus.cc:3054] T 486c1bbdc3e247cdbbf13e1f7456e729 P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:59:09.896113 25419 tablet_bootstrap.cc:654] T c16639dc193644c8bbcf05ed9f02295e P 2ff0106754a84b39989b9e042b4af3d6: Neither blocks nor log segments found. Creating new log.
I20250109 08:59:09.897183 25118 catalog_manager.cc:5526] T 486c1bbdc3e247cdbbf13e1f7456e729 P 2ff0106754a84b39989b9e042b4af3d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2ff0106754a84b39989b9e042b4af3d6 (127.19.226.193). New cstate: current_term: 1 leader_uuid: "2ff0106754a84b39989b9e042b4af3d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } health_report { overall_health: UNKNOWN } } }
I20250109 08:59:09.898602 25119 catalog_manager.cc:5526] T c16639dc193644c8bbcf05ed9f02295e P 9d26e8b39691431590fd77e182046c66 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9d26e8b39691431590fd77e182046c66 (127.19.226.194). New cstate: current_term: 1 leader_uuid: "9d26e8b39691431590fd77e182046c66" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } health_report { overall_health: UNKNOWN } } }
I20250109 08:59:09.899065 25118 catalog_manager.cc:5526] T da8d4429a2364a3781155ecc9a3b18b3 P 2ff0106754a84b39989b9e042b4af3d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2ff0106754a84b39989b9e042b4af3d6 (127.19.226.193). New cstate: current_term: 1 leader_uuid: "2ff0106754a84b39989b9e042b4af3d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } health_report { overall_health: UNKNOWN } } }
I20250109 08:59:09.900978 25118 catalog_manager.cc:5526] T 422f918bde184ce1ae2532c9dee609a7 P 2ff0106754a84b39989b9e042b4af3d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2ff0106754a84b39989b9e042b4af3d6 (127.19.226.193). New cstate: current_term: 1 leader_uuid: "2ff0106754a84b39989b9e042b4af3d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } health_report { overall_health: UNKNOWN } } }
I20250109 08:59:09.907442 25370 raft_consensus.cc:2463] T 486c1bbdc3e247cdbbf13e1f7456e729 P e6d654389db14a49a1f1bf3dbe538a27 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ff0106754a84b39989b9e042b4af3d6 in term 1.
I20250109 08:59:09.910984 25419 tablet_bootstrap.cc:492] T c16639dc193644c8bbcf05ed9f02295e P 2ff0106754a84b39989b9e042b4af3d6: No bootstrap required, opened a new log
I20250109 08:59:09.911533 25419 ts_tablet_manager.cc:1397] T c16639dc193644c8bbcf05ed9f02295e P 2ff0106754a84b39989b9e042b4af3d6: Time spent bootstrapping tablet: real 0.022s user 0.009s sys 0.011s
I20250109 08:59:09.913879 25419 raft_consensus.cc:357] T c16639dc193644c8bbcf05ed9f02295e P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:09.914507 25419 raft_consensus.cc:383] T c16639dc193644c8bbcf05ed9f02295e P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:59:09.914753 25419 raft_consensus.cc:738] T c16639dc193644c8bbcf05ed9f02295e P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ff0106754a84b39989b9e042b4af3d6, State: Initialized, Role: FOLLOWER
I20250109 08:59:09.915367 25419 consensus_queue.cc:260] T c16639dc193644c8bbcf05ed9f02295e P 2ff0106754a84b39989b9e042b4af3d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:09.917531 25419 ts_tablet_manager.cc:1428] T c16639dc193644c8bbcf05ed9f02295e P 2ff0106754a84b39989b9e042b4af3d6: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20250109 08:59:09.916688 25117 catalog_manager.cc:5526] T 9f829cc37f3b447a800c8ee3c51bd346 P e6d654389db14a49a1f1bf3dbe538a27 reported cstate change: term changed from 0 to 1, leader changed from <none> to e6d654389db14a49a1f1bf3dbe538a27 (127.19.226.195). New cstate: current_term: 1 leader_uuid: "e6d654389db14a49a1f1bf3dbe538a27" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } health_report { overall_health: HEALTHY } } }
I20250109 08:59:09.918650 25419 tablet_bootstrap.cc:492] T 9f829cc37f3b447a800c8ee3c51bd346 P 2ff0106754a84b39989b9e042b4af3d6: Bootstrap starting.
I20250109 08:59:09.918829 25117 catalog_manager.cc:5526] T 075f0030bd964a1687ccee531080fd1c P e6d654389db14a49a1f1bf3dbe538a27 reported cstate change: term changed from 0 to 1, leader changed from <none> to e6d654389db14a49a1f1bf3dbe538a27 (127.19.226.195). New cstate: current_term: 1 leader_uuid: "e6d654389db14a49a1f1bf3dbe538a27" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } health_report { overall_health: HEALTHY } } }
I20250109 08:59:09.924296 25419 tablet_bootstrap.cc:654] T 9f829cc37f3b447a800c8ee3c51bd346 P 2ff0106754a84b39989b9e042b4af3d6: Neither blocks nor log segments found. Creating new log.
I20250109 08:59:09.929728 25419 tablet_bootstrap.cc:492] T 9f829cc37f3b447a800c8ee3c51bd346 P 2ff0106754a84b39989b9e042b4af3d6: No bootstrap required, opened a new log
I20250109 08:59:09.930157 25419 ts_tablet_manager.cc:1397] T 9f829cc37f3b447a800c8ee3c51bd346 P 2ff0106754a84b39989b9e042b4af3d6: Time spent bootstrapping tablet: real 0.012s user 0.009s sys 0.000s
I20250109 08:59:09.932149 25419 raft_consensus.cc:357] T 9f829cc37f3b447a800c8ee3c51bd346 P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:09.932649 25419 raft_consensus.cc:383] T 9f829cc37f3b447a800c8ee3c51bd346 P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:59:09.932816 25419 raft_consensus.cc:738] T 9f829cc37f3b447a800c8ee3c51bd346 P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ff0106754a84b39989b9e042b4af3d6, State: Initialized, Role: FOLLOWER
I20250109 08:59:09.933276 25419 consensus_queue.cc:260] T 9f829cc37f3b447a800c8ee3c51bd346 P 2ff0106754a84b39989b9e042b4af3d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:09.934825 25419 ts_tablet_manager.cc:1428] T 9f829cc37f3b447a800c8ee3c51bd346 P 2ff0106754a84b39989b9e042b4af3d6: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20250109 08:59:10.057816 25296 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "1d01a6ffe04e4c7bae266bbc80ee5d54"
dest_uuid: "9d26e8b39691431590fd77e182046c66"
mode: GRACEFUL
new_leader_uuid: "2ff0106754a84b39989b9e042b4af3d6"
from {username='slave'} at 127.0.0.1:45190
I20250109 08:59:10.058439 25296 raft_consensus.cc:604] T 1d01a6ffe04e4c7bae266bbc80ee5d54 P 9d26e8b39691431590fd77e182046c66 [term 1 LEADER]: Received request to transfer leadership to TS 2ff0106754a84b39989b9e042b4af3d6
I20250109 08:59:10.070505 25370 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "9f829cc37f3b447a800c8ee3c51bd346"
dest_uuid: "e6d654389db14a49a1f1bf3dbe538a27"
mode: GRACEFUL
new_leader_uuid: "2ff0106754a84b39989b9e042b4af3d6"
from {username='slave'} at 127.0.0.1:50372
I20250109 08:59:10.070997 25370 raft_consensus.cc:604] T 9f829cc37f3b447a800c8ee3c51bd346 P e6d654389db14a49a1f1bf3dbe538a27 [term 1 LEADER]: Received request to transfer leadership to TS 2ff0106754a84b39989b9e042b4af3d6
I20250109 08:59:10.075706 25296 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "c16639dc193644c8bbcf05ed9f02295e"
dest_uuid: "9d26e8b39691431590fd77e182046c66"
mode: GRACEFUL
new_leader_uuid: "2ff0106754a84b39989b9e042b4af3d6"
from {username='slave'} at 127.0.0.1:45190
I20250109 08:59:10.076179 25296 raft_consensus.cc:604] T c16639dc193644c8bbcf05ed9f02295e P 9d26e8b39691431590fd77e182046c66 [term 1 LEADER]: Received request to transfer leadership to TS 2ff0106754a84b39989b9e042b4af3d6
I20250109 08:59:10.083178 25371 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "075f0030bd964a1687ccee531080fd1c"
dest_uuid: "e6d654389db14a49a1f1bf3dbe538a27"
mode: GRACEFUL
new_leader_uuid: "9d26e8b39691431590fd77e182046c66"
from {username='slave'} at 127.0.0.1:50372
I20250109 08:59:10.083693 25371 raft_consensus.cc:604] T 075f0030bd964a1687ccee531080fd1c P e6d654389db14a49a1f1bf3dbe538a27 [term 1 LEADER]: Received request to transfer leadership to TS 9d26e8b39691431590fd77e182046c66
I20250109 08:59:10.084498 25415 consensus_queue.cc:1035] T 1d01a6ffe04e4c7bae266bbc80ee5d54 P 9d26e8b39691431590fd77e182046c66 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:59:10.088935 25370 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "7622fd22d50942618088f5e65422e713"
dest_uuid: "e6d654389db14a49a1f1bf3dbe538a27"
mode: GRACEFUL
new_leader_uuid: "9d26e8b39691431590fd77e182046c66"
from {username='slave'} at 127.0.0.1:50372
I20250109 08:59:10.089644 25370 raft_consensus.cc:604] T 7622fd22d50942618088f5e65422e713 P e6d654389db14a49a1f1bf3dbe538a27 [term 1 LEADER]: Received request to transfer leadership to TS 9d26e8b39691431590fd77e182046c66
I20250109 08:59:10.106166 25415 consensus_queue.cc:1035] T 1d01a6ffe04e4c7bae266bbc80ee5d54 P 9d26e8b39691431590fd77e182046c66 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250109 08:59:10.122434 25415 raft_consensus.cc:988] T 1d01a6ffe04e4c7bae266bbc80ee5d54 P 9d26e8b39691431590fd77e182046c66: : Instructing follower 2ff0106754a84b39989b9e042b4af3d6 to start an election
I20250109 08:59:10.122931 25452 raft_consensus.cc:1076] T 1d01a6ffe04e4c7bae266bbc80ee5d54 P 9d26e8b39691431590fd77e182046c66 [term 1 LEADER]: Signalling peer 2ff0106754a84b39989b9e042b4af3d6 to start an election
I20250109 08:59:10.124578 25221 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "1d01a6ffe04e4c7bae266bbc80ee5d54"
dest_uuid: "2ff0106754a84b39989b9e042b4af3d6"
from {username='slave'} at 127.0.0.1:55364
I20250109 08:59:10.124866 25222 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "486c1bbdc3e247cdbbf13e1f7456e729"
dest_uuid: "2ff0106754a84b39989b9e042b4af3d6"
mode: GRACEFUL
new_leader_uuid: "9d26e8b39691431590fd77e182046c66"
from {username='slave'} at 127.0.0.1:55368
I20250109 08:59:10.125217 25221 raft_consensus.cc:491] T 1d01a6ffe04e4c7bae266bbc80ee5d54 P 2ff0106754a84b39989b9e042b4af3d6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:59:10.125500 25222 raft_consensus.cc:604] T 486c1bbdc3e247cdbbf13e1f7456e729 P 2ff0106754a84b39989b9e042b4af3d6 [term 1 LEADER]: Received request to transfer leadership to TS 9d26e8b39691431590fd77e182046c66
I20250109 08:59:10.125669 25221 raft_consensus.cc:3054] T 1d01a6ffe04e4c7bae266bbc80ee5d54 P 2ff0106754a84b39989b9e042b4af3d6 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:59:10.128185 25445 raft_consensus.cc:491] T c16639dc193644c8bbcf05ed9f02295e P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:59:10.128748 25445 raft_consensus.cc:513] T c16639dc193644c8bbcf05ed9f02295e P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:10.130630 25445 leader_election.cc:290] T c16639dc193644c8bbcf05ed9f02295e P 2ff0106754a84b39989b9e042b4af3d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9d26e8b39691431590fd77e182046c66 (127.19.226.194:33163), e6d654389db14a49a1f1bf3dbe538a27 (127.19.226.195:39911)
I20250109 08:59:10.130954 25456 raft_consensus.cc:491] T 075f0030bd964a1687ccee531080fd1c P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:59:10.131469 25456 raft_consensus.cc:513] T 075f0030bd964a1687ccee531080fd1c P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:10.134567 25370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "075f0030bd964a1687ccee531080fd1c" candidate_uuid: "2ff0106754a84b39989b9e042b4af3d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6d654389db14a49a1f1bf3dbe538a27" is_pre_election: true
I20250109 08:59:10.133497 25456 leader_election.cc:290] T 075f0030bd964a1687ccee531080fd1c P 2ff0106754a84b39989b9e042b4af3d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9d26e8b39691431590fd77e182046c66 (127.19.226.194:33163), e6d654389db14a49a1f1bf3dbe538a27 (127.19.226.195:39911)
I20250109 08:59:10.133628 25222 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "709470e8d22046dd87f2a20e8a8ce745"
dest_uuid: "2ff0106754a84b39989b9e042b4af3d6"
mode: GRACEFUL
new_leader_uuid: "9d26e8b39691431590fd77e182046c66"
from {username='slave'} at 127.0.0.1:55368
I20250109 08:59:10.132800 25296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c16639dc193644c8bbcf05ed9f02295e" candidate_uuid: "2ff0106754a84b39989b9e042b4af3d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d26e8b39691431590fd77e182046c66" is_pre_election: true
I20250109 08:59:10.132692 25221 raft_consensus.cc:513] T 1d01a6ffe04e4c7bae266bbc80ee5d54 P 2ff0106754a84b39989b9e042b4af3d6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } } peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } }
I20250109 08:59:10.141011 25222 raft_consensus.cc:604] T 709470e8d22046dd87f2a20e8a8ce745 P 2ff0106754a84b39989b9e042b4af3d6 [term 1 LEADER]: Received request to transfer leadership to TS 9d26e8b39691431590fd77e182046c66
I20250109 08:59:10.134037 25294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "075f0030bd964a1687ccee531080fd1c" candidate_uuid: "2ff0106754a84b39989b9e042b4af3d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d26e8b39691431590fd77e182046c66" is_pre_election: true
I20250109 08:59:10.133502 25371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c16639dc193644c8bbcf05ed9f02295e" candidate_uuid: "2ff0106754a84b39989b9e042b4af3d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6d654389db14a49a1f1bf3dbe538a27" is_pre_election: true
I20250109 08:59:10.142745 25294 raft_consensus.cc:2388] T 075f0030bd964a1687ccee531080fd1c P 9d26e8b39691431590fd77e182046c66 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2ff0106754a84b39989b9e042b4af3d6 in current term 1: Already voted for candidate e6d654389db14a49a1f1bf3dbe538a27 in this term.
I20250109 08:59:10.144083 25296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d01a6ffe04e4c7bae266bbc80ee5d54" candidate_uuid: "2ff0106754a84b39989b9e042b4af3d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9d26e8b39691431590fd77e182046c66"
I20250109 08:59:10.144414 25187 leader_election.cc:304] T 075f0030bd964a1687ccee531080fd1c P 2ff0106754a84b39989b9e042b4af3d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ff0106754a84b39989b9e042b4af3d6; no voters: 9d26e8b39691431590fd77e182046c66, e6d654389db14a49a1f1bf3dbe538a27
I20250109 08:59:10.144862 25296 raft_consensus.cc:3049] T 1d01a6ffe04e4c7bae266bbc80ee5d54 P 9d26e8b39691431590fd77e182046c66 [term 1 LEADER]: Stepping down as leader of term 1
I20250109 08:59:10.145262 25296 raft_consensus.cc:738] T 1d01a6ffe04e4c7bae266bbc80ee5d54 P 9d26e8b39691431590fd77e182046c66 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9d26e8b39691431590fd77e182046c66, State: Running, Role: LEADER
I20250109 08:59:10.145845 25456 raft_consensus.cc:3054] T 075f0030bd964a1687ccee531080fd1c P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:59:10.142800 25221 leader_election.cc:290] T 1d01a6ffe04e4c7bae266bbc80ee5d54 P 2ff0106754a84b39989b9e042b4af3d6 [CANDIDATE]: Term 2 election: Requested vote from peers e6d654389db14a49a1f1bf3dbe538a27 (127.19.226.195:39911), 9d26e8b39691431590fd77e182046c66 (127.19.226.194:33163)
I20250109 08:59:10.146013 25296 consensus_queue.cc:260] T 1d01a6ffe04e4c7bae266bbc80ee5d54 P 9d26e8b39691431590fd77e182046c66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } } peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } }
I20250109 08:59:10.143013 25371 raft_consensus.cc:2388] T c16639dc193644c8bbcf05ed9f02295e P e6d654389db14a49a1f1bf3dbe538a27 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2ff0106754a84b39989b9e042b4af3d6 in current term 1: Already voted for candidate 9d26e8b39691431590fd77e182046c66 in this term.
I20250109 08:59:10.148459 25296 raft_consensus.cc:3054] T 1d01a6ffe04e4c7bae266bbc80ee5d54 P 9d26e8b39691431590fd77e182046c66 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:59:10.149592 25221 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "422f918bde184ce1ae2532c9dee609a7"
dest_uuid: "2ff0106754a84b39989b9e042b4af3d6"
mode: GRACEFUL
new_leader_uuid: "e6d654389db14a49a1f1bf3dbe538a27"
from {username='slave'} at 127.0.0.1:55368
I20250109 08:59:10.149703 25188 leader_election.cc:304] T c16639dc193644c8bbcf05ed9f02295e P 2ff0106754a84b39989b9e042b4af3d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ff0106754a84b39989b9e042b4af3d6; no voters: 9d26e8b39691431590fd77e182046c66, e6d654389db14a49a1f1bf3dbe538a27
I20250109 08:59:10.150346 25221 raft_consensus.cc:604] T 422f918bde184ce1ae2532c9dee609a7 P 2ff0106754a84b39989b9e042b4af3d6 [term 1 LEADER]: Received request to transfer leadership to TS e6d654389db14a49a1f1bf3dbe538a27
I20250109 08:59:10.150851 25445 raft_consensus.cc:3054] T c16639dc193644c8bbcf05ed9f02295e P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:59:10.153832 25456 raft_consensus.cc:2743] T 075f0030bd964a1687ccee531080fd1c P 2ff0106754a84b39989b9e042b4af3d6 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:59:10.153777 25370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d01a6ffe04e4c7bae266bbc80ee5d54" candidate_uuid: "2ff0106754a84b39989b9e042b4af3d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e6d654389db14a49a1f1bf3dbe538a27"
I20250109 08:59:10.154625 25296 raft_consensus.cc:2463] T 1d01a6ffe04e4c7bae266bbc80ee5d54 P 9d26e8b39691431590fd77e182046c66 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ff0106754a84b39989b9e042b4af3d6 in term 2.
I20250109 08:59:10.154798 25370 raft_consensus.cc:3054] T 1d01a6ffe04e4c7bae266bbc80ee5d54 P e6d654389db14a49a1f1bf3dbe538a27 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:59:10.155685 25187 leader_election.cc:304] T 1d01a6ffe04e4c7bae266bbc80ee5d54 P 2ff0106754a84b39989b9e042b4af3d6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ff0106754a84b39989b9e042b4af3d6, 9d26e8b39691431590fd77e182046c66; no voters:
I20250109 08:59:10.156497 25456 raft_consensus.cc:2798] T 1d01a6ffe04e4c7bae266bbc80ee5d54 P 2ff0106754a84b39989b9e042b4af3d6 [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:59:10.156956 25456 raft_consensus.cc:695] T 1d01a6ffe04e4c7bae266bbc80ee5d54 P 2ff0106754a84b39989b9e042b4af3d6 [term 2 LEADER]: Becoming Leader. State: Replica: 2ff0106754a84b39989b9e042b4af3d6, State: Running, Role: LEADER
I20250109 08:59:10.157512 25455 raft_consensus.cc:491] T 9f829cc37f3b447a800c8ee3c51bd346 P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:59:10.157843 25456 consensus_queue.cc:237] T 1d01a6ffe04e4c7bae266bbc80ee5d54 P 2ff0106754a84b39989b9e042b4af3d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } } peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } }
I20250109 08:59:10.157968 25455 raft_consensus.cc:513] T 9f829cc37f3b447a800c8ee3c51bd346 P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:10.158389 25445 raft_consensus.cc:2743] T c16639dc193644c8bbcf05ed9f02295e P 2ff0106754a84b39989b9e042b4af3d6 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:59:10.160075 25455 leader_election.cc:290] T 9f829cc37f3b447a800c8ee3c51bd346 P 2ff0106754a84b39989b9e042b4af3d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9d26e8b39691431590fd77e182046c66 (127.19.226.194:33163), e6d654389db14a49a1f1bf3dbe538a27 (127.19.226.195:39911)
I20250109 08:59:10.161003 25370 raft_consensus.cc:2463] T 1d01a6ffe04e4c7bae266bbc80ee5d54 P e6d654389db14a49a1f1bf3dbe538a27 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ff0106754a84b39989b9e042b4af3d6 in term 2.
I20250109 08:59:10.161372 25371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f829cc37f3b447a800c8ee3c51bd346" candidate_uuid: "2ff0106754a84b39989b9e042b4af3d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6d654389db14a49a1f1bf3dbe538a27" is_pre_election: true
I20250109 08:59:10.163712 25296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f829cc37f3b447a800c8ee3c51bd346" candidate_uuid: "2ff0106754a84b39989b9e042b4af3d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d26e8b39691431590fd77e182046c66" is_pre_election: true
I20250109 08:59:10.164499 25296 raft_consensus.cc:2388] T 9f829cc37f3b447a800c8ee3c51bd346 P 9d26e8b39691431590fd77e182046c66 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2ff0106754a84b39989b9e042b4af3d6 in current term 1: Already voted for candidate e6d654389db14a49a1f1bf3dbe538a27 in this term.
I20250109 08:59:10.165354 25187 leader_election.cc:304] T 9f829cc37f3b447a800c8ee3c51bd346 P 2ff0106754a84b39989b9e042b4af3d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ff0106754a84b39989b9e042b4af3d6; no voters: 9d26e8b39691431590fd77e182046c66, e6d654389db14a49a1f1bf3dbe538a27
I20250109 08:59:10.165979 25455 raft_consensus.cc:3054] T 9f829cc37f3b447a800c8ee3c51bd346 P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:59:10.166098 25118 catalog_manager.cc:5526] T 1d01a6ffe04e4c7bae266bbc80ee5d54 P 2ff0106754a84b39989b9e042b4af3d6 reported cstate change: term changed from 1 to 2, leader changed from 9d26e8b39691431590fd77e182046c66 (127.19.226.194) to 2ff0106754a84b39989b9e042b4af3d6 (127.19.226.193). New cstate: current_term: 2 leader_uuid: "2ff0106754a84b39989b9e042b4af3d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } health_report { overall_health: UNKNOWN } } }
I20250109 08:59:10.168054 25456 consensus_queue.cc:1035] T 422f918bde184ce1ae2532c9dee609a7 P 2ff0106754a84b39989b9e042b4af3d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:59:10.176093 25445 raft_consensus.cc:988] T 422f918bde184ce1ae2532c9dee609a7 P 2ff0106754a84b39989b9e042b4af3d6: : Instructing follower e6d654389db14a49a1f1bf3dbe538a27 to start an election
I20250109 08:59:10.176726 25424 raft_consensus.cc:1076] T 422f918bde184ce1ae2532c9dee609a7 P 2ff0106754a84b39989b9e042b4af3d6 [term 1 LEADER]: Signalling peer e6d654389db14a49a1f1bf3dbe538a27 to start an election
I20250109 08:59:10.177433 25455 raft_consensus.cc:2743] T 9f829cc37f3b447a800c8ee3c51bd346 P 2ff0106754a84b39989b9e042b4af3d6 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:59:10.181732 25424 consensus_queue.cc:1035] T 709470e8d22046dd87f2a20e8a8ce745 P 2ff0106754a84b39989b9e042b4af3d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250109 08:59:10.183565 25424 consensus_queue.cc:1035] T 422f918bde184ce1ae2532c9dee609a7 P 2ff0106754a84b39989b9e042b4af3d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:59:10.184957 25370 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "422f918bde184ce1ae2532c9dee609a7"
dest_uuid: "e6d654389db14a49a1f1bf3dbe538a27"
from {username='slave'} at 127.0.0.1:50366
I20250109 08:59:10.185566 25370 raft_consensus.cc:491] T 422f918bde184ce1ae2532c9dee609a7 P e6d654389db14a49a1f1bf3dbe538a27 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:59:10.185932 25370 raft_consensus.cc:3054] T 422f918bde184ce1ae2532c9dee609a7 P e6d654389db14a49a1f1bf3dbe538a27 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:59:10.193168 25370 raft_consensus.cc:513] T 422f918bde184ce1ae2532c9dee609a7 P e6d654389db14a49a1f1bf3dbe538a27 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:10.196146 25222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "422f918bde184ce1ae2532c9dee609a7" candidate_uuid: "e6d654389db14a49a1f1bf3dbe538a27" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2ff0106754a84b39989b9e042b4af3d6"
I20250109 08:59:10.196662 25296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "422f918bde184ce1ae2532c9dee609a7" candidate_uuid: "e6d654389db14a49a1f1bf3dbe538a27" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9d26e8b39691431590fd77e182046c66"
I20250109 08:59:10.196795 25222 raft_consensus.cc:3049] T 422f918bde184ce1ae2532c9dee609a7 P 2ff0106754a84b39989b9e042b4af3d6 [term 1 LEADER]: Stepping down as leader of term 1
I20250109 08:59:10.197587 25222 raft_consensus.cc:738] T 422f918bde184ce1ae2532c9dee609a7 P 2ff0106754a84b39989b9e042b4af3d6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2ff0106754a84b39989b9e042b4af3d6, State: Running, Role: LEADER
I20250109 08:59:10.198518 25222 consensus_queue.cc:260] T 422f918bde184ce1ae2532c9dee609a7 P 2ff0106754a84b39989b9e042b4af3d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:10.199591 25222 raft_consensus.cc:3054] T 422f918bde184ce1ae2532c9dee609a7 P 2ff0106754a84b39989b9e042b4af3d6 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:59:10.202821 25455 raft_consensus.cc:988] T 709470e8d22046dd87f2a20e8a8ce745 P 2ff0106754a84b39989b9e042b4af3d6: : Instructing follower 9d26e8b39691431590fd77e182046c66 to start an election
I20250109 08:59:10.203294 25455 raft_consensus.cc:1076] T 709470e8d22046dd87f2a20e8a8ce745 P 2ff0106754a84b39989b9e042b4af3d6 [term 1 LEADER]: Signalling peer 9d26e8b39691431590fd77e182046c66 to start an election
I20250109 08:59:10.207619 25455 consensus_queue.cc:1035] T 709470e8d22046dd87f2a20e8a8ce745 P 2ff0106754a84b39989b9e042b4af3d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:59:10.196849 25370 leader_election.cc:290] T 422f918bde184ce1ae2532c9dee609a7 P e6d654389db14a49a1f1bf3dbe538a27 [CANDIDATE]: Term 2 election: Requested vote from peers 2ff0106754a84b39989b9e042b4af3d6 (127.19.226.193:34345), 9d26e8b39691431590fd77e182046c66 (127.19.226.194:33163)
I20250109 08:59:10.207952 25296 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "709470e8d22046dd87f2a20e8a8ce745"
dest_uuid: "9d26e8b39691431590fd77e182046c66"
from {username='slave'} at 127.0.0.1:45178
I20250109 08:59:10.209681 25296 raft_consensus.cc:491] T 709470e8d22046dd87f2a20e8a8ce745 P 9d26e8b39691431590fd77e182046c66 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:59:10.210111 25296 raft_consensus.cc:3054] T 709470e8d22046dd87f2a20e8a8ce745 P 9d26e8b39691431590fd77e182046c66 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:59:10.210436 25222 raft_consensus.cc:2463] T 422f918bde184ce1ae2532c9dee609a7 P 2ff0106754a84b39989b9e042b4af3d6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6d654389db14a49a1f1bf3dbe538a27 in term 2.
I20250109 08:59:10.211587 25335 leader_election.cc:304] T 422f918bde184ce1ae2532c9dee609a7 P e6d654389db14a49a1f1bf3dbe538a27 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2ff0106754a84b39989b9e042b4af3d6, e6d654389db14a49a1f1bf3dbe538a27; no voters: 9d26e8b39691431590fd77e182046c66
I20250109 08:59:10.213099 25453 raft_consensus.cc:2798] T 422f918bde184ce1ae2532c9dee609a7 P e6d654389db14a49a1f1bf3dbe538a27 [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:59:10.213580 25453 raft_consensus.cc:695] T 422f918bde184ce1ae2532c9dee609a7 P e6d654389db14a49a1f1bf3dbe538a27 [term 2 LEADER]: Becoming Leader. State: Replica: e6d654389db14a49a1f1bf3dbe538a27, State: Running, Role: LEADER
I20250109 08:59:10.214429 25453 consensus_queue.cc:237] T 422f918bde184ce1ae2532c9dee609a7 P e6d654389db14a49a1f1bf3dbe538a27 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:10.217545 25296 raft_consensus.cc:513] T 709470e8d22046dd87f2a20e8a8ce745 P 9d26e8b39691431590fd77e182046c66 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:10.220652 25222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "709470e8d22046dd87f2a20e8a8ce745" candidate_uuid: "9d26e8b39691431590fd77e182046c66" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2ff0106754a84b39989b9e042b4af3d6"
I20250109 08:59:10.221311 25222 raft_consensus.cc:3049] T 709470e8d22046dd87f2a20e8a8ce745 P 2ff0106754a84b39989b9e042b4af3d6 [term 1 LEADER]: Stepping down as leader of term 1
I20250109 08:59:10.221606 25222 raft_consensus.cc:738] T 709470e8d22046dd87f2a20e8a8ce745 P 2ff0106754a84b39989b9e042b4af3d6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2ff0106754a84b39989b9e042b4af3d6, State: Running, Role: LEADER
I20250109 08:59:10.222340 25222 consensus_queue.cc:260] T 709470e8d22046dd87f2a20e8a8ce745 P 2ff0106754a84b39989b9e042b4af3d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:10.223748 25222 raft_consensus.cc:3054] T 709470e8d22046dd87f2a20e8a8ce745 P 2ff0106754a84b39989b9e042b4af3d6 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:59:10.225730 25117 catalog_manager.cc:5526] T 422f918bde184ce1ae2532c9dee609a7 P e6d654389db14a49a1f1bf3dbe538a27 reported cstate change: term changed from 1 to 2, leader changed from 2ff0106754a84b39989b9e042b4af3d6 (127.19.226.193) to e6d654389db14a49a1f1bf3dbe538a27 (127.19.226.195). New cstate: current_term: 2 leader_uuid: "e6d654389db14a49a1f1bf3dbe538a27" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } health_report { overall_health: HEALTHY } } }
I20250109 08:59:10.228889 25296 leader_election.cc:290] T 709470e8d22046dd87f2a20e8a8ce745 P 9d26e8b39691431590fd77e182046c66 [CANDIDATE]: Term 2 election: Requested vote from peers 2ff0106754a84b39989b9e042b4af3d6 (127.19.226.193:34345), e6d654389db14a49a1f1bf3dbe538a27 (127.19.226.195:39911)
I20250109 08:59:10.229787 25222 raft_consensus.cc:2463] T 709470e8d22046dd87f2a20e8a8ce745 P 2ff0106754a84b39989b9e042b4af3d6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d26e8b39691431590fd77e182046c66 in term 2.
I20250109 08:59:10.229897 25370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "709470e8d22046dd87f2a20e8a8ce745" candidate_uuid: "9d26e8b39691431590fd77e182046c66" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e6d654389db14a49a1f1bf3dbe538a27"
I20250109 08:59:10.230753 25261 leader_election.cc:304] T 709470e8d22046dd87f2a20e8a8ce745 P 9d26e8b39691431590fd77e182046c66 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ff0106754a84b39989b9e042b4af3d6, 9d26e8b39691431590fd77e182046c66; no voters:
I20250109 08:59:10.231329 25370 raft_consensus.cc:3054] T 709470e8d22046dd87f2a20e8a8ce745 P e6d654389db14a49a1f1bf3dbe538a27 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:59:10.231703 25452 raft_consensus.cc:2798] T 709470e8d22046dd87f2a20e8a8ce745 P 9d26e8b39691431590fd77e182046c66 [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:59:10.232195 25452 raft_consensus.cc:695] T 709470e8d22046dd87f2a20e8a8ce745 P 9d26e8b39691431590fd77e182046c66 [term 2 LEADER]: Becoming Leader. State: Replica: 9d26e8b39691431590fd77e182046c66, State: Running, Role: LEADER
I20250109 08:59:10.232893 25452 consensus_queue.cc:237] T 709470e8d22046dd87f2a20e8a8ce745 P 9d26e8b39691431590fd77e182046c66 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:10.233256 25455 consensus_queue.cc:1035] T da8d4429a2364a3781155ecc9a3b18b3 P 2ff0106754a84b39989b9e042b4af3d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:59:10.238061 25370 raft_consensus.cc:2463] T 709470e8d22046dd87f2a20e8a8ce745 P e6d654389db14a49a1f1bf3dbe538a27 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d26e8b39691431590fd77e182046c66 in term 2.
I20250109 08:59:10.245110 25117 catalog_manager.cc:5526] T 709470e8d22046dd87f2a20e8a8ce745 P 9d26e8b39691431590fd77e182046c66 reported cstate change: term changed from 1 to 2, leader changed from 2ff0106754a84b39989b9e042b4af3d6 (127.19.226.193) to 9d26e8b39691431590fd77e182046c66 (127.19.226.194). New cstate: current_term: 2 leader_uuid: "9d26e8b39691431590fd77e182046c66" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } health_report { overall_health: UNKNOWN } } }
I20250109 08:59:10.246220 25456 consensus_queue.cc:1035] T da8d4429a2364a3781155ecc9a3b18b3 P 2ff0106754a84b39989b9e042b4af3d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:59:10.287775 25453 consensus_queue.cc:1035] T 9f829cc37f3b447a800c8ee3c51bd346 P e6d654389db14a49a1f1bf3dbe538a27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:59:10.293315 25453 consensus_queue.cc:1035] T 075f0030bd964a1687ccee531080fd1c P e6d654389db14a49a1f1bf3dbe538a27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:59:10.294658 25453 raft_consensus.cc:988] T 9f829cc37f3b447a800c8ee3c51bd346 P e6d654389db14a49a1f1bf3dbe538a27: : Instructing follower 2ff0106754a84b39989b9e042b4af3d6 to start an election
I20250109 08:59:10.295689 25450 raft_consensus.cc:1076] T 9f829cc37f3b447a800c8ee3c51bd346 P e6d654389db14a49a1f1bf3dbe538a27 [term 1 LEADER]: Signalling peer 2ff0106754a84b39989b9e042b4af3d6 to start an election
I20250109 08:59:10.297691 25221 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "9f829cc37f3b447a800c8ee3c51bd346"
dest_uuid: "2ff0106754a84b39989b9e042b4af3d6"
from {username='slave'} at 127.0.0.1:55350
I20250109 08:59:10.298360 25221 raft_consensus.cc:491] T 9f829cc37f3b447a800c8ee3c51bd346 P 2ff0106754a84b39989b9e042b4af3d6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:59:10.298769 25221 raft_consensus.cc:3054] T 9f829cc37f3b447a800c8ee3c51bd346 P 2ff0106754a84b39989b9e042b4af3d6 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:59:10.303632 25450 consensus_queue.cc:1035] T 9f829cc37f3b447a800c8ee3c51bd346 P e6d654389db14a49a1f1bf3dbe538a27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250109 08:59:10.306778 25450 raft_consensus.cc:988] T 075f0030bd964a1687ccee531080fd1c P e6d654389db14a49a1f1bf3dbe538a27: : Instructing follower 9d26e8b39691431590fd77e182046c66 to start an election
I20250109 08:59:10.306407 25221 raft_consensus.cc:513] T 9f829cc37f3b447a800c8ee3c51bd346 P 2ff0106754a84b39989b9e042b4af3d6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:10.307268 25450 raft_consensus.cc:1076] T 075f0030bd964a1687ccee531080fd1c P e6d654389db14a49a1f1bf3dbe538a27 [term 1 LEADER]: Signalling peer 9d26e8b39691431590fd77e182046c66 to start an election
I20250109 08:59:10.309260 25296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f829cc37f3b447a800c8ee3c51bd346" candidate_uuid: "2ff0106754a84b39989b9e042b4af3d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9d26e8b39691431590fd77e182046c66"
I20250109 08:59:10.309654 25295 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "075f0030bd964a1687ccee531080fd1c"
dest_uuid: "9d26e8b39691431590fd77e182046c66"
from {username='slave'} at 127.0.0.1:45172
I20250109 08:59:10.310268 25295 raft_consensus.cc:491] T 075f0030bd964a1687ccee531080fd1c P 9d26e8b39691431590fd77e182046c66 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:59:10.310648 25295 raft_consensus.cc:3054] T 075f0030bd964a1687ccee531080fd1c P 9d26e8b39691431590fd77e182046c66 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:59:10.312674 25221 leader_election.cc:290] T 9f829cc37f3b447a800c8ee3c51bd346 P 2ff0106754a84b39989b9e042b4af3d6 [CANDIDATE]: Term 2 election: Requested vote from peers 9d26e8b39691431590fd77e182046c66 (127.19.226.194:33163), e6d654389db14a49a1f1bf3dbe538a27 (127.19.226.195:39911)
I20250109 08:59:10.312914 25370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f829cc37f3b447a800c8ee3c51bd346" candidate_uuid: "2ff0106754a84b39989b9e042b4af3d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e6d654389db14a49a1f1bf3dbe538a27"
I20250109 08:59:10.313694 25370 raft_consensus.cc:3049] T 9f829cc37f3b447a800c8ee3c51bd346 P e6d654389db14a49a1f1bf3dbe538a27 [term 1 LEADER]: Stepping down as leader of term 1
I20250109 08:59:10.313997 25370 raft_consensus.cc:738] T 9f829cc37f3b447a800c8ee3c51bd346 P e6d654389db14a49a1f1bf3dbe538a27 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e6d654389db14a49a1f1bf3dbe538a27, State: Running, Role: LEADER
I20250109 08:59:10.314783 25370 consensus_queue.cc:260] T 9f829cc37f3b447a800c8ee3c51bd346 P e6d654389db14a49a1f1bf3dbe538a27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:10.315788 25370 raft_consensus.cc:3054] T 9f829cc37f3b447a800c8ee3c51bd346 P e6d654389db14a49a1f1bf3dbe538a27 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:59:10.321774 25450 consensus_queue.cc:1035] T 075f0030bd964a1687ccee531080fd1c P e6d654389db14a49a1f1bf3dbe538a27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250109 08:59:10.323241 25370 raft_consensus.cc:2463] T 9f829cc37f3b447a800c8ee3c51bd346 P e6d654389db14a49a1f1bf3dbe538a27 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ff0106754a84b39989b9e042b4af3d6 in term 2.
I20250109 08:59:10.323547 25295 raft_consensus.cc:513] T 075f0030bd964a1687ccee531080fd1c P 9d26e8b39691431590fd77e182046c66 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:10.324414 25188 leader_election.cc:304] T 9f829cc37f3b447a800c8ee3c51bd346 P 2ff0106754a84b39989b9e042b4af3d6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2ff0106754a84b39989b9e042b4af3d6, e6d654389db14a49a1f1bf3dbe538a27; no voters: 9d26e8b39691431590fd77e182046c66
I20250109 08:59:10.324824 25294 raft_consensus.cc:1235] T 075f0030bd964a1687ccee531080fd1c P 9d26e8b39691431590fd77e182046c66 [term 2 FOLLOWER]: Rejecting Update request from peer e6d654389db14a49a1f1bf3dbe538a27 for earlier term 1. Current term is 2. Ops: []
I20250109 08:59:10.325510 25456 raft_consensus.cc:2798] T 9f829cc37f3b447a800c8ee3c51bd346 P 2ff0106754a84b39989b9e042b4af3d6 [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:59:10.325824 25295 leader_election.cc:290] T 075f0030bd964a1687ccee531080fd1c P 9d26e8b39691431590fd77e182046c66 [CANDIDATE]: Term 2 election: Requested vote from peers 2ff0106754a84b39989b9e042b4af3d6 (127.19.226.193:34345), e6d654389db14a49a1f1bf3dbe538a27 (127.19.226.195:39911)
I20250109 08:59:10.326015 25456 raft_consensus.cc:695] T 9f829cc37f3b447a800c8ee3c51bd346 P 2ff0106754a84b39989b9e042b4af3d6 [term 2 LEADER]: Becoming Leader. State: Replica: 2ff0106754a84b39989b9e042b4af3d6, State: Running, Role: LEADER
I20250109 08:59:10.326952 25456 consensus_queue.cc:237] T 9f829cc37f3b447a800c8ee3c51bd346 P 2ff0106754a84b39989b9e042b4af3d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:10.327946 25371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "075f0030bd964a1687ccee531080fd1c" candidate_uuid: "9d26e8b39691431590fd77e182046c66" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e6d654389db14a49a1f1bf3dbe538a27"
I20250109 08:59:10.328575 25371 raft_consensus.cc:3049] T 075f0030bd964a1687ccee531080fd1c P e6d654389db14a49a1f1bf3dbe538a27 [term 1 LEADER]: Stepping down as leader of term 1
I20250109 08:59:10.328913 25371 raft_consensus.cc:738] T 075f0030bd964a1687ccee531080fd1c P e6d654389db14a49a1f1bf3dbe538a27 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e6d654389db14a49a1f1bf3dbe538a27, State: Running, Role: LEADER
I20250109 08:59:10.329689 25371 consensus_queue.cc:260] T 075f0030bd964a1687ccee531080fd1c P e6d654389db14a49a1f1bf3dbe538a27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:10.330643 25371 raft_consensus.cc:3054] T 075f0030bd964a1687ccee531080fd1c P e6d654389db14a49a1f1bf3dbe538a27 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:59:10.332278 25488 consensus_queue.cc:1035] T 7622fd22d50942618088f5e65422e713 P e6d654389db14a49a1f1bf3dbe538a27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:59:10.328428 25221 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "075f0030bd964a1687ccee531080fd1c" candidate_uuid: "9d26e8b39691431590fd77e182046c66" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2ff0106754a84b39989b9e042b4af3d6"
I20250109 08:59:10.336341 25119 catalog_manager.cc:5526] T 9f829cc37f3b447a800c8ee3c51bd346 P 2ff0106754a84b39989b9e042b4af3d6 reported cstate change: term changed from 1 to 2, leader changed from e6d654389db14a49a1f1bf3dbe538a27 (127.19.226.195) to 2ff0106754a84b39989b9e042b4af3d6 (127.19.226.193). New cstate: current_term: 2 leader_uuid: "2ff0106754a84b39989b9e042b4af3d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } health_report { overall_health: UNKNOWN } } }
I20250109 08:59:10.345069 25488 consensus_queue.cc:1035] T 7622fd22d50942618088f5e65422e713 P e6d654389db14a49a1f1bf3dbe538a27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250109 08:59:10.348488 25452 consensus_queue.cc:1035] T c16639dc193644c8bbcf05ed9f02295e P 9d26e8b39691431590fd77e182046c66 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250109 08:59:10.350658 25371 raft_consensus.cc:2463] T 075f0030bd964a1687ccee531080fd1c P e6d654389db14a49a1f1bf3dbe538a27 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d26e8b39691431590fd77e182046c66 in term 2.
I20250109 08:59:10.354287 25263 leader_election.cc:304] T 075f0030bd964a1687ccee531080fd1c P 9d26e8b39691431590fd77e182046c66 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9d26e8b39691431590fd77e182046c66, e6d654389db14a49a1f1bf3dbe538a27; no voters: 2ff0106754a84b39989b9e042b4af3d6
I20250109 08:59:10.358022 25452 raft_consensus.cc:2798] T 075f0030bd964a1687ccee531080fd1c P 9d26e8b39691431590fd77e182046c66 [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:59:10.358747 25452 raft_consensus.cc:695] T 075f0030bd964a1687ccee531080fd1c P 9d26e8b39691431590fd77e182046c66 [term 2 LEADER]: Becoming Leader. State: Replica: 9d26e8b39691431590fd77e182046c66, State: Running, Role: LEADER
I20250109 08:59:10.359802 25488 raft_consensus.cc:988] T 7622fd22d50942618088f5e65422e713 P e6d654389db14a49a1f1bf3dbe538a27: : Instructing follower 9d26e8b39691431590fd77e182046c66 to start an election
I20250109 08:59:10.360518 25450 raft_consensus.cc:1076] T 7622fd22d50942618088f5e65422e713 P e6d654389db14a49a1f1bf3dbe538a27 [term 1 LEADER]: Signalling peer 9d26e8b39691431590fd77e182046c66 to start an election
I20250109 08:59:10.359683 25452 consensus_queue.cc:237] T 075f0030bd964a1687ccee531080fd1c P 9d26e8b39691431590fd77e182046c66 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:10.362517 25295 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "7622fd22d50942618088f5e65422e713"
dest_uuid: "9d26e8b39691431590fd77e182046c66"
from {username='slave'} at 127.0.0.1:45172
I20250109 08:59:10.363181 25295 raft_consensus.cc:491] T 7622fd22d50942618088f5e65422e713 P 9d26e8b39691431590fd77e182046c66 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:59:10.363559 25295 raft_consensus.cc:3054] T 7622fd22d50942618088f5e65422e713 P 9d26e8b39691431590fd77e182046c66 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:59:10.371479 25295 raft_consensus.cc:513] T 7622fd22d50942618088f5e65422e713 P 9d26e8b39691431590fd77e182046c66 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:10.377669 25294 raft_consensus.cc:1235] T 7622fd22d50942618088f5e65422e713 P 9d26e8b39691431590fd77e182046c66 [term 2 FOLLOWER]: Rejecting Update request from peer e6d654389db14a49a1f1bf3dbe538a27 for earlier term 1. Current term is 2. Ops: []
I20250109 08:59:10.378558 25415 consensus_queue.cc:1035] T c16639dc193644c8bbcf05ed9f02295e P 9d26e8b39691431590fd77e182046c66 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:59:10.380259 25295 leader_election.cc:290] T 7622fd22d50942618088f5e65422e713 P 9d26e8b39691431590fd77e182046c66 [CANDIDATE]: Term 2 election: Requested vote from peers 2ff0106754a84b39989b9e042b4af3d6 (127.19.226.193:34345), e6d654389db14a49a1f1bf3dbe538a27 (127.19.226.195:39911)
I20250109 08:59:10.381238 25370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7622fd22d50942618088f5e65422e713" candidate_uuid: "9d26e8b39691431590fd77e182046c66" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e6d654389db14a49a1f1bf3dbe538a27"
I20250109 08:59:10.381893 25370 raft_consensus.cc:3049] T 7622fd22d50942618088f5e65422e713 P e6d654389db14a49a1f1bf3dbe538a27 [term 1 LEADER]: Stepping down as leader of term 1
I20250109 08:59:10.382186 25370 raft_consensus.cc:738] T 7622fd22d50942618088f5e65422e713 P e6d654389db14a49a1f1bf3dbe538a27 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e6d654389db14a49a1f1bf3dbe538a27, State: Running, Role: LEADER
I20250109 08:59:10.382921 25370 consensus_queue.cc:260] T 7622fd22d50942618088f5e65422e713 P e6d654389db14a49a1f1bf3dbe538a27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:10.384166 25370 raft_consensus.cc:3054] T 7622fd22d50942618088f5e65422e713 P e6d654389db14a49a1f1bf3dbe538a27 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:59:10.386258 25222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7622fd22d50942618088f5e65422e713" candidate_uuid: "9d26e8b39691431590fd77e182046c66" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2ff0106754a84b39989b9e042b4af3d6"
I20250109 08:59:10.387843 25222 raft_consensus.cc:3054] T 7622fd22d50942618088f5e65422e713 P 2ff0106754a84b39989b9e042b4af3d6 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:59:10.387872 25477 raft_consensus.cc:988] T c16639dc193644c8bbcf05ed9f02295e P 9d26e8b39691431590fd77e182046c66: : Instructing follower 2ff0106754a84b39989b9e042b4af3d6 to start an election
I20250109 08:59:10.388514 25415 raft_consensus.cc:1076] T c16639dc193644c8bbcf05ed9f02295e P 9d26e8b39691431590fd77e182046c66 [term 1 LEADER]: Signalling peer 2ff0106754a84b39989b9e042b4af3d6 to start an election
I20250109 08:59:10.390396 25220 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "c16639dc193644c8bbcf05ed9f02295e"
dest_uuid: "2ff0106754a84b39989b9e042b4af3d6"
from {username='slave'} at 127.0.0.1:55364
I20250109 08:59:10.391027 25220 raft_consensus.cc:491] T c16639dc193644c8bbcf05ed9f02295e P 2ff0106754a84b39989b9e042b4af3d6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:59:10.391422 25220 raft_consensus.cc:3054] T c16639dc193644c8bbcf05ed9f02295e P 2ff0106754a84b39989b9e042b4af3d6 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:59:10.390182 25119 catalog_manager.cc:5526] T 075f0030bd964a1687ccee531080fd1c P 9d26e8b39691431590fd77e182046c66 reported cstate change: term changed from 1 to 2, leader changed from e6d654389db14a49a1f1bf3dbe538a27 (127.19.226.195) to 9d26e8b39691431590fd77e182046c66 (127.19.226.194). New cstate: current_term: 2 leader_uuid: "9d26e8b39691431590fd77e182046c66" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } health_report { overall_health: UNKNOWN } } }
I20250109 08:59:10.394040 25370 raft_consensus.cc:2463] T 7622fd22d50942618088f5e65422e713 P e6d654389db14a49a1f1bf3dbe538a27 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d26e8b39691431590fd77e182046c66 in term 2.
I20250109 08:59:10.395180 25263 leader_election.cc:304] T 7622fd22d50942618088f5e65422e713 P 9d26e8b39691431590fd77e182046c66 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d26e8b39691431590fd77e182046c66, e6d654389db14a49a1f1bf3dbe538a27; no voters:
I20250109 08:59:10.396031 25477 raft_consensus.cc:2798] T 7622fd22d50942618088f5e65422e713 P 9d26e8b39691431590fd77e182046c66 [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:59:10.396111 25222 raft_consensus.cc:2463] T 7622fd22d50942618088f5e65422e713 P 2ff0106754a84b39989b9e042b4af3d6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d26e8b39691431590fd77e182046c66 in term 2.
I20250109 08:59:10.396569 25477 raft_consensus.cc:695] T 7622fd22d50942618088f5e65422e713 P 9d26e8b39691431590fd77e182046c66 [term 2 LEADER]: Becoming Leader. State: Replica: 9d26e8b39691431590fd77e182046c66, State: Running, Role: LEADER
I20250109 08:59:10.397534 25477 consensus_queue.cc:237] T 7622fd22d50942618088f5e65422e713 P 9d26e8b39691431590fd77e182046c66 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:10.402266 25220 raft_consensus.cc:513] T c16639dc193644c8bbcf05ed9f02295e P 2ff0106754a84b39989b9e042b4af3d6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:10.402829 25456 consensus_queue.cc:1035] T 486c1bbdc3e247cdbbf13e1f7456e729 P 2ff0106754a84b39989b9e042b4af3d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:59:10.405285 25220 leader_election.cc:290] T c16639dc193644c8bbcf05ed9f02295e P 2ff0106754a84b39989b9e042b4af3d6 [CANDIDATE]: Term 2 election: Requested vote from peers 9d26e8b39691431590fd77e182046c66 (127.19.226.194:33163), e6d654389db14a49a1f1bf3dbe538a27 (127.19.226.195:39911)
I20250109 08:59:10.405891 25295 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c16639dc193644c8bbcf05ed9f02295e" candidate_uuid: "2ff0106754a84b39989b9e042b4af3d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9d26e8b39691431590fd77e182046c66"
I20250109 08:59:10.406540 25295 raft_consensus.cc:3049] T c16639dc193644c8bbcf05ed9f02295e P 9d26e8b39691431590fd77e182046c66 [term 1 LEADER]: Stepping down as leader of term 1
I20250109 08:59:10.406831 25295 raft_consensus.cc:738] T c16639dc193644c8bbcf05ed9f02295e P 9d26e8b39691431590fd77e182046c66 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9d26e8b39691431590fd77e182046c66, State: Running, Role: LEADER
I20250109 08:59:10.406951 25369 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c16639dc193644c8bbcf05ed9f02295e" candidate_uuid: "2ff0106754a84b39989b9e042b4af3d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e6d654389db14a49a1f1bf3dbe538a27"
I20250109 08:59:10.407554 25369 raft_consensus.cc:3054] T c16639dc193644c8bbcf05ed9f02295e P e6d654389db14a49a1f1bf3dbe538a27 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:59:10.407593 25295 consensus_queue.cc:260] T c16639dc193644c8bbcf05ed9f02295e P 9d26e8b39691431590fd77e182046c66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:10.410398 25295 raft_consensus.cc:3054] T c16639dc193644c8bbcf05ed9f02295e P 9d26e8b39691431590fd77e182046c66 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:59:10.411553 25118 catalog_manager.cc:5526] T 7622fd22d50942618088f5e65422e713 P 9d26e8b39691431590fd77e182046c66 reported cstate change: term changed from 1 to 2, leader changed from e6d654389db14a49a1f1bf3dbe538a27 (127.19.226.195) to 9d26e8b39691431590fd77e182046c66 (127.19.226.194). New cstate: current_term: 2 leader_uuid: "9d26e8b39691431590fd77e182046c66" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } health_report { overall_health: UNKNOWN } } }
I20250109 08:59:10.417656 25295 raft_consensus.cc:2463] T c16639dc193644c8bbcf05ed9f02295e P 9d26e8b39691431590fd77e182046c66 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ff0106754a84b39989b9e042b4af3d6 in term 2.
I20250109 08:59:10.419726 25369 raft_consensus.cc:2463] T c16639dc193644c8bbcf05ed9f02295e P e6d654389db14a49a1f1bf3dbe538a27 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ff0106754a84b39989b9e042b4af3d6 in term 2.
I20250109 08:59:10.419876 25187 leader_election.cc:304] T c16639dc193644c8bbcf05ed9f02295e P 2ff0106754a84b39989b9e042b4af3d6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ff0106754a84b39989b9e042b4af3d6, 9d26e8b39691431590fd77e182046c66; no voters:
I20250109 08:59:10.421432 25424 raft_consensus.cc:2798] T c16639dc193644c8bbcf05ed9f02295e P 2ff0106754a84b39989b9e042b4af3d6 [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:59:10.421885 25424 raft_consensus.cc:695] T c16639dc193644c8bbcf05ed9f02295e P 2ff0106754a84b39989b9e042b4af3d6 [term 2 LEADER]: Becoming Leader. State: Replica: 2ff0106754a84b39989b9e042b4af3d6, State: Running, Role: LEADER
I20250109 08:59:10.422737 25424 consensus_queue.cc:237] T c16639dc193644c8bbcf05ed9f02295e P 2ff0106754a84b39989b9e042b4af3d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:10.426008 25455 consensus_queue.cc:1035] T 486c1bbdc3e247cdbbf13e1f7456e729 P 2ff0106754a84b39989b9e042b4af3d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250109 08:59:10.434842 25455 raft_consensus.cc:988] T 486c1bbdc3e247cdbbf13e1f7456e729 P 2ff0106754a84b39989b9e042b4af3d6: : Instructing follower 9d26e8b39691431590fd77e182046c66 to start an election
I20250109 08:59:10.435398 25445 raft_consensus.cc:1076] T 486c1bbdc3e247cdbbf13e1f7456e729 P 2ff0106754a84b39989b9e042b4af3d6 [term 1 LEADER]: Signalling peer 9d26e8b39691431590fd77e182046c66 to start an election
I20250109 08:59:10.434952 25118 catalog_manager.cc:5526] T c16639dc193644c8bbcf05ed9f02295e P 2ff0106754a84b39989b9e042b4af3d6 reported cstate change: term changed from 1 to 2, leader changed from 9d26e8b39691431590fd77e182046c66 (127.19.226.194) to 2ff0106754a84b39989b9e042b4af3d6 (127.19.226.193). New cstate: current_term: 2 leader_uuid: "2ff0106754a84b39989b9e042b4af3d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } health_report { overall_health: UNKNOWN } } }
I20250109 08:59:10.436928 25295 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "486c1bbdc3e247cdbbf13e1f7456e729"
dest_uuid: "9d26e8b39691431590fd77e182046c66"
from {username='slave'} at 127.0.0.1:45178
I20250109 08:59:10.437371 25295 raft_consensus.cc:491] T 486c1bbdc3e247cdbbf13e1f7456e729 P 9d26e8b39691431590fd77e182046c66 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:59:10.437697 25295 raft_consensus.cc:3054] T 486c1bbdc3e247cdbbf13e1f7456e729 P 9d26e8b39691431590fd77e182046c66 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:59:10.443724 25295 raft_consensus.cc:513] T 486c1bbdc3e247cdbbf13e1f7456e729 P 9d26e8b39691431590fd77e182046c66 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:10.446323 25295 leader_election.cc:290] T 486c1bbdc3e247cdbbf13e1f7456e729 P 9d26e8b39691431590fd77e182046c66 [CANDIDATE]: Term 2 election: Requested vote from peers 2ff0106754a84b39989b9e042b4af3d6 (127.19.226.193:34345), e6d654389db14a49a1f1bf3dbe538a27 (127.19.226.195:39911)
I20250109 08:59:10.446439 25220 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "486c1bbdc3e247cdbbf13e1f7456e729" candidate_uuid: "9d26e8b39691431590fd77e182046c66" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2ff0106754a84b39989b9e042b4af3d6"
I20250109 08:59:10.446712 25369 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "486c1bbdc3e247cdbbf13e1f7456e729" candidate_uuid: "9d26e8b39691431590fd77e182046c66" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e6d654389db14a49a1f1bf3dbe538a27"
I20250109 08:59:10.447127 25220 raft_consensus.cc:3049] T 486c1bbdc3e247cdbbf13e1f7456e729 P 2ff0106754a84b39989b9e042b4af3d6 [term 1 LEADER]: Stepping down as leader of term 1
I20250109 08:59:10.447481 25220 raft_consensus.cc:738] T 486c1bbdc3e247cdbbf13e1f7456e729 P 2ff0106754a84b39989b9e042b4af3d6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2ff0106754a84b39989b9e042b4af3d6, State: Running, Role: LEADER
I20250109 08:59:10.447234 25369 raft_consensus.cc:3054] T 486c1bbdc3e247cdbbf13e1f7456e729 P e6d654389db14a49a1f1bf3dbe538a27 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:59:10.448264 25220 consensus_queue.cc:260] T 486c1bbdc3e247cdbbf13e1f7456e729 P 2ff0106754a84b39989b9e042b4af3d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:10.449399 25220 raft_consensus.cc:3054] T 486c1bbdc3e247cdbbf13e1f7456e729 P 2ff0106754a84b39989b9e042b4af3d6 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:59:10.453886 25369 raft_consensus.cc:2463] T 486c1bbdc3e247cdbbf13e1f7456e729 P e6d654389db14a49a1f1bf3dbe538a27 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d26e8b39691431590fd77e182046c66 in term 2.
I20250109 08:59:10.453886 25220 raft_consensus.cc:2463] T 486c1bbdc3e247cdbbf13e1f7456e729 P 2ff0106754a84b39989b9e042b4af3d6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d26e8b39691431590fd77e182046c66 in term 2.
I20250109 08:59:10.454988 25263 leader_election.cc:304] T 486c1bbdc3e247cdbbf13e1f7456e729 P 9d26e8b39691431590fd77e182046c66 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d26e8b39691431590fd77e182046c66, e6d654389db14a49a1f1bf3dbe538a27; no voters:
I20250109 08:59:10.455637 25477 raft_consensus.cc:2798] T 486c1bbdc3e247cdbbf13e1f7456e729 P 9d26e8b39691431590fd77e182046c66 [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:59:10.456068 25477 raft_consensus.cc:695] T 486c1bbdc3e247cdbbf13e1f7456e729 P 9d26e8b39691431590fd77e182046c66 [term 2 LEADER]: Becoming Leader. State: Replica: 9d26e8b39691431590fd77e182046c66, State: Running, Role: LEADER
I20250109 08:59:10.456845 25477 consensus_queue.cc:237] T 486c1bbdc3e247cdbbf13e1f7456e729 P 9d26e8b39691431590fd77e182046c66 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:10.462584 25119 catalog_manager.cc:5526] T 486c1bbdc3e247cdbbf13e1f7456e729 P 9d26e8b39691431590fd77e182046c66 reported cstate change: term changed from 1 to 2, leader changed from 2ff0106754a84b39989b9e042b4af3d6 (127.19.226.193) to 9d26e8b39691431590fd77e182046c66 (127.19.226.194). New cstate: current_term: 2 leader_uuid: "9d26e8b39691431590fd77e182046c66" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } health_report { overall_health: UNKNOWN } } }
I20250109 08:59:10.577666 25295 raft_consensus.cc:1270] T 1d01a6ffe04e4c7bae266bbc80ee5d54 P 9d26e8b39691431590fd77e182046c66 [term 2 FOLLOWER]: Refusing update from remote peer 2ff0106754a84b39989b9e042b4af3d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:59:10.578815 25445 consensus_queue.cc:1035] T 1d01a6ffe04e4c7bae266bbc80ee5d54 P 2ff0106754a84b39989b9e042b4af3d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:59:10.585647 25369 raft_consensus.cc:1270] T 1d01a6ffe04e4c7bae266bbc80ee5d54 P e6d654389db14a49a1f1bf3dbe538a27 [term 2 FOLLOWER]: Refusing update from remote peer 2ff0106754a84b39989b9e042b4af3d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:59:10.587260 25424 consensus_queue.cc:1035] T 1d01a6ffe04e4c7bae266bbc80ee5d54 P 2ff0106754a84b39989b9e042b4af3d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:59:10.699954 25220 raft_consensus.cc:1270] T 422f918bde184ce1ae2532c9dee609a7 P 2ff0106754a84b39989b9e042b4af3d6 [term 2 FOLLOWER]: Refusing update from remote peer e6d654389db14a49a1f1bf3dbe538a27: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:59:10.701056 25488 consensus_queue.cc:1035] T 422f918bde184ce1ae2532c9dee609a7 P e6d654389db14a49a1f1bf3dbe538a27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:59:10.703243 25295 raft_consensus.cc:3054] T 422f918bde184ce1ae2532c9dee609a7 P 9d26e8b39691431590fd77e182046c66 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:59:10.711601 25295 raft_consensus.cc:1270] T 422f918bde184ce1ae2532c9dee609a7 P 9d26e8b39691431590fd77e182046c66 [term 2 FOLLOWER]: Refusing update from remote peer e6d654389db14a49a1f1bf3dbe538a27: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:59:10.712846 25450 consensus_queue.cc:1035] T 422f918bde184ce1ae2532c9dee609a7 P e6d654389db14a49a1f1bf3dbe538a27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:59:10.762651 25295 raft_consensus.cc:3054] T 9f829cc37f3b447a800c8ee3c51bd346 P 9d26e8b39691431590fd77e182046c66 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:59:10.766907 25220 raft_consensus.cc:1270] T 709470e8d22046dd87f2a20e8a8ce745 P 2ff0106754a84b39989b9e042b4af3d6 [term 2 FOLLOWER]: Refusing update from remote peer 9d26e8b39691431590fd77e182046c66: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:59:10.767692 25222 raft_consensus.cc:3054] T 075f0030bd964a1687ccee531080fd1c P 2ff0106754a84b39989b9e042b4af3d6 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:59:10.768455 25295 raft_consensus.cc:1270] T 9f829cc37f3b447a800c8ee3c51bd346 P 9d26e8b39691431590fd77e182046c66 [term 2 FOLLOWER]: Refusing update from remote peer 2ff0106754a84b39989b9e042b4af3d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:59:10.768337 25477 consensus_queue.cc:1035] T 709470e8d22046dd87f2a20e8a8ce745 P 9d26e8b39691431590fd77e182046c66 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20250109 08:59:10.769708 25424 consensus_queue.cc:1035] T 9f829cc37f3b447a800c8ee3c51bd346 P 2ff0106754a84b39989b9e042b4af3d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:59:10.777474 25370 raft_consensus.cc:1270] T 709470e8d22046dd87f2a20e8a8ce745 P e6d654389db14a49a1f1bf3dbe538a27 [term 2 FOLLOWER]: Refusing update from remote peer 9d26e8b39691431590fd77e182046c66: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:59:10.779651 25452 consensus_queue.cc:1035] T 709470e8d22046dd87f2a20e8a8ce745 P 9d26e8b39691431590fd77e182046c66 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:59:10.780162 25370 raft_consensus.cc:1270] T 7622fd22d50942618088f5e65422e713 P e6d654389db14a49a1f1bf3dbe538a27 [term 2 FOLLOWER]: Refusing update from remote peer 9d26e8b39691431590fd77e182046c66: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:59:10.781177 25369 raft_consensus.cc:1270] T 9f829cc37f3b447a800c8ee3c51bd346 P e6d654389db14a49a1f1bf3dbe538a27 [term 2 FOLLOWER]: Refusing update from remote peer 2ff0106754a84b39989b9e042b4af3d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:59:10.781567 25452 consensus_queue.cc:1035] T 7622fd22d50942618088f5e65422e713 P 9d26e8b39691431590fd77e182046c66 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:59:10.785172 25424 consensus_queue.cc:1035] T 9f829cc37f3b447a800c8ee3c51bd346 P 2ff0106754a84b39989b9e042b4af3d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:59:10.785480 25222 raft_consensus.cc:1270] T 075f0030bd964a1687ccee531080fd1c P 2ff0106754a84b39989b9e042b4af3d6 [term 2 FOLLOWER]: Refusing update from remote peer 9d26e8b39691431590fd77e182046c66: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:59:10.788141 25452 consensus_queue.cc:1035] T 075f0030bd964a1687ccee531080fd1c P 9d26e8b39691431590fd77e182046c66 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:59:10.797350 25220 raft_consensus.cc:1270] T 7622fd22d50942618088f5e65422e713 P 2ff0106754a84b39989b9e042b4af3d6 [term 2 FOLLOWER]: Refusing update from remote peer 9d26e8b39691431590fd77e182046c66: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:59:10.798792 25369 raft_consensus.cc:1270] T 075f0030bd964a1687ccee531080fd1c P e6d654389db14a49a1f1bf3dbe538a27 [term 2 FOLLOWER]: Refusing update from remote peer 9d26e8b39691431590fd77e182046c66: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:59:10.800143 25452 consensus_queue.cc:1035] T 7622fd22d50942618088f5e65422e713 P 9d26e8b39691431590fd77e182046c66 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:59:10.800843 25415 consensus_queue.cc:1035] T 075f0030bd964a1687ccee531080fd1c P 9d26e8b39691431590fd77e182046c66 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:59:10.896771 25369 raft_consensus.cc:1270] T c16639dc193644c8bbcf05ed9f02295e P e6d654389db14a49a1f1bf3dbe538a27 [term 2 FOLLOWER]: Refusing update from remote peer 2ff0106754a84b39989b9e042b4af3d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:59:10.898098 25424 consensus_queue.cc:1035] T c16639dc193644c8bbcf05ed9f02295e P 2ff0106754a84b39989b9e042b4af3d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:59:10.898988 25222 raft_consensus.cc:1270] T 486c1bbdc3e247cdbbf13e1f7456e729 P 2ff0106754a84b39989b9e042b4af3d6 [term 2 FOLLOWER]: Refusing update from remote peer 9d26e8b39691431590fd77e182046c66: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:59:10.900651 25415 consensus_queue.cc:1035] T 486c1bbdc3e247cdbbf13e1f7456e729 P 9d26e8b39691431590fd77e182046c66 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:59:10.907269 25295 raft_consensus.cc:1270] T c16639dc193644c8bbcf05ed9f02295e P 9d26e8b39691431590fd77e182046c66 [term 2 FOLLOWER]: Refusing update from remote peer 2ff0106754a84b39989b9e042b4af3d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:59:10.908753 25424 consensus_queue.cc:1035] T c16639dc193644c8bbcf05ed9f02295e P 2ff0106754a84b39989b9e042b4af3d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:59:10.917037 25369 raft_consensus.cc:1270] T 486c1bbdc3e247cdbbf13e1f7456e729 P e6d654389db14a49a1f1bf3dbe538a27 [term 2 FOLLOWER]: Refusing update from remote peer 9d26e8b39691431590fd77e182046c66: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:59:10.918371 25452 consensus_queue.cc:1035] T 486c1bbdc3e247cdbbf13e1f7456e729 P 9d26e8b39691431590fd77e182046c66 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:59:13.153420 20363 test_util.cc:274] Using random seed: 825053479
I20250109 08:59:13.183598 25117 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48134:
name: "soft_deleted_table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250109 08:59:13.186322 25117 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250109 08:59:13.237485 25211 tablet_service.cc:1467] Processing CreateTablet for tablet 3b37da4b59cf41b8889527eb2b0793c9 (DEFAULT_TABLE table=soft_deleted_table [id=66f95f53887e43ad8d999c406068e0a1]), partition=RANGE (key) PARTITION VALUES < 238609294
I20250109 08:59:13.238097 25286 tablet_service.cc:1467] Processing CreateTablet for tablet 3b37da4b59cf41b8889527eb2b0793c9 (DEFAULT_TABLE table=soft_deleted_table [id=66f95f53887e43ad8d999c406068e0a1]), partition=RANGE (key) PARTITION VALUES < 238609294
I20250109 08:59:13.239068 25211 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b37da4b59cf41b8889527eb2b0793c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:59:13.238884 25358 tablet_service.cc:1467] Processing CreateTablet for tablet 3b37da4b59cf41b8889527eb2b0793c9 (DEFAULT_TABLE table=soft_deleted_table [id=66f95f53887e43ad8d999c406068e0a1]), partition=RANGE (key) PARTITION VALUES < 238609294
I20250109 08:59:13.239619 25286 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b37da4b59cf41b8889527eb2b0793c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:59:13.240402 25358 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b37da4b59cf41b8889527eb2b0793c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:59:13.245357 25285 tablet_service.cc:1467] Processing CreateTablet for tablet 8022fc0563b947669983f72b9b7450b3 (DEFAULT_TABLE table=soft_deleted_table [id=66f95f53887e43ad8d999c406068e0a1]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20250109 08:59:13.246762 25285 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8022fc0563b947669983f72b9b7450b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:59:13.247015 25279 tablet_service.cc:1467] Processing CreateTablet for tablet 346220ee65da44b8862c9f28d7aa6aa3 (DEFAULT_TABLE table=soft_deleted_table [id=66f95f53887e43ad8d999c406068e0a1]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20250109 08:59:13.248387 25279 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 346220ee65da44b8862c9f28d7aa6aa3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:59:13.248559 25210 tablet_service.cc:1467] Processing CreateTablet for tablet 8022fc0563b947669983f72b9b7450b3 (DEFAULT_TABLE table=soft_deleted_table [id=66f95f53887e43ad8d999c406068e0a1]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20250109 08:59:13.249867 25210 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8022fc0563b947669983f72b9b7450b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:59:13.249511 25281 tablet_service.cc:1467] Processing CreateTablet for tablet 9bf180fefb074fd192f6f80ae1460ba5 (DEFAULT_TABLE table=soft_deleted_table [id=66f95f53887e43ad8d999c406068e0a1]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20250109 08:59:13.250778 25281 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9bf180fefb074fd192f6f80ae1460ba5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:59:13.253334 25208 tablet_service.cc:1467] Processing CreateTablet for tablet 9bf180fefb074fd192f6f80ae1460ba5 (DEFAULT_TABLE table=soft_deleted_table [id=66f95f53887e43ad8d999c406068e0a1]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20250109 08:59:13.254680 25208 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9bf180fefb074fd192f6f80ae1460ba5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:59:13.254436 25280 tablet_service.cc:1467] Processing CreateTablet for tablet 9a3240fe2e094ba2bc002579bdb29e84 (DEFAULT_TABLE table=soft_deleted_table [id=66f95f53887e43ad8d999c406068e0a1]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20250109 08:59:13.255656 25280 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a3240fe2e094ba2bc002579bdb29e84. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:59:13.257546 25207 tablet_service.cc:1467] Processing CreateTablet for tablet cb8ed58651e74cbabff2b027ba4a9a1e (DEFAULT_TABLE table=soft_deleted_table [id=66f95f53887e43ad8d999c406068e0a1]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20250109 08:59:13.258796 25283 tablet_service.cc:1467] Processing CreateTablet for tablet 51277cd5f9374666bc60232309efc100 (DEFAULT_TABLE table=soft_deleted_table [id=66f95f53887e43ad8d999c406068e0a1]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20250109 08:59:13.261507 25361 tablet_service.cc:1467] Processing CreateTablet for tablet 8022fc0563b947669983f72b9b7450b3 (DEFAULT_TABLE table=soft_deleted_table [id=66f95f53887e43ad8d999c406068e0a1]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20250109 08:59:13.262755 25361 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8022fc0563b947669983f72b9b7450b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:59:13.253337 25284 tablet_service.cc:1467] Processing CreateTablet for tablet cb8ed58651e74cbabff2b027ba4a9a1e (DEFAULT_TABLE table=soft_deleted_table [id=66f95f53887e43ad8d999c406068e0a1]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20250109 08:59:13.265513 25357 tablet_service.cc:1467] Processing CreateTablet for tablet 9bf180fefb074fd192f6f80ae1460ba5 (DEFAULT_TABLE table=soft_deleted_table [id=66f95f53887e43ad8d999c406068e0a1]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20250109 08:59:13.266773 25357 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9bf180fefb074fd192f6f80ae1460ba5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:59:13.257557 25282 tablet_service.cc:1467] Processing CreateTablet for tablet 3f4e7967061b4173bd00a9539a371fe3 (DEFAULT_TABLE table=soft_deleted_table [id=66f95f53887e43ad8d999c406068e0a1]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20250109 08:59:13.259939 25204 tablet_service.cc:1467] Processing CreateTablet for tablet 346220ee65da44b8862c9f28d7aa6aa3 (DEFAULT_TABLE table=soft_deleted_table [id=66f95f53887e43ad8d999c406068e0a1]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20250109 08:59:13.269593 25360 tablet_service.cc:1467] Processing CreateTablet for tablet cb8ed58651e74cbabff2b027ba4a9a1e (DEFAULT_TABLE table=soft_deleted_table [id=66f95f53887e43ad8d999c406068e0a1]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20250109 08:59:13.258831 25207 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb8ed58651e74cbabff2b027ba4a9a1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:59:13.270895 25360 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb8ed58651e74cbabff2b027ba4a9a1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:59:13.272768 25204 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 346220ee65da44b8862c9f28d7aa6aa3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:59:13.273537 25354 tablet_service.cc:1467] Processing CreateTablet for tablet 346220ee65da44b8862c9f28d7aa6aa3 (DEFAULT_TABLE table=soft_deleted_table [id=66f95f53887e43ad8d999c406068e0a1]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20250109 08:59:13.274789 25354 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 346220ee65da44b8862c9f28d7aa6aa3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:59:13.277539 25353 tablet_service.cc:1467] Processing CreateTablet for tablet 9a3240fe2e094ba2bc002579bdb29e84 (DEFAULT_TABLE table=soft_deleted_table [id=66f95f53887e43ad8d999c406068e0a1]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20250109 08:59:13.278865 25353 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a3240fe2e094ba2bc002579bdb29e84. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:59:13.281539 25359 tablet_service.cc:1467] Processing CreateTablet for tablet 3f4e7967061b4173bd00a9539a371fe3 (DEFAULT_TABLE table=soft_deleted_table [id=66f95f53887e43ad8d999c406068e0a1]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20250109 08:59:13.282811 25359 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f4e7967061b4173bd00a9539a371fe3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:59:13.285115 25206 tablet_service.cc:1467] Processing CreateTablet for tablet ed0e3d98ad79424caf0e24ecf3b5f295 (DEFAULT_TABLE table=soft_deleted_table [id=66f95f53887e43ad8d999c406068e0a1]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20250109 08:59:13.285531 25355 tablet_service.cc:1467] Processing CreateTablet for tablet ed0e3d98ad79424caf0e24ecf3b5f295 (DEFAULT_TABLE table=soft_deleted_table [id=66f95f53887e43ad8d999c406068e0a1]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20250109 08:59:13.286590 25206 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed0e3d98ad79424caf0e24ecf3b5f295. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:59:13.286870 25355 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed0e3d98ad79424caf0e24ecf3b5f295. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:59:13.289558 25212 tablet_service.cc:1467] Processing CreateTablet for tablet 9a3240fe2e094ba2bc002579bdb29e84 (DEFAULT_TABLE table=soft_deleted_table [id=66f95f53887e43ad8d999c406068e0a1]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20250109 08:59:13.290848 25212 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a3240fe2e094ba2bc002579bdb29e84. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:59:13.293522 25209 tablet_service.cc:1467] Processing CreateTablet for tablet 3f4e7967061b4173bd00a9539a371fe3 (DEFAULT_TABLE table=soft_deleted_table [id=66f95f53887e43ad8d999c406068e0a1]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20250109 08:59:13.294858 25209 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f4e7967061b4173bd00a9539a371fe3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:59:13.297539 25205 tablet_service.cc:1467] Processing CreateTablet for tablet 51277cd5f9374666bc60232309efc100 (DEFAULT_TABLE table=soft_deleted_table [id=66f95f53887e43ad8d999c406068e0a1]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20250109 08:59:13.298916 25205 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51277cd5f9374666bc60232309efc100. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:59:13.260061 25283 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51277cd5f9374666bc60232309efc100. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:59:13.308658 25282 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f4e7967061b4173bd00a9539a371fe3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:59:13.312803 25284 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb8ed58651e74cbabff2b027ba4a9a1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:59:13.313370 25287 tablet_service.cc:1467] Processing CreateTablet for tablet ed0e3d98ad79424caf0e24ecf3b5f295 (DEFAULT_TABLE table=soft_deleted_table [id=66f95f53887e43ad8d999c406068e0a1]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20250109 08:59:13.314690 25287 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed0e3d98ad79424caf0e24ecf3b5f295. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:59:13.316776 25356 tablet_service.cc:1467] Processing CreateTablet for tablet 51277cd5f9374666bc60232309efc100 (DEFAULT_TABLE table=soft_deleted_table [id=66f95f53887e43ad8d999c406068e0a1]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20250109 08:59:13.397845 25356 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51277cd5f9374666bc60232309efc100. 1 dirs total, 0 dirs full, 0 dirs failed
I20250109 08:59:13.421584 25509 tablet_bootstrap.cc:492] T 346220ee65da44b8862c9f28d7aa6aa3 P 9d26e8b39691431590fd77e182046c66: Bootstrap starting.
I20250109 08:59:13.456341 25509 tablet_bootstrap.cc:654] T 346220ee65da44b8862c9f28d7aa6aa3 P 9d26e8b39691431590fd77e182046c66: Neither blocks nor log segments found. Creating new log.
I20250109 08:59:13.462002 25510 tablet_bootstrap.cc:492] T 3b37da4b59cf41b8889527eb2b0793c9 P e6d654389db14a49a1f1bf3dbe538a27: Bootstrap starting.
I20250109 08:59:13.465406 25511 tablet_bootstrap.cc:492] T ed0e3d98ad79424caf0e24ecf3b5f295 P 2ff0106754a84b39989b9e042b4af3d6: Bootstrap starting.
I20250109 08:59:13.474669 25511 tablet_bootstrap.cc:654] T ed0e3d98ad79424caf0e24ecf3b5f295 P 2ff0106754a84b39989b9e042b4af3d6: Neither blocks nor log segments found. Creating new log.
I20250109 08:59:13.480463 25510 tablet_bootstrap.cc:654] T 3b37da4b59cf41b8889527eb2b0793c9 P e6d654389db14a49a1f1bf3dbe538a27: Neither blocks nor log segments found. Creating new log.
I20250109 08:59:13.482808 25509 tablet_bootstrap.cc:492] T 346220ee65da44b8862c9f28d7aa6aa3 P 9d26e8b39691431590fd77e182046c66: No bootstrap required, opened a new log
I20250109 08:59:13.483276 25509 ts_tablet_manager.cc:1397] T 346220ee65da44b8862c9f28d7aa6aa3 P 9d26e8b39691431590fd77e182046c66: Time spent bootstrapping tablet: real 0.062s user 0.010s sys 0.015s
I20250109 08:59:13.484093 25511 tablet_bootstrap.cc:492] T ed0e3d98ad79424caf0e24ecf3b5f295 P 2ff0106754a84b39989b9e042b4af3d6: No bootstrap required, opened a new log
I20250109 08:59:13.484592 25511 ts_tablet_manager.cc:1397] T ed0e3d98ad79424caf0e24ecf3b5f295 P 2ff0106754a84b39989b9e042b4af3d6: Time spent bootstrapping tablet: real 0.019s user 0.007s sys 0.006s
I20250109 08:59:13.486160 25510 tablet_bootstrap.cc:492] T 3b37da4b59cf41b8889527eb2b0793c9 P e6d654389db14a49a1f1bf3dbe538a27: No bootstrap required, opened a new log
I20250109 08:59:13.485898 25509 raft_consensus.cc:357] T 346220ee65da44b8862c9f28d7aa6aa3 P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:13.486552 25509 raft_consensus.cc:383] T 346220ee65da44b8862c9f28d7aa6aa3 P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:59:13.486667 25510 ts_tablet_manager.cc:1397] T 3b37da4b59cf41b8889527eb2b0793c9 P e6d654389db14a49a1f1bf3dbe538a27: Time spent bootstrapping tablet: real 0.025s user 0.003s sys 0.007s
I20250109 08:59:13.486802 25509 raft_consensus.cc:738] T 346220ee65da44b8862c9f28d7aa6aa3 P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d26e8b39691431590fd77e182046c66, State: Initialized, Role: FOLLOWER
I20250109 08:59:13.487530 25509 consensus_queue.cc:260] T 346220ee65da44b8862c9f28d7aa6aa3 P 9d26e8b39691431590fd77e182046c66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:13.487533 25511 raft_consensus.cc:357] T ed0e3d98ad79424caf0e24ecf3b5f295 P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:13.488493 25511 raft_consensus.cc:383] T ed0e3d98ad79424caf0e24ecf3b5f295 P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:59:13.488777 25511 raft_consensus.cc:738] T ed0e3d98ad79424caf0e24ecf3b5f295 P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ff0106754a84b39989b9e042b4af3d6, State: Initialized, Role: FOLLOWER
W20250109 08:59:13.489933 25168 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20250109 08:59:13.489434 25511 consensus_queue.cc:260] T ed0e3d98ad79424caf0e24ecf3b5f295 P 2ff0106754a84b39989b9e042b4af3d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:13.490096 25509 ts_tablet_manager.cc:1428] T 346220ee65da44b8862c9f28d7aa6aa3 P 9d26e8b39691431590fd77e182046c66: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20250109 08:59:13.491266 25509 tablet_bootstrap.cc:492] T 51277cd5f9374666bc60232309efc100 P 9d26e8b39691431590fd77e182046c66: Bootstrap starting.
I20250109 08:59:13.492259 25511 ts_tablet_manager.cc:1428] T ed0e3d98ad79424caf0e24ecf3b5f295 P 2ff0106754a84b39989b9e042b4af3d6: Time spent starting tablet: real 0.007s user 0.005s sys 0.001s
I20250109 08:59:13.493615 25511 tablet_bootstrap.cc:492] T 9bf180fefb074fd192f6f80ae1460ba5 P 2ff0106754a84b39989b9e042b4af3d6: Bootstrap starting.
I20250109 08:59:13.493957 25510 raft_consensus.cc:357] T 3b37da4b59cf41b8889527eb2b0793c9 P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:13.497601 25509 tablet_bootstrap.cc:654] T 51277cd5f9374666bc60232309efc100 P 9d26e8b39691431590fd77e182046c66: Neither blocks nor log segments found. Creating new log.
I20250109 08:59:13.500134 25511 tablet_bootstrap.cc:654] T 9bf180fefb074fd192f6f80ae1460ba5 P 2ff0106754a84b39989b9e042b4af3d6: Neither blocks nor log segments found. Creating new log.
I20250109 08:59:13.504637 25509 tablet_bootstrap.cc:492] T 51277cd5f9374666bc60232309efc100 P 9d26e8b39691431590fd77e182046c66: No bootstrap required, opened a new log
I20250109 08:59:13.505075 25509 ts_tablet_manager.cc:1397] T 51277cd5f9374666bc60232309efc100 P 9d26e8b39691431590fd77e182046c66: Time spent bootstrapping tablet: real 0.014s user 0.013s sys 0.000s
I20250109 08:59:13.507169 25510 raft_consensus.cc:383] T 3b37da4b59cf41b8889527eb2b0793c9 P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:59:13.507642 25510 raft_consensus.cc:738] T 3b37da4b59cf41b8889527eb2b0793c9 P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6d654389db14a49a1f1bf3dbe538a27, State: Initialized, Role: FOLLOWER
I20250109 08:59:13.507827 25509 raft_consensus.cc:357] T 51277cd5f9374666bc60232309efc100 P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:13.508688 25509 raft_consensus.cc:383] T 51277cd5f9374666bc60232309efc100 P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:59:13.508978 25509 raft_consensus.cc:738] T 51277cd5f9374666bc60232309efc100 P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d26e8b39691431590fd77e182046c66, State: Initialized, Role: FOLLOWER
I20250109 08:59:13.508431 25510 consensus_queue.cc:260] T 3b37da4b59cf41b8889527eb2b0793c9 P e6d654389db14a49a1f1bf3dbe538a27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:13.509645 25509 consensus_queue.cc:260] T 51277cd5f9374666bc60232309efc100 P 9d26e8b39691431590fd77e182046c66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:13.511184 25510 ts_tablet_manager.cc:1428] T 3b37da4b59cf41b8889527eb2b0793c9 P e6d654389db14a49a1f1bf3dbe538a27: Time spent starting tablet: real 0.024s user 0.006s sys 0.003s
I20250109 08:59:13.511675 25452 raft_consensus.cc:491] T 51277cd5f9374666bc60232309efc100 P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:59:13.511998 25509 ts_tablet_manager.cc:1428] T 51277cd5f9374666bc60232309efc100 P 9d26e8b39691431590fd77e182046c66: Time spent starting tablet: real 0.007s user 0.004s sys 0.000s
I20250109 08:59:13.512176 25452 raft_consensus.cc:513] T 51277cd5f9374666bc60232309efc100 P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:13.512943 25509 tablet_bootstrap.cc:492] T ed0e3d98ad79424caf0e24ecf3b5f295 P 9d26e8b39691431590fd77e182046c66: Bootstrap starting.
I20250109 08:59:13.515578 25222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51277cd5f9374666bc60232309efc100" candidate_uuid: "9d26e8b39691431590fd77e182046c66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ff0106754a84b39989b9e042b4af3d6" is_pre_election: true
W20250109 08:59:13.517321 25261 leader_election.cc:343] T 51277cd5f9374666bc60232309efc100 P 9d26e8b39691431590fd77e182046c66 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2ff0106754a84b39989b9e042b4af3d6 (127.19.226.193:34345): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:59:13.519016 25371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51277cd5f9374666bc60232309efc100" candidate_uuid: "9d26e8b39691431590fd77e182046c66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6d654389db14a49a1f1bf3dbe538a27" is_pre_election: true
I20250109 08:59:13.520156 25509 tablet_bootstrap.cc:654] T ed0e3d98ad79424caf0e24ecf3b5f295 P 9d26e8b39691431590fd77e182046c66: Neither blocks nor log segments found. Creating new log.
W20250109 08:59:13.520702 25263 leader_election.cc:343] T 51277cd5f9374666bc60232309efc100 P 9d26e8b39691431590fd77e182046c66 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e6d654389db14a49a1f1bf3dbe538a27 (127.19.226.195:39911): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:59:13.521183 25263 leader_election.cc:304] T 51277cd5f9374666bc60232309efc100 P 9d26e8b39691431590fd77e182046c66 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9d26e8b39691431590fd77e182046c66; no voters: 2ff0106754a84b39989b9e042b4af3d6, e6d654389db14a49a1f1bf3dbe538a27
I20250109 08:59:13.514196 25452 leader_election.cc:290] T 51277cd5f9374666bc60232309efc100 P 9d26e8b39691431590fd77e182046c66 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2ff0106754a84b39989b9e042b4af3d6 (127.19.226.193:34345), e6d654389db14a49a1f1bf3dbe538a27 (127.19.226.195:39911)
I20250109 08:59:13.522650 25452 raft_consensus.cc:2743] T 51277cd5f9374666bc60232309efc100 P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:59:13.525110 25510 tablet_bootstrap.cc:492] T 3f4e7967061b4173bd00a9539a371fe3 P e6d654389db14a49a1f1bf3dbe538a27: Bootstrap starting.
I20250109 08:59:13.542491 25510 tablet_bootstrap.cc:654] T 3f4e7967061b4173bd00a9539a371fe3 P e6d654389db14a49a1f1bf3dbe538a27: Neither blocks nor log segments found. Creating new log.
I20250109 08:59:13.543952 25511 tablet_bootstrap.cc:492] T 9bf180fefb074fd192f6f80ae1460ba5 P 2ff0106754a84b39989b9e042b4af3d6: No bootstrap required, opened a new log
I20250109 08:59:13.544579 25511 ts_tablet_manager.cc:1397] T 9bf180fefb074fd192f6f80ae1460ba5 P 2ff0106754a84b39989b9e042b4af3d6: Time spent bootstrapping tablet: real 0.051s user 0.024s sys 0.027s
I20250109 08:59:13.545166 25517 raft_consensus.cc:491] T 346220ee65da44b8862c9f28d7aa6aa3 P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:59:13.545972 25517 raft_consensus.cc:513] T 346220ee65da44b8862c9f28d7aa6aa3 P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:13.547648 25511 raft_consensus.cc:357] T 9bf180fefb074fd192f6f80ae1460ba5 P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:13.548494 25511 raft_consensus.cc:383] T 9bf180fefb074fd192f6f80ae1460ba5 P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:59:13.548754 25511 raft_consensus.cc:738] T 9bf180fefb074fd192f6f80ae1460ba5 P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ff0106754a84b39989b9e042b4af3d6, State: Initialized, Role: FOLLOWER
I20250109 08:59:13.549417 25222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "346220ee65da44b8862c9f28d7aa6aa3" candidate_uuid: "9d26e8b39691431590fd77e182046c66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ff0106754a84b39989b9e042b4af3d6" is_pre_election: true
I20250109 08:59:13.549425 25511 consensus_queue.cc:260] T 9bf180fefb074fd192f6f80ae1460ba5 P 2ff0106754a84b39989b9e042b4af3d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
W20250109 08:59:13.551275 25261 leader_election.cc:343] T 346220ee65da44b8862c9f28d7aa6aa3 P 9d26e8b39691431590fd77e182046c66 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2ff0106754a84b39989b9e042b4af3d6 (127.19.226.193:34345): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:59:13.554139 25371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "346220ee65da44b8862c9f28d7aa6aa3" candidate_uuid: "9d26e8b39691431590fd77e182046c66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6d654389db14a49a1f1bf3dbe538a27" is_pre_election: true
I20250109 08:59:13.555081 25511 ts_tablet_manager.cc:1428] T 9bf180fefb074fd192f6f80ae1460ba5 P 2ff0106754a84b39989b9e042b4af3d6: Time spent starting tablet: real 0.010s user 0.005s sys 0.000s
W20250109 08:59:13.555757 25263 leader_election.cc:343] T 346220ee65da44b8862c9f28d7aa6aa3 P 9d26e8b39691431590fd77e182046c66 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e6d654389db14a49a1f1bf3dbe538a27 (127.19.226.195:39911): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:59:13.556200 25511 tablet_bootstrap.cc:492] T 8022fc0563b947669983f72b9b7450b3 P 2ff0106754a84b39989b9e042b4af3d6: Bootstrap starting.
I20250109 08:59:13.556408 25263 leader_election.cc:304] T 346220ee65da44b8862c9f28d7aa6aa3 P 9d26e8b39691431590fd77e182046c66 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9d26e8b39691431590fd77e182046c66; no voters: 2ff0106754a84b39989b9e042b4af3d6, e6d654389db14a49a1f1bf3dbe538a27
I20250109 08:59:13.557454 25452 raft_consensus.cc:2743] T 346220ee65da44b8862c9f28d7aa6aa3 P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:59:13.558243 25517 leader_election.cc:290] T 346220ee65da44b8862c9f28d7aa6aa3 P 9d26e8b39691431590fd77e182046c66 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2ff0106754a84b39989b9e042b4af3d6 (127.19.226.193:34345), e6d654389db14a49a1f1bf3dbe538a27 (127.19.226.195:39911)
I20250109 08:59:13.561467 25509 tablet_bootstrap.cc:492] T ed0e3d98ad79424caf0e24ecf3b5f295 P 9d26e8b39691431590fd77e182046c66: No bootstrap required, opened a new log
I20250109 08:59:13.562070 25509 ts_tablet_manager.cc:1397] T ed0e3d98ad79424caf0e24ecf3b5f295 P 9d26e8b39691431590fd77e182046c66: Time spent bootstrapping tablet: real 0.049s user 0.031s sys 0.015s
I20250109 08:59:13.563264 25511 tablet_bootstrap.cc:654] T 8022fc0563b947669983f72b9b7450b3 P 2ff0106754a84b39989b9e042b4af3d6: Neither blocks nor log segments found. Creating new log.
I20250109 08:59:13.565001 25509 raft_consensus.cc:357] T ed0e3d98ad79424caf0e24ecf3b5f295 P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:13.565833 25509 raft_consensus.cc:383] T ed0e3d98ad79424caf0e24ecf3b5f295 P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:59:13.566151 25509 raft_consensus.cc:738] T ed0e3d98ad79424caf0e24ecf3b5f295 P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d26e8b39691431590fd77e182046c66, State: Initialized, Role: FOLLOWER
I20250109 08:59:13.567176 25510 tablet_bootstrap.cc:492] T 3f4e7967061b4173bd00a9539a371fe3 P e6d654389db14a49a1f1bf3dbe538a27: No bootstrap required, opened a new log
I20250109 08:59:13.566860 25509 consensus_queue.cc:260] T ed0e3d98ad79424caf0e24ecf3b5f295 P 9d26e8b39691431590fd77e182046c66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:13.567612 25510 ts_tablet_manager.cc:1397] T 3f4e7967061b4173bd00a9539a371fe3 P e6d654389db14a49a1f1bf3dbe538a27: Time spent bootstrapping tablet: real 0.043s user 0.008s sys 0.005s
I20250109 08:59:13.570103 25509 ts_tablet_manager.cc:1428] T ed0e3d98ad79424caf0e24ecf3b5f295 P 9d26e8b39691431590fd77e182046c66: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20250109 08:59:13.570696 25511 tablet_bootstrap.cc:492] T 8022fc0563b947669983f72b9b7450b3 P 2ff0106754a84b39989b9e042b4af3d6: No bootstrap required, opened a new log
I20250109 08:59:13.571194 25511 ts_tablet_manager.cc:1397] T 8022fc0563b947669983f72b9b7450b3 P 2ff0106754a84b39989b9e042b4af3d6: Time spent bootstrapping tablet: real 0.015s user 0.006s sys 0.005s
I20250109 08:59:13.571285 25509 tablet_bootstrap.cc:492] T 8022fc0563b947669983f72b9b7450b3 P 9d26e8b39691431590fd77e182046c66: Bootstrap starting.
I20250109 08:59:13.570370 25510 raft_consensus.cc:357] T 3f4e7967061b4173bd00a9539a371fe3 P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:13.573194 25510 raft_consensus.cc:383] T 3f4e7967061b4173bd00a9539a371fe3 P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:59:13.573542 25510 raft_consensus.cc:738] T 3f4e7967061b4173bd00a9539a371fe3 P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6d654389db14a49a1f1bf3dbe538a27, State: Initialized, Role: FOLLOWER
I20250109 08:59:13.574030 25511 raft_consensus.cc:357] T 8022fc0563b947669983f72b9b7450b3 P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:13.574934 25511 raft_consensus.cc:383] T 8022fc0563b947669983f72b9b7450b3 P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:59:13.575166 25511 raft_consensus.cc:738] T 8022fc0563b947669983f72b9b7450b3 P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ff0106754a84b39989b9e042b4af3d6, State: Initialized, Role: FOLLOWER
I20250109 08:59:13.575733 25511 consensus_queue.cc:260] T 8022fc0563b947669983f72b9b7450b3 P 2ff0106754a84b39989b9e042b4af3d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:13.574337 25510 consensus_queue.cc:260] T 3f4e7967061b4173bd00a9539a371fe3 P e6d654389db14a49a1f1bf3dbe538a27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:13.577735 25509 tablet_bootstrap.cc:654] T 8022fc0563b947669983f72b9b7450b3 P 9d26e8b39691431590fd77e182046c66: Neither blocks nor log segments found. Creating new log.
I20250109 08:59:13.578352 25511 ts_tablet_manager.cc:1428] T 8022fc0563b947669983f72b9b7450b3 P 2ff0106754a84b39989b9e042b4af3d6: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20250109 08:59:13.579373 25511 tablet_bootstrap.cc:492] T cb8ed58651e74cbabff2b027ba4a9a1e P 2ff0106754a84b39989b9e042b4af3d6: Bootstrap starting.
I20250109 08:59:13.579428 25510 ts_tablet_manager.cc:1428] T 3f4e7967061b4173bd00a9539a371fe3 P e6d654389db14a49a1f1bf3dbe538a27: Time spent starting tablet: real 0.012s user 0.005s sys 0.002s
I20250109 08:59:13.580896 25510 tablet_bootstrap.cc:492] T 9bf180fefb074fd192f6f80ae1460ba5 P e6d654389db14a49a1f1bf3dbe538a27: Bootstrap starting.
I20250109 08:59:13.582535 25450 raft_consensus.cc:491] T 3f4e7967061b4173bd00a9539a371fe3 P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:59:13.583014 25450 raft_consensus.cc:513] T 3f4e7967061b4173bd00a9539a371fe3 P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:13.585021 25450 leader_election.cc:290] T 3f4e7967061b4173bd00a9539a371fe3 P e6d654389db14a49a1f1bf3dbe538a27 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2ff0106754a84b39989b9e042b4af3d6 (127.19.226.193:34345), 9d26e8b39691431590fd77e182046c66 (127.19.226.194:33163)
I20250109 08:59:13.586663 25222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f4e7967061b4173bd00a9539a371fe3" candidate_uuid: "e6d654389db14a49a1f1bf3dbe538a27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ff0106754a84b39989b9e042b4af3d6" is_pre_election: true
I20250109 08:59:13.587755 25510 tablet_bootstrap.cc:654] T 9bf180fefb074fd192f6f80ae1460ba5 P e6d654389db14a49a1f1bf3dbe538a27: Neither blocks nor log segments found. Creating new log.
I20250109 08:59:13.588634 25295 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f4e7967061b4173bd00a9539a371fe3" candidate_uuid: "e6d654389db14a49a1f1bf3dbe538a27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d26e8b39691431590fd77e182046c66" is_pre_election: true
W20250109 08:59:13.590348 25335 leader_election.cc:343] T 3f4e7967061b4173bd00a9539a371fe3 P e6d654389db14a49a1f1bf3dbe538a27 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2ff0106754a84b39989b9e042b4af3d6 (127.19.226.193:34345): Illegal state: must be running to vote when last-logged opid is not known
W20250109 08:59:13.591094 25336 leader_election.cc:343] T 3f4e7967061b4173bd00a9539a371fe3 P e6d654389db14a49a1f1bf3dbe538a27 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9d26e8b39691431590fd77e182046c66 (127.19.226.194:33163): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:59:13.591517 25336 leader_election.cc:304] T 3f4e7967061b4173bd00a9539a371fe3 P e6d654389db14a49a1f1bf3dbe538a27 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e6d654389db14a49a1f1bf3dbe538a27; no voters: 2ff0106754a84b39989b9e042b4af3d6, 9d26e8b39691431590fd77e182046c66
I20250109 08:59:13.592629 25450 raft_consensus.cc:2743] T 3f4e7967061b4173bd00a9539a371fe3 P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:59:13.597394 25511 tablet_bootstrap.cc:654] T cb8ed58651e74cbabff2b027ba4a9a1e P 2ff0106754a84b39989b9e042b4af3d6: Neither blocks nor log segments found. Creating new log.
I20250109 08:59:13.601715 25424 raft_consensus.cc:491] T 8022fc0563b947669983f72b9b7450b3 P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:59:13.602306 25424 raft_consensus.cc:513] T 8022fc0563b947669983f72b9b7450b3 P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:13.605640 25295 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8022fc0563b947669983f72b9b7450b3" candidate_uuid: "2ff0106754a84b39989b9e042b4af3d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d26e8b39691431590fd77e182046c66" is_pre_election: true
W20250109 08:59:13.607163 25187 leader_election.cc:343] T 8022fc0563b947669983f72b9b7450b3 P 2ff0106754a84b39989b9e042b4af3d6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9d26e8b39691431590fd77e182046c66 (127.19.226.194:33163): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:59:13.607959 25424 leader_election.cc:290] T 8022fc0563b947669983f72b9b7450b3 P 2ff0106754a84b39989b9e042b4af3d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9d26e8b39691431590fd77e182046c66 (127.19.226.194:33163), e6d654389db14a49a1f1bf3dbe538a27 (127.19.226.195:39911)
I20250109 08:59:13.609555 25371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8022fc0563b947669983f72b9b7450b3" candidate_uuid: "2ff0106754a84b39989b9e042b4af3d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6d654389db14a49a1f1bf3dbe538a27" is_pre_election: true
W20250109 08:59:13.611168 25188 leader_election.cc:343] T 8022fc0563b947669983f72b9b7450b3 P 2ff0106754a84b39989b9e042b4af3d6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e6d654389db14a49a1f1bf3dbe538a27 (127.19.226.195:39911): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:59:13.611482 25188 leader_election.cc:304] T 8022fc0563b947669983f72b9b7450b3 P 2ff0106754a84b39989b9e042b4af3d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ff0106754a84b39989b9e042b4af3d6; no voters: 9d26e8b39691431590fd77e182046c66, e6d654389db14a49a1f1bf3dbe538a27
I20250109 08:59:13.612026 25424 raft_consensus.cc:2743] T 8022fc0563b947669983f72b9b7450b3 P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:59:13.613418 25511 tablet_bootstrap.cc:492] T cb8ed58651e74cbabff2b027ba4a9a1e P 2ff0106754a84b39989b9e042b4af3d6: No bootstrap required, opened a new log
I20250109 08:59:13.613855 25511 ts_tablet_manager.cc:1397] T cb8ed58651e74cbabff2b027ba4a9a1e P 2ff0106754a84b39989b9e042b4af3d6: Time spent bootstrapping tablet: real 0.035s user 0.014s sys 0.008s
I20250109 08:59:13.614539 25510 tablet_bootstrap.cc:492] T 9bf180fefb074fd192f6f80ae1460ba5 P e6d654389db14a49a1f1bf3dbe538a27: No bootstrap required, opened a new log
I20250109 08:59:13.614598 25509 tablet_bootstrap.cc:492] T 8022fc0563b947669983f72b9b7450b3 P 9d26e8b39691431590fd77e182046c66: No bootstrap required, opened a new log
I20250109 08:59:13.615162 25510 ts_tablet_manager.cc:1397] T 9bf180fefb074fd192f6f80ae1460ba5 P e6d654389db14a49a1f1bf3dbe538a27: Time spent bootstrapping tablet: real 0.034s user 0.010s sys 0.006s
I20250109 08:59:13.615209 25509 ts_tablet_manager.cc:1397] T 8022fc0563b947669983f72b9b7450b3 P 9d26e8b39691431590fd77e182046c66: Time spent bootstrapping tablet: real 0.044s user 0.021s sys 0.021s
I20250109 08:59:13.616071 25511 raft_consensus.cc:357] T cb8ed58651e74cbabff2b027ba4a9a1e P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:13.616706 25511 raft_consensus.cc:383] T cb8ed58651e74cbabff2b027ba4a9a1e P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:59:13.616925 25511 raft_consensus.cc:738] T cb8ed58651e74cbabff2b027ba4a9a1e P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ff0106754a84b39989b9e042b4af3d6, State: Initialized, Role: FOLLOWER
I20250109 08:59:13.617587 25511 consensus_queue.cc:260] T cb8ed58651e74cbabff2b027ba4a9a1e P 2ff0106754a84b39989b9e042b4af3d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:13.618085 25509 raft_consensus.cc:357] T 8022fc0563b947669983f72b9b7450b3 P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:13.618857 25509 raft_consensus.cc:383] T 8022fc0563b947669983f72b9b7450b3 P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:59:13.619194 25509 raft_consensus.cc:738] T 8022fc0563b947669983f72b9b7450b3 P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d26e8b39691431590fd77e182046c66, State: Initialized, Role: FOLLOWER
I20250109 08:59:13.620162 25511 ts_tablet_manager.cc:1428] T cb8ed58651e74cbabff2b027ba4a9a1e P 2ff0106754a84b39989b9e042b4af3d6: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20250109 08:59:13.619868 25509 consensus_queue.cc:260] T 8022fc0563b947669983f72b9b7450b3 P 9d26e8b39691431590fd77e182046c66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:13.621192 25511 tablet_bootstrap.cc:492] T 3b37da4b59cf41b8889527eb2b0793c9 P 2ff0106754a84b39989b9e042b4af3d6: Bootstrap starting.
I20250109 08:59:13.622308 25509 ts_tablet_manager.cc:1428] T 8022fc0563b947669983f72b9b7450b3 P 9d26e8b39691431590fd77e182046c66: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20250109 08:59:13.618268 25510 raft_consensus.cc:357] T 9bf180fefb074fd192f6f80ae1460ba5 P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:13.623265 25510 raft_consensus.cc:383] T 9bf180fefb074fd192f6f80ae1460ba5 P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:59:13.623529 25510 raft_consensus.cc:738] T 9bf180fefb074fd192f6f80ae1460ba5 P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6d654389db14a49a1f1bf3dbe538a27, State: Initialized, Role: FOLLOWER
I20250109 08:59:13.624219 25510 consensus_queue.cc:260] T 9bf180fefb074fd192f6f80ae1460ba5 P e6d654389db14a49a1f1bf3dbe538a27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:13.626735 25510 ts_tablet_manager.cc:1428] T 9bf180fefb074fd192f6f80ae1460ba5 P e6d654389db14a49a1f1bf3dbe538a27: Time spent starting tablet: real 0.011s user 0.005s sys 0.000s
I20250109 08:59:13.627586 25510 tablet_bootstrap.cc:492] T 8022fc0563b947669983f72b9b7450b3 P e6d654389db14a49a1f1bf3dbe538a27: Bootstrap starting.
I20250109 08:59:13.628695 25511 tablet_bootstrap.cc:654] T 3b37da4b59cf41b8889527eb2b0793c9 P 2ff0106754a84b39989b9e042b4af3d6: Neither blocks nor log segments found. Creating new log.
I20250109 08:59:13.629134 25509 tablet_bootstrap.cc:492] T 9a3240fe2e094ba2bc002579bdb29e84 P 9d26e8b39691431590fd77e182046c66: Bootstrap starting.
I20250109 08:59:13.633569 25510 tablet_bootstrap.cc:654] T 8022fc0563b947669983f72b9b7450b3 P e6d654389db14a49a1f1bf3dbe538a27: Neither blocks nor log segments found. Creating new log.
I20250109 08:59:13.636015 25509 tablet_bootstrap.cc:654] T 9a3240fe2e094ba2bc002579bdb29e84 P 9d26e8b39691431590fd77e182046c66: Neither blocks nor log segments found. Creating new log.
I20250109 08:59:13.655268 25509 tablet_bootstrap.cc:492] T 9a3240fe2e094ba2bc002579bdb29e84 P 9d26e8b39691431590fd77e182046c66: No bootstrap required, opened a new log
I20250109 08:59:13.657254 25509 ts_tablet_manager.cc:1397] T 9a3240fe2e094ba2bc002579bdb29e84 P 9d26e8b39691431590fd77e182046c66: Time spent bootstrapping tablet: real 0.028s user 0.010s sys 0.015s
I20250109 08:59:13.657990 25511 tablet_bootstrap.cc:492] T 3b37da4b59cf41b8889527eb2b0793c9 P 2ff0106754a84b39989b9e042b4af3d6: No bootstrap required, opened a new log
I20250109 08:59:13.658514 25511 ts_tablet_manager.cc:1397] T 3b37da4b59cf41b8889527eb2b0793c9 P 2ff0106754a84b39989b9e042b4af3d6: Time spent bootstrapping tablet: real 0.038s user 0.017s sys 0.005s
I20250109 08:59:13.656684 25510 tablet_bootstrap.cc:492] T 8022fc0563b947669983f72b9b7450b3 P e6d654389db14a49a1f1bf3dbe538a27: No bootstrap required, opened a new log
I20250109 08:59:13.660712 25510 ts_tablet_manager.cc:1397] T 8022fc0563b947669983f72b9b7450b3 P e6d654389db14a49a1f1bf3dbe538a27: Time spent bootstrapping tablet: real 0.033s user 0.025s sys 0.004s
I20250109 08:59:13.660212 25509 raft_consensus.cc:357] T 9a3240fe2e094ba2bc002579bdb29e84 P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:13.661113 25509 raft_consensus.cc:383] T 9a3240fe2e094ba2bc002579bdb29e84 P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:59:13.661093 25511 raft_consensus.cc:357] T 3b37da4b59cf41b8889527eb2b0793c9 P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:13.661608 25509 raft_consensus.cc:738] T 9a3240fe2e094ba2bc002579bdb29e84 P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d26e8b39691431590fd77e182046c66, State: Initialized, Role: FOLLOWER
I20250109 08:59:13.661984 25511 raft_consensus.cc:383] T 3b37da4b59cf41b8889527eb2b0793c9 P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:59:13.662302 25511 raft_consensus.cc:738] T 3b37da4b59cf41b8889527eb2b0793c9 P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ff0106754a84b39989b9e042b4af3d6, State: Initialized, Role: FOLLOWER
I20250109 08:59:13.662573 25509 consensus_queue.cc:260] T 9a3240fe2e094ba2bc002579bdb29e84 P 9d26e8b39691431590fd77e182046c66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:13.662997 25510 raft_consensus.cc:357] T 8022fc0563b947669983f72b9b7450b3 P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:13.662997 25511 consensus_queue.cc:260] T 3b37da4b59cf41b8889527eb2b0793c9 P 2ff0106754a84b39989b9e042b4af3d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:13.663671 25510 raft_consensus.cc:383] T 8022fc0563b947669983f72b9b7450b3 P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:59:13.664074 25510 raft_consensus.cc:738] T 8022fc0563b947669983f72b9b7450b3 P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6d654389db14a49a1f1bf3dbe538a27, State: Initialized, Role: FOLLOWER
I20250109 08:59:13.664855 25510 consensus_queue.cc:260] T 8022fc0563b947669983f72b9b7450b3 P e6d654389db14a49a1f1bf3dbe538a27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:13.665469 25509 ts_tablet_manager.cc:1428] T 9a3240fe2e094ba2bc002579bdb29e84 P 9d26e8b39691431590fd77e182046c66: Time spent starting tablet: real 0.008s user 0.006s sys 0.001s
I20250109 08:59:13.665869 25511 ts_tablet_manager.cc:1428] T 3b37da4b59cf41b8889527eb2b0793c9 P 2ff0106754a84b39989b9e042b4af3d6: Time spent starting tablet: real 0.007s user 0.006s sys 0.001s
I20250109 08:59:13.666546 25509 tablet_bootstrap.cc:492] T 3f4e7967061b4173bd00a9539a371fe3 P 9d26e8b39691431590fd77e182046c66: Bootstrap starting.
I20250109 08:59:13.666956 25450 raft_consensus.cc:491] T 3b37da4b59cf41b8889527eb2b0793c9 P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:59:13.667279 25511 tablet_bootstrap.cc:492] T 9a3240fe2e094ba2bc002579bdb29e84 P 2ff0106754a84b39989b9e042b4af3d6: Bootstrap starting.
I20250109 08:59:13.667440 25450 raft_consensus.cc:513] T 3b37da4b59cf41b8889527eb2b0793c9 P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:13.669608 25450 leader_election.cc:290] T 3b37da4b59cf41b8889527eb2b0793c9 P e6d654389db14a49a1f1bf3dbe538a27 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2ff0106754a84b39989b9e042b4af3d6 (127.19.226.193:34345), 9d26e8b39691431590fd77e182046c66 (127.19.226.194:33163)
I20250109 08:59:13.671101 25295 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b37da4b59cf41b8889527eb2b0793c9" candidate_uuid: "e6d654389db14a49a1f1bf3dbe538a27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d26e8b39691431590fd77e182046c66" is_pre_election: true
I20250109 08:59:13.673004 25222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b37da4b59cf41b8889527eb2b0793c9" candidate_uuid: "e6d654389db14a49a1f1bf3dbe538a27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ff0106754a84b39989b9e042b4af3d6" is_pre_election: true
I20250109 08:59:13.673519 25509 tablet_bootstrap.cc:654] T 3f4e7967061b4173bd00a9539a371fe3 P 9d26e8b39691431590fd77e182046c66: Neither blocks nor log segments found. Creating new log.
I20250109 08:59:13.673683 25222 raft_consensus.cc:2463] T 3b37da4b59cf41b8889527eb2b0793c9 P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6d654389db14a49a1f1bf3dbe538a27 in term 0.
W20250109 08:59:13.674875 25336 leader_election.cc:343] T 3b37da4b59cf41b8889527eb2b0793c9 P e6d654389db14a49a1f1bf3dbe538a27 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9d26e8b39691431590fd77e182046c66 (127.19.226.194:33163): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:59:13.678584 25511 tablet_bootstrap.cc:654] T 9a3240fe2e094ba2bc002579bdb29e84 P 2ff0106754a84b39989b9e042b4af3d6: Neither blocks nor log segments found. Creating new log.
I20250109 08:59:13.680447 25510 ts_tablet_manager.cc:1428] T 8022fc0563b947669983f72b9b7450b3 P e6d654389db14a49a1f1bf3dbe538a27: Time spent starting tablet: real 0.019s user 0.005s sys 0.002s
I20250109 08:59:13.676003 25335 leader_election.cc:304] T 3b37da4b59cf41b8889527eb2b0793c9 P e6d654389db14a49a1f1bf3dbe538a27 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2ff0106754a84b39989b9e042b4af3d6, e6d654389db14a49a1f1bf3dbe538a27; no voters: 9d26e8b39691431590fd77e182046c66
I20250109 08:59:13.681542 25510 tablet_bootstrap.cc:492] T 346220ee65da44b8862c9f28d7aa6aa3 P e6d654389db14a49a1f1bf3dbe538a27: Bootstrap starting.
I20250109 08:59:13.681982 25450 raft_consensus.cc:2798] T 3b37da4b59cf41b8889527eb2b0793c9 P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:59:13.682479 25450 raft_consensus.cc:491] T 3b37da4b59cf41b8889527eb2b0793c9 P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:59:13.682870 25450 raft_consensus.cc:3054] T 3b37da4b59cf41b8889527eb2b0793c9 P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:59:13.689775 25450 raft_consensus.cc:513] T 3b37da4b59cf41b8889527eb2b0793c9 P e6d654389db14a49a1f1bf3dbe538a27 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:13.691741 25450 leader_election.cc:290] T 3b37da4b59cf41b8889527eb2b0793c9 P e6d654389db14a49a1f1bf3dbe538a27 [CANDIDATE]: Term 1 election: Requested vote from peers 2ff0106754a84b39989b9e042b4af3d6 (127.19.226.193:34345), 9d26e8b39691431590fd77e182046c66 (127.19.226.194:33163)
I20250109 08:59:13.693291 25295 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b37da4b59cf41b8889527eb2b0793c9" candidate_uuid: "e6d654389db14a49a1f1bf3dbe538a27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d26e8b39691431590fd77e182046c66"
I20250109 08:59:13.693590 25510 tablet_bootstrap.cc:654] T 346220ee65da44b8862c9f28d7aa6aa3 P e6d654389db14a49a1f1bf3dbe538a27: Neither blocks nor log segments found. Creating new log.
I20250109 08:59:13.693768 25509 tablet_bootstrap.cc:492] T 3f4e7967061b4173bd00a9539a371fe3 P 9d26e8b39691431590fd77e182046c66: No bootstrap required, opened a new log
I20250109 08:59:13.694299 25509 ts_tablet_manager.cc:1397] T 3f4e7967061b4173bd00a9539a371fe3 P 9d26e8b39691431590fd77e182046c66: Time spent bootstrapping tablet: real 0.028s user 0.014s sys 0.003s
W20250109 08:59:13.694778 25336 leader_election.cc:343] T 3b37da4b59cf41b8889527eb2b0793c9 P e6d654389db14a49a1f1bf3dbe538a27 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9d26e8b39691431590fd77e182046c66 (127.19.226.194:33163): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:59:13.695652 25222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b37da4b59cf41b8889527eb2b0793c9" candidate_uuid: "e6d654389db14a49a1f1bf3dbe538a27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ff0106754a84b39989b9e042b4af3d6"
I20250109 08:59:13.696261 25222 raft_consensus.cc:3054] T 3b37da4b59cf41b8889527eb2b0793c9 P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:59:13.696496 25509 raft_consensus.cc:357] T 3f4e7967061b4173bd00a9539a371fe3 P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:13.698379 25509 raft_consensus.cc:383] T 3f4e7967061b4173bd00a9539a371fe3 P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:59:13.698719 25509 raft_consensus.cc:738] T 3f4e7967061b4173bd00a9539a371fe3 P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d26e8b39691431590fd77e182046c66, State: Initialized, Role: FOLLOWER
I20250109 08:59:13.699151 25510 tablet_bootstrap.cc:492] T 346220ee65da44b8862c9f28d7aa6aa3 P e6d654389db14a49a1f1bf3dbe538a27: No bootstrap required, opened a new log
I20250109 08:59:13.699652 25510 ts_tablet_manager.cc:1397] T 346220ee65da44b8862c9f28d7aa6aa3 P e6d654389db14a49a1f1bf3dbe538a27: Time spent bootstrapping tablet: real 0.018s user 0.008s sys 0.003s
I20250109 08:59:13.699672 25509 consensus_queue.cc:260] T 3f4e7967061b4173bd00a9539a371fe3 P 9d26e8b39691431590fd77e182046c66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:13.703001 25509 ts_tablet_manager.cc:1428] T 3f4e7967061b4173bd00a9539a371fe3 P 9d26e8b39691431590fd77e182046c66: Time spent starting tablet: real 0.008s user 0.005s sys 0.002s
I20250109 08:59:13.703318 25222 raft_consensus.cc:2463] T 3b37da4b59cf41b8889527eb2b0793c9 P 2ff0106754a84b39989b9e042b4af3d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6d654389db14a49a1f1bf3dbe538a27 in term 1.
I20250109 08:59:13.704847 25335 leader_election.cc:304] T 3b37da4b59cf41b8889527eb2b0793c9 P e6d654389db14a49a1f1bf3dbe538a27 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2ff0106754a84b39989b9e042b4af3d6, e6d654389db14a49a1f1bf3dbe538a27; no voters: 9d26e8b39691431590fd77e182046c66
I20250109 08:59:13.705046 25510 raft_consensus.cc:357] T 346220ee65da44b8862c9f28d7aa6aa3 P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:13.705942 25510 raft_consensus.cc:383] T 346220ee65da44b8862c9f28d7aa6aa3 P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:59:13.706298 25510 raft_consensus.cc:738] T 346220ee65da44b8862c9f28d7aa6aa3 P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6d654389db14a49a1f1bf3dbe538a27, State: Initialized, Role: FOLLOWER
I20250109 08:59:13.706432 25450 raft_consensus.cc:2798] T 3b37da4b59cf41b8889527eb2b0793c9 P e6d654389db14a49a1f1bf3dbe538a27 [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:59:13.706996 25510 consensus_queue.cc:260] T 346220ee65da44b8862c9f28d7aa6aa3 P e6d654389db14a49a1f1bf3dbe538a27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:13.707648 25511 tablet_bootstrap.cc:492] T 9a3240fe2e094ba2bc002579bdb29e84 P 2ff0106754a84b39989b9e042b4af3d6: No bootstrap required, opened a new log
I20250109 08:59:13.708201 25511 ts_tablet_manager.cc:1397] T 9a3240fe2e094ba2bc002579bdb29e84 P 2ff0106754a84b39989b9e042b4af3d6: Time spent bootstrapping tablet: real 0.041s user 0.010s sys 0.009s
I20250109 08:59:13.709026 25509 tablet_bootstrap.cc:492] T 9bf180fefb074fd192f6f80ae1460ba5 P 9d26e8b39691431590fd77e182046c66: Bootstrap starting.
I20250109 08:59:13.711158 25517 raft_consensus.cc:491] T 8022fc0563b947669983f72b9b7450b3 P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:59:13.711717 25517 raft_consensus.cc:513] T 8022fc0563b947669983f72b9b7450b3 P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:13.713780 25517 leader_election.cc:290] T 8022fc0563b947669983f72b9b7450b3 P 9d26e8b39691431590fd77e182046c66 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2ff0106754a84b39989b9e042b4af3d6 (127.19.226.193:34345), e6d654389db14a49a1f1bf3dbe538a27 (127.19.226.195:39911)
I20250109 08:59:13.710930 25511 raft_consensus.cc:357] T 9a3240fe2e094ba2bc002579bdb29e84 P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:13.715587 25511 raft_consensus.cc:383] T 9a3240fe2e094ba2bc002579bdb29e84 P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:59:13.715953 25511 raft_consensus.cc:738] T 9a3240fe2e094ba2bc002579bdb29e84 P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ff0106754a84b39989b9e042b4af3d6, State: Initialized, Role: FOLLOWER
I20250109 08:59:13.715830 25222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8022fc0563b947669983f72b9b7450b3" candidate_uuid: "9d26e8b39691431590fd77e182046c66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ff0106754a84b39989b9e042b4af3d6" is_pre_election: true
I20250109 08:59:13.716437 25371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8022fc0563b947669983f72b9b7450b3" candidate_uuid: "9d26e8b39691431590fd77e182046c66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6d654389db14a49a1f1bf3dbe538a27" is_pre_election: true
I20250109 08:59:13.716778 25222 raft_consensus.cc:2463] T 8022fc0563b947669983f72b9b7450b3 P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d26e8b39691431590fd77e182046c66 in term 0.
I20250109 08:59:13.717137 25371 raft_consensus.cc:2463] T 8022fc0563b947669983f72b9b7450b3 P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d26e8b39691431590fd77e182046c66 in term 0.
I20250109 08:59:13.716961 25511 consensus_queue.cc:260] T 9a3240fe2e094ba2bc002579bdb29e84 P 2ff0106754a84b39989b9e042b4af3d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:13.718012 25261 leader_election.cc:304] T 8022fc0563b947669983f72b9b7450b3 P 9d26e8b39691431590fd77e182046c66 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ff0106754a84b39989b9e042b4af3d6, 9d26e8b39691431590fd77e182046c66; no voters:
I20250109 08:59:13.719367 25517 raft_consensus.cc:2798] T 8022fc0563b947669983f72b9b7450b3 P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:59:13.719455 25509 tablet_bootstrap.cc:654] T 9bf180fefb074fd192f6f80ae1460ba5 P 9d26e8b39691431590fd77e182046c66: Neither blocks nor log segments found. Creating new log.
I20250109 08:59:13.719831 25517 raft_consensus.cc:491] T 8022fc0563b947669983f72b9b7450b3 P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:59:13.720355 25517 raft_consensus.cc:3054] T 8022fc0563b947669983f72b9b7450b3 P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:59:13.728734 25517 raft_consensus.cc:513] T 8022fc0563b947669983f72b9b7450b3 P 9d26e8b39691431590fd77e182046c66 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:13.730589 25509 tablet_bootstrap.cc:492] T 9bf180fefb074fd192f6f80ae1460ba5 P 9d26e8b39691431590fd77e182046c66: No bootstrap required, opened a new log
I20250109 08:59:13.730983 25450 raft_consensus.cc:695] T 3b37da4b59cf41b8889527eb2b0793c9 P e6d654389db14a49a1f1bf3dbe538a27 [term 1 LEADER]: Becoming Leader. State: Replica: e6d654389db14a49a1f1bf3dbe538a27, State: Running, Role: LEADER
I20250109 08:59:13.731626 25222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8022fc0563b947669983f72b9b7450b3" candidate_uuid: "9d26e8b39691431590fd77e182046c66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ff0106754a84b39989b9e042b4af3d6"
I20250109 08:59:13.731148 25509 ts_tablet_manager.cc:1397] T 9bf180fefb074fd192f6f80ae1460ba5 P 9d26e8b39691431590fd77e182046c66: Time spent bootstrapping tablet: real 0.022s user 0.014s sys 0.000s
I20250109 08:59:13.732285 25222 raft_consensus.cc:3054] T 8022fc0563b947669983f72b9b7450b3 P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:59:13.732010 25450 consensus_queue.cc:237] T 3b37da4b59cf41b8889527eb2b0793c9 P e6d654389db14a49a1f1bf3dbe538a27 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:13.735180 25509 raft_consensus.cc:357] T 9bf180fefb074fd192f6f80ae1460ba5 P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:13.735944 25509 raft_consensus.cc:383] T 9bf180fefb074fd192f6f80ae1460ba5 P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:59:13.724771 25511 ts_tablet_manager.cc:1428] T 9a3240fe2e094ba2bc002579bdb29e84 P 2ff0106754a84b39989b9e042b4af3d6: Time spent starting tablet: real 0.016s user 0.004s sys 0.005s
I20250109 08:59:13.736408 25517 leader_election.cc:290] T 8022fc0563b947669983f72b9b7450b3 P 9d26e8b39691431590fd77e182046c66 [CANDIDATE]: Term 1 election: Requested vote from peers 2ff0106754a84b39989b9e042b4af3d6 (127.19.226.193:34345), e6d654389db14a49a1f1bf3dbe538a27 (127.19.226.195:39911)
I20250109 08:59:13.737248 25511 tablet_bootstrap.cc:492] T 51277cd5f9374666bc60232309efc100 P 2ff0106754a84b39989b9e042b4af3d6: Bootstrap starting.
I20250109 08:59:13.737452 25371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8022fc0563b947669983f72b9b7450b3" candidate_uuid: "9d26e8b39691431590fd77e182046c66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6d654389db14a49a1f1bf3dbe538a27"
I20250109 08:59:13.738080 25371 raft_consensus.cc:3054] T 8022fc0563b947669983f72b9b7450b3 P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:59:13.741782 25532 raft_consensus.cc:491] T cb8ed58651e74cbabff2b027ba4a9a1e P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:59:13.742309 25532 raft_consensus.cc:513] T cb8ed58651e74cbabff2b027ba4a9a1e P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:13.743299 25222 raft_consensus.cc:2463] T 8022fc0563b947669983f72b9b7450b3 P 2ff0106754a84b39989b9e042b4af3d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d26e8b39691431590fd77e182046c66 in term 1.
I20250109 08:59:13.745249 25295 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8ed58651e74cbabff2b027ba4a9a1e" candidate_uuid: "2ff0106754a84b39989b9e042b4af3d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d26e8b39691431590fd77e182046c66" is_pre_election: true
I20250109 08:59:13.745643 25369 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8ed58651e74cbabff2b027ba4a9a1e" candidate_uuid: "2ff0106754a84b39989b9e042b4af3d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6d654389db14a49a1f1bf3dbe538a27" is_pre_election: true
I20250109 08:59:13.746701 25510 ts_tablet_manager.cc:1428] T 346220ee65da44b8862c9f28d7aa6aa3 P e6d654389db14a49a1f1bf3dbe538a27: Time spent starting tablet: real 0.047s user 0.001s sys 0.011s
W20250109 08:59:13.746904 25187 leader_election.cc:343] T cb8ed58651e74cbabff2b027ba4a9a1e P 2ff0106754a84b39989b9e042b4af3d6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9d26e8b39691431590fd77e182046c66 (127.19.226.194:33163): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:59:13.747244 25511 tablet_bootstrap.cc:654] T 51277cd5f9374666bc60232309efc100 P 2ff0106754a84b39989b9e042b4af3d6: Neither blocks nor log segments found. Creating new log.
W20250109 08:59:13.747586 25188 leader_election.cc:343] T cb8ed58651e74cbabff2b027ba4a9a1e P 2ff0106754a84b39989b9e042b4af3d6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e6d654389db14a49a1f1bf3dbe538a27 (127.19.226.195:39911): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:59:13.747747 25510 tablet_bootstrap.cc:492] T cb8ed58651e74cbabff2b027ba4a9a1e P e6d654389db14a49a1f1bf3dbe538a27: Bootstrap starting.
I20250109 08:59:13.748095 25188 leader_election.cc:304] T cb8ed58651e74cbabff2b027ba4a9a1e P 2ff0106754a84b39989b9e042b4af3d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ff0106754a84b39989b9e042b4af3d6; no voters: 9d26e8b39691431590fd77e182046c66, e6d654389db14a49a1f1bf3dbe538a27
I20250109 08:59:13.749218 25424 raft_consensus.cc:2743] T cb8ed58651e74cbabff2b027ba4a9a1e P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:59:13.751022 25371 raft_consensus.cc:2463] T 8022fc0563b947669983f72b9b7450b3 P e6d654389db14a49a1f1bf3dbe538a27 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d26e8b39691431590fd77e182046c66 in term 1.
I20250109 08:59:13.752218 25263 leader_election.cc:304] T 8022fc0563b947669983f72b9b7450b3 P 9d26e8b39691431590fd77e182046c66 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d26e8b39691431590fd77e182046c66, e6d654389db14a49a1f1bf3dbe538a27; no voters:
I20250109 08:59:13.747171 25532 leader_election.cc:290] T cb8ed58651e74cbabff2b027ba4a9a1e P 2ff0106754a84b39989b9e042b4af3d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9d26e8b39691431590fd77e182046c66 (127.19.226.194:33163), e6d654389db14a49a1f1bf3dbe538a27 (127.19.226.195:39911)
I20250109 08:59:13.752897 25517 raft_consensus.cc:2798] T 8022fc0563b947669983f72b9b7450b3 P 9d26e8b39691431590fd77e182046c66 [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:59:13.736270 25509 raft_consensus.cc:738] T 9bf180fefb074fd192f6f80ae1460ba5 P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d26e8b39691431590fd77e182046c66, State: Initialized, Role: FOLLOWER
I20250109 08:59:13.753638 25517 raft_consensus.cc:695] T 8022fc0563b947669983f72b9b7450b3 P 9d26e8b39691431590fd77e182046c66 [term 1 LEADER]: Becoming Leader. State: Replica: 9d26e8b39691431590fd77e182046c66, State: Running, Role: LEADER
I20250109 08:59:13.754033 25509 consensus_queue.cc:260] T 9bf180fefb074fd192f6f80ae1460ba5 P 9d26e8b39691431590fd77e182046c66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:13.754510 25517 consensus_queue.cc:237] T 8022fc0563b947669983f72b9b7450b3 P 9d26e8b39691431590fd77e182046c66 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:13.756536 25510 tablet_bootstrap.cc:654] T cb8ed58651e74cbabff2b027ba4a9a1e P e6d654389db14a49a1f1bf3dbe538a27: Neither blocks nor log segments found. Creating new log.
I20250109 08:59:13.756827 25509 ts_tablet_manager.cc:1428] T 9bf180fefb074fd192f6f80ae1460ba5 P 9d26e8b39691431590fd77e182046c66: Time spent starting tablet: real 0.024s user 0.006s sys 0.001s
I20250109 08:59:13.757939 25509 tablet_bootstrap.cc:492] T cb8ed58651e74cbabff2b027ba4a9a1e P 9d26e8b39691431590fd77e182046c66: Bootstrap starting.
I20250109 08:59:13.763166 25511 tablet_bootstrap.cc:492] T 51277cd5f9374666bc60232309efc100 P 2ff0106754a84b39989b9e042b4af3d6: No bootstrap required, opened a new log
I20250109 08:59:13.763460 25510 tablet_bootstrap.cc:492] T cb8ed58651e74cbabff2b027ba4a9a1e P e6d654389db14a49a1f1bf3dbe538a27: No bootstrap required, opened a new log
I20250109 08:59:13.763723 25511 ts_tablet_manager.cc:1397] T 51277cd5f9374666bc60232309efc100 P 2ff0106754a84b39989b9e042b4af3d6: Time spent bootstrapping tablet: real 0.027s user 0.008s sys 0.004s
I20250109 08:59:13.763912 25510 ts_tablet_manager.cc:1397] T cb8ed58651e74cbabff2b027ba4a9a1e P e6d654389db14a49a1f1bf3dbe538a27: Time spent bootstrapping tablet: real 0.016s user 0.006s sys 0.005s
I20250109 08:59:13.767210 25509 tablet_bootstrap.cc:654] T cb8ed58651e74cbabff2b027ba4a9a1e P 9d26e8b39691431590fd77e182046c66: Neither blocks nor log segments found. Creating new log.
I20250109 08:59:13.767037 25510 raft_consensus.cc:357] T cb8ed58651e74cbabff2b027ba4a9a1e P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:13.767874 25510 raft_consensus.cc:383] T cb8ed58651e74cbabff2b027ba4a9a1e P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:59:13.768193 25510 raft_consensus.cc:738] T cb8ed58651e74cbabff2b027ba4a9a1e P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6d654389db14a49a1f1bf3dbe538a27, State: Initialized, Role: FOLLOWER
I20250109 08:59:13.766912 25511 raft_consensus.cc:357] T 51277cd5f9374666bc60232309efc100 P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:13.773315 25511 raft_consensus.cc:383] T 51277cd5f9374666bc60232309efc100 P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:59:13.773689 25511 raft_consensus.cc:738] T 51277cd5f9374666bc60232309efc100 P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ff0106754a84b39989b9e042b4af3d6, State: Initialized, Role: FOLLOWER
I20250109 08:59:13.774494 25511 consensus_queue.cc:260] T 51277cd5f9374666bc60232309efc100 P 2ff0106754a84b39989b9e042b4af3d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:13.775219 25510 consensus_queue.cc:260] T cb8ed58651e74cbabff2b027ba4a9a1e P e6d654389db14a49a1f1bf3dbe538a27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:13.777909 25510 ts_tablet_manager.cc:1428] T cb8ed58651e74cbabff2b027ba4a9a1e P e6d654389db14a49a1f1bf3dbe538a27: Time spent starting tablet: real 0.014s user 0.001s sys 0.007s
I20250109 08:59:13.778950 25510 tablet_bootstrap.cc:492] T 9a3240fe2e094ba2bc002579bdb29e84 P e6d654389db14a49a1f1bf3dbe538a27: Bootstrap starting.
I20250109 08:59:13.776185 25118 catalog_manager.cc:5526] T 3b37da4b59cf41b8889527eb2b0793c9 P e6d654389db14a49a1f1bf3dbe538a27 reported cstate change: term changed from 0 to 1, leader changed from <none> to e6d654389db14a49a1f1bf3dbe538a27 (127.19.226.195). New cstate: current_term: 1 leader_uuid: "e6d654389db14a49a1f1bf3dbe538a27" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } health_report { overall_health: HEALTHY } } }
I20250109 08:59:13.781612 25117 catalog_manager.cc:5526] T 8022fc0563b947669983f72b9b7450b3 P 9d26e8b39691431590fd77e182046c66 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9d26e8b39691431590fd77e182046c66 (127.19.226.194). New cstate: current_term: 1 leader_uuid: "9d26e8b39691431590fd77e182046c66" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } health_report { overall_health: UNKNOWN } } }
I20250109 08:59:13.782471 25511 ts_tablet_manager.cc:1428] T 51277cd5f9374666bc60232309efc100 P 2ff0106754a84b39989b9e042b4af3d6: Time spent starting tablet: real 0.018s user 0.006s sys 0.002s
I20250109 08:59:13.783654 25511 tablet_bootstrap.cc:492] T 3f4e7967061b4173bd00a9539a371fe3 P 2ff0106754a84b39989b9e042b4af3d6: Bootstrap starting.
I20250109 08:59:13.786006 25510 tablet_bootstrap.cc:654] T 9a3240fe2e094ba2bc002579bdb29e84 P e6d654389db14a49a1f1bf3dbe538a27: Neither blocks nor log segments found. Creating new log.
I20250109 08:59:13.790310 25511 tablet_bootstrap.cc:654] T 3f4e7967061b4173bd00a9539a371fe3 P 2ff0106754a84b39989b9e042b4af3d6: Neither blocks nor log segments found. Creating new log.
I20250109 08:59:13.792243 25517 raft_consensus.cc:491] T 9bf180fefb074fd192f6f80ae1460ba5 P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:59:13.792847 25509 tablet_bootstrap.cc:492] T cb8ed58651e74cbabff2b027ba4a9a1e P 9d26e8b39691431590fd77e182046c66: No bootstrap required, opened a new log
I20250109 08:59:13.792790 25517 raft_consensus.cc:513] T 9bf180fefb074fd192f6f80ae1460ba5 P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:13.793589 25509 ts_tablet_manager.cc:1397] T cb8ed58651e74cbabff2b027ba4a9a1e P 9d26e8b39691431590fd77e182046c66: Time spent bootstrapping tablet: real 0.036s user 0.010s sys 0.008s
I20250109 08:59:13.794842 25517 leader_election.cc:290] T 9bf180fefb074fd192f6f80ae1460ba5 P 9d26e8b39691431590fd77e182046c66 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2ff0106754a84b39989b9e042b4af3d6 (127.19.226.193:34345), e6d654389db14a49a1f1bf3dbe538a27 (127.19.226.195:39911)
I20250109 08:59:13.796291 25222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bf180fefb074fd192f6f80ae1460ba5" candidate_uuid: "9d26e8b39691431590fd77e182046c66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ff0106754a84b39989b9e042b4af3d6" is_pre_election: true
I20250109 08:59:13.796650 25371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bf180fefb074fd192f6f80ae1460ba5" candidate_uuid: "9d26e8b39691431590fd77e182046c66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6d654389db14a49a1f1bf3dbe538a27" is_pre_election: true
I20250109 08:59:13.797127 25222 raft_consensus.cc:2463] T 9bf180fefb074fd192f6f80ae1460ba5 P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d26e8b39691431590fd77e182046c66 in term 0.
I20250109 08:59:13.797338 25371 raft_consensus.cc:2463] T 9bf180fefb074fd192f6f80ae1460ba5 P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d26e8b39691431590fd77e182046c66 in term 0.
I20250109 08:59:13.796921 25509 raft_consensus.cc:357] T cb8ed58651e74cbabff2b027ba4a9a1e P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:13.797984 25509 raft_consensus.cc:383] T cb8ed58651e74cbabff2b027ba4a9a1e P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:59:13.798336 25509 raft_consensus.cc:738] T cb8ed58651e74cbabff2b027ba4a9a1e P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d26e8b39691431590fd77e182046c66, State: Initialized, Role: FOLLOWER
I20250109 08:59:13.798326 25261 leader_election.cc:304] T 9bf180fefb074fd192f6f80ae1460ba5 P 9d26e8b39691431590fd77e182046c66 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ff0106754a84b39989b9e042b4af3d6, 9d26e8b39691431590fd77e182046c66; no voters:
I20250109 08:59:13.799074 25509 consensus_queue.cc:260] T cb8ed58651e74cbabff2b027ba4a9a1e P 9d26e8b39691431590fd77e182046c66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:13.800302 25452 raft_consensus.cc:2798] T 9bf180fefb074fd192f6f80ae1460ba5 P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:59:13.800678 25452 raft_consensus.cc:491] T 9bf180fefb074fd192f6f80ae1460ba5 P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:59:13.800984 25452 raft_consensus.cc:3054] T 9bf180fefb074fd192f6f80ae1460ba5 P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:59:13.801546 25509 ts_tablet_manager.cc:1428] T cb8ed58651e74cbabff2b027ba4a9a1e P 9d26e8b39691431590fd77e182046c66: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20250109 08:59:13.802459 25509 tablet_bootstrap.cc:492] T 3b37da4b59cf41b8889527eb2b0793c9 P 9d26e8b39691431590fd77e182046c66: Bootstrap starting.
I20250109 08:59:13.807901 25452 raft_consensus.cc:513] T 9bf180fefb074fd192f6f80ae1460ba5 P 9d26e8b39691431590fd77e182046c66 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:13.808566 25509 tablet_bootstrap.cc:654] T 3b37da4b59cf41b8889527eb2b0793c9 P 9d26e8b39691431590fd77e182046c66: Neither blocks nor log segments found. Creating new log.
I20250109 08:59:13.810036 25452 leader_election.cc:290] T 9bf180fefb074fd192f6f80ae1460ba5 P 9d26e8b39691431590fd77e182046c66 [CANDIDATE]: Term 1 election: Requested vote from peers 2ff0106754a84b39989b9e042b4af3d6 (127.19.226.193:34345), e6d654389db14a49a1f1bf3dbe538a27 (127.19.226.195:39911)
I20250109 08:59:13.811033 25222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bf180fefb074fd192f6f80ae1460ba5" candidate_uuid: "9d26e8b39691431590fd77e182046c66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ff0106754a84b39989b9e042b4af3d6"
I20250109 08:59:13.811815 25222 raft_consensus.cc:3054] T 9bf180fefb074fd192f6f80ae1460ba5 P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:59:13.817960 25371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bf180fefb074fd192f6f80ae1460ba5" candidate_uuid: "9d26e8b39691431590fd77e182046c66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6d654389db14a49a1f1bf3dbe538a27"
I20250109 08:59:13.818732 25371 raft_consensus.cc:3054] T 9bf180fefb074fd192f6f80ae1460ba5 P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:59:13.821622 25509 tablet_bootstrap.cc:492] T 3b37da4b59cf41b8889527eb2b0793c9 P 9d26e8b39691431590fd77e182046c66: No bootstrap required, opened a new log
I20250109 08:59:13.822165 25509 ts_tablet_manager.cc:1397] T 3b37da4b59cf41b8889527eb2b0793c9 P 9d26e8b39691431590fd77e182046c66: Time spent bootstrapping tablet: real 0.020s user 0.010s sys 0.008s
I20250109 08:59:13.822250 25222 raft_consensus.cc:2463] T 9bf180fefb074fd192f6f80ae1460ba5 P 2ff0106754a84b39989b9e042b4af3d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d26e8b39691431590fd77e182046c66 in term 1.
I20250109 08:59:13.823791 25261 leader_election.cc:304] T 9bf180fefb074fd192f6f80ae1460ba5 P 9d26e8b39691431590fd77e182046c66 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ff0106754a84b39989b9e042b4af3d6, 9d26e8b39691431590fd77e182046c66; no voters:
I20250109 08:59:13.824679 25452 raft_consensus.cc:2798] T 9bf180fefb074fd192f6f80ae1460ba5 P 9d26e8b39691431590fd77e182046c66 [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:59:13.824672 25509 raft_consensus.cc:357] T 3b37da4b59cf41b8889527eb2b0793c9 P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:13.825361 25452 raft_consensus.cc:695] T 9bf180fefb074fd192f6f80ae1460ba5 P 9d26e8b39691431590fd77e182046c66 [term 1 LEADER]: Becoming Leader. State: Replica: 9d26e8b39691431590fd77e182046c66, State: Running, Role: LEADER
I20250109 08:59:13.825517 25509 raft_consensus.cc:383] T 3b37da4b59cf41b8889527eb2b0793c9 P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:59:13.825786 25509 raft_consensus.cc:738] T 3b37da4b59cf41b8889527eb2b0793c9 P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d26e8b39691431590fd77e182046c66, State: Initialized, Role: FOLLOWER
I20250109 08:59:13.826504 25532 raft_consensus.cc:491] T 9a3240fe2e094ba2bc002579bdb29e84 P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:59:13.826236 25452 consensus_queue.cc:237] T 9bf180fefb074fd192f6f80ae1460ba5 P 9d26e8b39691431590fd77e182046c66 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:13.826958 25532 raft_consensus.cc:513] T 9a3240fe2e094ba2bc002579bdb29e84 P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:13.828917 25532 leader_election.cc:290] T 9a3240fe2e094ba2bc002579bdb29e84 P 2ff0106754a84b39989b9e042b4af3d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9d26e8b39691431590fd77e182046c66 (127.19.226.194:33163), e6d654389db14a49a1f1bf3dbe538a27 (127.19.226.195:39911)
I20250109 08:59:13.829782 25295 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a3240fe2e094ba2bc002579bdb29e84" candidate_uuid: "2ff0106754a84b39989b9e042b4af3d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d26e8b39691431590fd77e182046c66" is_pre_election: true
I20250109 08:59:13.830293 25369 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a3240fe2e094ba2bc002579bdb29e84" candidate_uuid: "2ff0106754a84b39989b9e042b4af3d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6d654389db14a49a1f1bf3dbe538a27" is_pre_election: true
I20250109 08:59:13.830514 25295 raft_consensus.cc:2463] T 9a3240fe2e094ba2bc002579bdb29e84 P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ff0106754a84b39989b9e042b4af3d6 in term 0.
W20250109 08:59:13.831451 25188 leader_election.cc:343] T 9a3240fe2e094ba2bc002579bdb29e84 P 2ff0106754a84b39989b9e042b4af3d6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e6d654389db14a49a1f1bf3dbe538a27 (127.19.226.195:39911): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:59:13.831979 25187 leader_election.cc:304] T 9a3240fe2e094ba2bc002579bdb29e84 P 2ff0106754a84b39989b9e042b4af3d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2ff0106754a84b39989b9e042b4af3d6, 9d26e8b39691431590fd77e182046c66; no voters: e6d654389db14a49a1f1bf3dbe538a27
I20250109 08:59:13.832808 25532 raft_consensus.cc:2798] T 9a3240fe2e094ba2bc002579bdb29e84 P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:59:13.833148 25532 raft_consensus.cc:491] T 9a3240fe2e094ba2bc002579bdb29e84 P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:59:13.833432 25532 raft_consensus.cc:3054] T 9a3240fe2e094ba2bc002579bdb29e84 P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:59:13.838766 25371 raft_consensus.cc:2463] T 9bf180fefb074fd192f6f80ae1460ba5 P e6d654389db14a49a1f1bf3dbe538a27 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d26e8b39691431590fd77e182046c66 in term 1.
I20250109 08:59:13.839684 25509 consensus_queue.cc:260] T 3b37da4b59cf41b8889527eb2b0793c9 P 9d26e8b39691431590fd77e182046c66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:13.841425 25532 raft_consensus.cc:513] T 9a3240fe2e094ba2bc002579bdb29e84 P 2ff0106754a84b39989b9e042b4af3d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:13.842226 25509 ts_tablet_manager.cc:1428] T 3b37da4b59cf41b8889527eb2b0793c9 P 9d26e8b39691431590fd77e182046c66: Time spent starting tablet: real 0.020s user 0.006s sys 0.000s
I20250109 08:59:13.844475 25295 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a3240fe2e094ba2bc002579bdb29e84" candidate_uuid: "2ff0106754a84b39989b9e042b4af3d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d26e8b39691431590fd77e182046c66"
I20250109 08:59:13.844991 25371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a3240fe2e094ba2bc002579bdb29e84" candidate_uuid: "2ff0106754a84b39989b9e042b4af3d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6d654389db14a49a1f1bf3dbe538a27"
I20250109 08:59:13.845338 25295 raft_consensus.cc:3054] T 9a3240fe2e094ba2bc002579bdb29e84 P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:59:13.845070 25510 tablet_bootstrap.cc:492] T 9a3240fe2e094ba2bc002579bdb29e84 P e6d654389db14a49a1f1bf3dbe538a27: No bootstrap required, opened a new log
W20250109 08:59:13.846383 25188 leader_election.cc:343] T 9a3240fe2e094ba2bc002579bdb29e84 P 2ff0106754a84b39989b9e042b4af3d6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e6d654389db14a49a1f1bf3dbe538a27 (127.19.226.195:39911): Illegal state: must be running to vote when last-logged opid is not known
I20250109 08:59:13.844430 25532 leader_election.cc:290] T 9a3240fe2e094ba2bc002579bdb29e84 P 2ff0106754a84b39989b9e042b4af3d6 [CANDIDATE]: Term 1 election: Requested vote from peers 9d26e8b39691431590fd77e182046c66 (127.19.226.194:33163), e6d654389db14a49a1f1bf3dbe538a27 (127.19.226.195:39911)
I20250109 08:59:13.846987 25510 ts_tablet_manager.cc:1397] T 9a3240fe2e094ba2bc002579bdb29e84 P e6d654389db14a49a1f1bf3dbe538a27: Time spent bootstrapping tablet: real 0.068s user 0.021s sys 0.015s
I20250109 08:59:13.850306 25510 raft_consensus.cc:357] T 9a3240fe2e094ba2bc002579bdb29e84 P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:13.851004 25510 raft_consensus.cc:383] T 9a3240fe2e094ba2bc002579bdb29e84 P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:59:13.851293 25510 raft_consensus.cc:738] T 9a3240fe2e094ba2bc002579bdb29e84 P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6d654389db14a49a1f1bf3dbe538a27, State: Initialized, Role: FOLLOWER
I20250109 08:59:13.853567 25295 raft_consensus.cc:2463] T 9a3240fe2e094ba2bc002579bdb29e84 P 9d26e8b39691431590fd77e182046c66 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ff0106754a84b39989b9e042b4af3d6 in term 1.
I20250109 08:59:13.851926 25510 consensus_queue.cc:260] T 9a3240fe2e094ba2bc002579bdb29e84 P e6d654389db14a49a1f1bf3dbe538a27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:13.855377 25187 leader_election.cc:304] T 9a3240fe2e094ba2bc002579bdb29e84 P 2ff0106754a84b39989b9e042b4af3d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2ff0106754a84b39989b9e042b4af3d6, 9d26e8b39691431590fd77e182046c66; no voters: e6d654389db14a49a1f1bf3dbe538a27
I20250109 08:59:13.856235 25511 tablet_bootstrap.cc:492] T 3f4e7967061b4173bd00a9539a371fe3 P 2ff0106754a84b39989b9e042b4af3d6: No bootstrap required, opened a new log
I20250109 08:59:13.856590 25424 raft_consensus.cc:2798] T 9a3240fe2e094ba2bc002579bdb29e84 P 2ff0106754a84b39989b9e042b4af3d6 [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:59:13.856808 25511 ts_tablet_manager.cc:1397] T 3f4e7967061b4173bd00a9539a371fe3 P 2ff0106754a84b39989b9e042b4af3d6: Time spent bootstrapping tablet: real 0.073s user 0.026s sys 0.012s
I20250109 08:59:13.857100 25424 raft_consensus.cc:695] T 9a3240fe2e094ba2bc002579bdb29e84 P 2ff0106754a84b39989b9e042b4af3d6 [term 1 LEADER]: Becoming Leader. State: Replica: 2ff0106754a84b39989b9e042b4af3d6, State: Running, Role: LEADER
I20250109 08:59:13.857915 25424 consensus_queue.cc:237] T 9a3240fe2e094ba2bc002579bdb29e84 P 2ff0106754a84b39989b9e042b4af3d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:13.859440 25511 raft_consensus.cc:357] T 3f4e7967061b4173bd00a9539a371fe3 P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:13.860174 25511 raft_consensus.cc:383] T 3f4e7967061b4173bd00a9539a371fe3 P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:59:13.860459 25511 raft_consensus.cc:738] T 3f4e7967061b4173bd00a9539a371fe3 P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ff0106754a84b39989b9e042b4af3d6, State: Initialized, Role: FOLLOWER
I20250109 08:59:13.861069 25511 consensus_queue.cc:260] T 3f4e7967061b4173bd00a9539a371fe3 P 2ff0106754a84b39989b9e042b4af3d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:13.861531 25510 ts_tablet_manager.cc:1428] T 9a3240fe2e094ba2bc002579bdb29e84 P e6d654389db14a49a1f1bf3dbe538a27: Time spent starting tablet: real 0.014s user 0.007s sys 0.000s
I20250109 08:59:13.862820 25510 tablet_bootstrap.cc:492] T ed0e3d98ad79424caf0e24ecf3b5f295 P e6d654389db14a49a1f1bf3dbe538a27: Bootstrap starting.
I20250109 08:59:13.863222 25511 ts_tablet_manager.cc:1428] T 3f4e7967061b4173bd00a9539a371fe3 P 2ff0106754a84b39989b9e042b4af3d6: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20250109 08:59:13.864406 25511 tablet_bootstrap.cc:492] T 346220ee65da44b8862c9f28d7aa6aa3 P 2ff0106754a84b39989b9e042b4af3d6: Bootstrap starting.
I20250109 08:59:13.870256 25511 tablet_bootstrap.cc:654] T 346220ee65da44b8862c9f28d7aa6aa3 P 2ff0106754a84b39989b9e042b4af3d6: Neither blocks nor log segments found. Creating new log.
I20250109 08:59:13.870291 25532 raft_consensus.cc:491] T 3f4e7967061b4173bd00a9539a371fe3 P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:59:13.870895 25532 raft_consensus.cc:513] T 3f4e7967061b4173bd00a9539a371fe3 P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:13.872877 25532 leader_election.cc:290] T 3f4e7967061b4173bd00a9539a371fe3 P 2ff0106754a84b39989b9e042b4af3d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9d26e8b39691431590fd77e182046c66 (127.19.226.194:33163), e6d654389db14a49a1f1bf3dbe538a27 (127.19.226.195:39911)
I20250109 08:59:13.873723 25295 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f4e7967061b4173bd00a9539a371fe3" candidate_uuid: "2ff0106754a84b39989b9e042b4af3d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d26e8b39691431590fd77e182046c66" is_pre_election: true
I20250109 08:59:13.874127 25510 tablet_bootstrap.cc:654] T ed0e3d98ad79424caf0e24ecf3b5f295 P e6d654389db14a49a1f1bf3dbe538a27: Neither blocks nor log segments found. Creating new log.
I20250109 08:59:13.874511 25295 raft_consensus.cc:2463] T 3f4e7967061b4173bd00a9539a371fe3 P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ff0106754a84b39989b9e042b4af3d6 in term 0.
I20250109 08:59:13.876144 25187 leader_election.cc:304] T 3f4e7967061b4173bd00a9539a371fe3 P 2ff0106754a84b39989b9e042b4af3d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ff0106754a84b39989b9e042b4af3d6, 9d26e8b39691431590fd77e182046c66; no voters:
I20250109 08:59:13.876996 25532 raft_consensus.cc:2798] T 3f4e7967061b4173bd00a9539a371fe3 P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:59:13.877417 25532 raft_consensus.cc:491] T 3f4e7967061b4173bd00a9539a371fe3 P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:59:13.877745 25532 raft_consensus.cc:3054] T 3f4e7967061b4173bd00a9539a371fe3 P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:59:13.884792 25532 raft_consensus.cc:513] T 3f4e7967061b4173bd00a9539a371fe3 P 2ff0106754a84b39989b9e042b4af3d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:13.885910 25371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f4e7967061b4173bd00a9539a371fe3" candidate_uuid: "2ff0106754a84b39989b9e042b4af3d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6d654389db14a49a1f1bf3dbe538a27" is_pre_election: true
I20250109 08:59:13.886754 25532 leader_election.cc:290] T 3f4e7967061b4173bd00a9539a371fe3 P 2ff0106754a84b39989b9e042b4af3d6 [CANDIDATE]: Term 1 election: Requested vote from peers 9d26e8b39691431590fd77e182046c66 (127.19.226.194:33163), e6d654389db14a49a1f1bf3dbe538a27 (127.19.226.195:39911)
I20250109 08:59:13.886658 25371 raft_consensus.cc:2463] T 3f4e7967061b4173bd00a9539a371fe3 P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ff0106754a84b39989b9e042b4af3d6 in term 0.
I20250109 08:59:13.887890 25369 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f4e7967061b4173bd00a9539a371fe3" candidate_uuid: "2ff0106754a84b39989b9e042b4af3d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6d654389db14a49a1f1bf3dbe538a27"
I20250109 08:59:13.888901 25369 raft_consensus.cc:3054] T 3f4e7967061b4173bd00a9539a371fe3 P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:59:13.896046 25510 tablet_bootstrap.cc:492] T ed0e3d98ad79424caf0e24ecf3b5f295 P e6d654389db14a49a1f1bf3dbe538a27: No bootstrap required, opened a new log
I20250109 08:59:13.896620 25510 ts_tablet_manager.cc:1397] T ed0e3d98ad79424caf0e24ecf3b5f295 P e6d654389db14a49a1f1bf3dbe538a27: Time spent bootstrapping tablet: real 0.034s user 0.014s sys 0.003s
I20250109 08:59:13.887830 25295 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f4e7967061b4173bd00a9539a371fe3" candidate_uuid: "2ff0106754a84b39989b9e042b4af3d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d26e8b39691431590fd77e182046c66"
I20250109 08:59:13.897158 25295 raft_consensus.cc:3054] T 3f4e7967061b4173bd00a9539a371fe3 P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:59:13.899396 25510 raft_consensus.cc:357] T ed0e3d98ad79424caf0e24ecf3b5f295 P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:13.900147 25510 raft_consensus.cc:383] T ed0e3d98ad79424caf0e24ecf3b5f295 P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:59:13.900435 25510 raft_consensus.cc:738] T ed0e3d98ad79424caf0e24ecf3b5f295 P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6d654389db14a49a1f1bf3dbe538a27, State: Initialized, Role: FOLLOWER
I20250109 08:59:13.901064 25510 consensus_queue.cc:260] T ed0e3d98ad79424caf0e24ecf3b5f295 P e6d654389db14a49a1f1bf3dbe538a27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:13.901578 25511 tablet_bootstrap.cc:492] T 346220ee65da44b8862c9f28d7aa6aa3 P 2ff0106754a84b39989b9e042b4af3d6: No bootstrap required, opened a new log
I20250109 08:59:13.900853 25117 catalog_manager.cc:5526] T 9bf180fefb074fd192f6f80ae1460ba5 P 9d26e8b39691431590fd77e182046c66 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9d26e8b39691431590fd77e182046c66 (127.19.226.194). New cstate: current_term: 1 leader_uuid: "9d26e8b39691431590fd77e182046c66" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } health_report { overall_health: UNKNOWN } } }
I20250109 08:59:13.902089 25511 ts_tablet_manager.cc:1397] T 346220ee65da44b8862c9f28d7aa6aa3 P 2ff0106754a84b39989b9e042b4af3d6: Time spent bootstrapping tablet: real 0.038s user 0.024s sys 0.012s
I20250109 08:59:13.904541 25510 ts_tablet_manager.cc:1428] T ed0e3d98ad79424caf0e24ecf3b5f295 P e6d654389db14a49a1f1bf3dbe538a27: Time spent starting tablet: real 0.008s user 0.003s sys 0.004s
I20250109 08:59:13.905480 25510 tablet_bootstrap.cc:492] T 51277cd5f9374666bc60232309efc100 P e6d654389db14a49a1f1bf3dbe538a27: Bootstrap starting.
I20250109 08:59:13.906240 25532 raft_consensus.cc:491] T ed0e3d98ad79424caf0e24ecf3b5f295 P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:59:13.906714 25532 raft_consensus.cc:513] T ed0e3d98ad79424caf0e24ecf3b5f295 P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:13.907037 25511 raft_consensus.cc:357] T 346220ee65da44b8862c9f28d7aa6aa3 P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:13.907728 25511 raft_consensus.cc:383] T 346220ee65da44b8862c9f28d7aa6aa3 P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:59:13.907984 25511 raft_consensus.cc:738] T 346220ee65da44b8862c9f28d7aa6aa3 P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ff0106754a84b39989b9e042b4af3d6, State: Initialized, Role: FOLLOWER
I20250109 08:59:13.908721 25532 leader_election.cc:290] T ed0e3d98ad79424caf0e24ecf3b5f295 P 2ff0106754a84b39989b9e042b4af3d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9d26e8b39691431590fd77e182046c66 (127.19.226.194:33163), e6d654389db14a49a1f1bf3dbe538a27 (127.19.226.195:39911)
I20250109 08:59:13.908594 25511 consensus_queue.cc:260] T 346220ee65da44b8862c9f28d7aa6aa3 P 2ff0106754a84b39989b9e042b4af3d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:13.908458 25119 catalog_manager.cc:5526] T 9a3240fe2e094ba2bc002579bdb29e84 P 2ff0106754a84b39989b9e042b4af3d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2ff0106754a84b39989b9e042b4af3d6 (127.19.226.193). New cstate: current_term: 1 leader_uuid: "2ff0106754a84b39989b9e042b4af3d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } health_report { overall_health: UNKNOWN } } }
I20250109 08:59:13.909551 25294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed0e3d98ad79424caf0e24ecf3b5f295" candidate_uuid: "2ff0106754a84b39989b9e042b4af3d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d26e8b39691431590fd77e182046c66" is_pre_election: true
I20250109 08:59:13.910197 25294 raft_consensus.cc:2463] T ed0e3d98ad79424caf0e24ecf3b5f295 P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ff0106754a84b39989b9e042b4af3d6 in term 0.
I20250109 08:59:13.910840 25511 ts_tablet_manager.cc:1428] T 346220ee65da44b8862c9f28d7aa6aa3 P 2ff0106754a84b39989b9e042b4af3d6: Time spent starting tablet: real 0.008s user 0.003s sys 0.003s
I20250109 08:59:13.911322 25187 leader_election.cc:304] T ed0e3d98ad79424caf0e24ecf3b5f295 P 2ff0106754a84b39989b9e042b4af3d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ff0106754a84b39989b9e042b4af3d6, 9d26e8b39691431590fd77e182046c66; no voters:
I20250109 08:59:13.912281 25532 raft_consensus.cc:2798] T ed0e3d98ad79424caf0e24ecf3b5f295 P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:59:13.912719 25532 raft_consensus.cc:491] T ed0e3d98ad79424caf0e24ecf3b5f295 P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:59:13.913069 25532 raft_consensus.cc:3054] T ed0e3d98ad79424caf0e24ecf3b5f295 P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:59:13.912431 25371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed0e3d98ad79424caf0e24ecf3b5f295" candidate_uuid: "2ff0106754a84b39989b9e042b4af3d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6d654389db14a49a1f1bf3dbe538a27" is_pre_election: true
I20250109 08:59:13.913650 25371 raft_consensus.cc:2463] T ed0e3d98ad79424caf0e24ecf3b5f295 P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ff0106754a84b39989b9e042b4af3d6 in term 0.
I20250109 08:59:13.914959 25510 tablet_bootstrap.cc:654] T 51277cd5f9374666bc60232309efc100 P e6d654389db14a49a1f1bf3dbe538a27: Neither blocks nor log segments found. Creating new log.
I20250109 08:59:13.914489 25369 raft_consensus.cc:2463] T 3f4e7967061b4173bd00a9539a371fe3 P e6d654389db14a49a1f1bf3dbe538a27 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ff0106754a84b39989b9e042b4af3d6 in term 1.
I20250109 08:59:13.916196 25188 leader_election.cc:304] T 3f4e7967061b4173bd00a9539a371fe3 P 2ff0106754a84b39989b9e042b4af3d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ff0106754a84b39989b9e042b4af3d6, e6d654389db14a49a1f1bf3dbe538a27; no voters:
I20250109 08:59:13.918663 25295 raft_consensus.cc:2463] T 3f4e7967061b4173bd00a9539a371fe3 P 9d26e8b39691431590fd77e182046c66 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ff0106754a84b39989b9e042b4af3d6 in term 1.
I20250109 08:59:13.920281 25532 raft_consensus.cc:513] T ed0e3d98ad79424caf0e24ecf3b5f295 P 2ff0106754a84b39989b9e042b4af3d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:13.922523 25532 leader_election.cc:290] T ed0e3d98ad79424caf0e24ecf3b5f295 P 2ff0106754a84b39989b9e042b4af3d6 [CANDIDATE]: Term 1 election: Requested vote from peers 9d26e8b39691431590fd77e182046c66 (127.19.226.194:33163), e6d654389db14a49a1f1bf3dbe538a27 (127.19.226.195:39911)
I20250109 08:59:13.923080 25532 raft_consensus.cc:2798] T 3f4e7967061b4173bd00a9539a371fe3 P 2ff0106754a84b39989b9e042b4af3d6 [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:59:13.923538 25532 raft_consensus.cc:695] T 3f4e7967061b4173bd00a9539a371fe3 P 2ff0106754a84b39989b9e042b4af3d6 [term 1 LEADER]: Becoming Leader. State: Replica: 2ff0106754a84b39989b9e042b4af3d6, State: Running, Role: LEADER
I20250109 08:59:13.923547 25369 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed0e3d98ad79424caf0e24ecf3b5f295" candidate_uuid: "2ff0106754a84b39989b9e042b4af3d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6d654389db14a49a1f1bf3dbe538a27"
I20250109 08:59:13.924253 25369 raft_consensus.cc:3054] T ed0e3d98ad79424caf0e24ecf3b5f295 P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:59:13.924522 25532 consensus_queue.cc:237] T 3f4e7967061b4173bd00a9539a371fe3 P 2ff0106754a84b39989b9e042b4af3d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:13.931875 25369 raft_consensus.cc:2463] T ed0e3d98ad79424caf0e24ecf3b5f295 P e6d654389db14a49a1f1bf3dbe538a27 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ff0106754a84b39989b9e042b4af3d6 in term 1.
I20250109 08:59:13.936930 25188 leader_election.cc:304] T ed0e3d98ad79424caf0e24ecf3b5f295 P 2ff0106754a84b39989b9e042b4af3d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ff0106754a84b39989b9e042b4af3d6, e6d654389db14a49a1f1bf3dbe538a27; no voters:
I20250109 08:59:13.942240 25532 raft_consensus.cc:2798] T ed0e3d98ad79424caf0e24ecf3b5f295 P 2ff0106754a84b39989b9e042b4af3d6 [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:59:13.943681 25532 raft_consensus.cc:695] T ed0e3d98ad79424caf0e24ecf3b5f295 P 2ff0106754a84b39989b9e042b4af3d6 [term 1 LEADER]: Becoming Leader. State: Replica: 2ff0106754a84b39989b9e042b4af3d6, State: Running, Role: LEADER
I20250109 08:59:13.944561 25532 consensus_queue.cc:237] T ed0e3d98ad79424caf0e24ecf3b5f295 P 2ff0106754a84b39989b9e042b4af3d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:13.942203 25295 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed0e3d98ad79424caf0e24ecf3b5f295" candidate_uuid: "2ff0106754a84b39989b9e042b4af3d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d26e8b39691431590fd77e182046c66"
I20250109 08:59:13.947248 25295 raft_consensus.cc:3054] T ed0e3d98ad79424caf0e24ecf3b5f295 P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:59:13.948884 25119 catalog_manager.cc:5526] T 3f4e7967061b4173bd00a9539a371fe3 P 2ff0106754a84b39989b9e042b4af3d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2ff0106754a84b39989b9e042b4af3d6 (127.19.226.193). New cstate: current_term: 1 leader_uuid: "2ff0106754a84b39989b9e042b4af3d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } health_report { overall_health: UNKNOWN } } }
I20250109 08:59:13.953388 25510 tablet_bootstrap.cc:492] T 51277cd5f9374666bc60232309efc100 P e6d654389db14a49a1f1bf3dbe538a27: No bootstrap required, opened a new log
I20250109 08:59:13.953940 25510 ts_tablet_manager.cc:1397] T 51277cd5f9374666bc60232309efc100 P e6d654389db14a49a1f1bf3dbe538a27: Time spent bootstrapping tablet: real 0.049s user 0.013s sys 0.020s
I20250109 08:59:13.957943 25510 raft_consensus.cc:357] T 51277cd5f9374666bc60232309efc100 P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:13.958637 25510 raft_consensus.cc:383] T 51277cd5f9374666bc60232309efc100 P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250109 08:59:13.958897 25510 raft_consensus.cc:738] T 51277cd5f9374666bc60232309efc100 P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6d654389db14a49a1f1bf3dbe538a27, State: Initialized, Role: FOLLOWER
I20250109 08:59:13.959494 25510 consensus_queue.cc:260] T 51277cd5f9374666bc60232309efc100 P e6d654389db14a49a1f1bf3dbe538a27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:13.961449 25510 ts_tablet_manager.cc:1428] T 51277cd5f9374666bc60232309efc100 P e6d654389db14a49a1f1bf3dbe538a27: Time spent starting tablet: real 0.007s user 0.003s sys 0.004s
I20250109 08:59:13.964834 25295 raft_consensus.cc:2463] T ed0e3d98ad79424caf0e24ecf3b5f295 P 9d26e8b39691431590fd77e182046c66 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ff0106754a84b39989b9e042b4af3d6 in term 1.
I20250109 08:59:13.966744 25450 raft_consensus.cc:491] T 51277cd5f9374666bc60232309efc100 P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:59:13.967273 25450 raft_consensus.cc:513] T 51277cd5f9374666bc60232309efc100 P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:13.967180 25119 catalog_manager.cc:5526] T ed0e3d98ad79424caf0e24ecf3b5f295 P 2ff0106754a84b39989b9e042b4af3d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2ff0106754a84b39989b9e042b4af3d6 (127.19.226.193). New cstate: current_term: 1 leader_uuid: "2ff0106754a84b39989b9e042b4af3d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } health_report { overall_health: UNKNOWN } } }
I20250109 08:59:13.969594 25450 leader_election.cc:290] T 51277cd5f9374666bc60232309efc100 P e6d654389db14a49a1f1bf3dbe538a27 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2ff0106754a84b39989b9e042b4af3d6 (127.19.226.193:34345), 9d26e8b39691431590fd77e182046c66 (127.19.226.194:33163)
I20250109 08:59:13.970129 25222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51277cd5f9374666bc60232309efc100" candidate_uuid: "e6d654389db14a49a1f1bf3dbe538a27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ff0106754a84b39989b9e042b4af3d6" is_pre_election: true
I20250109 08:59:13.970832 25222 raft_consensus.cc:2463] T 51277cd5f9374666bc60232309efc100 P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6d654389db14a49a1f1bf3dbe538a27 in term 0.
I20250109 08:59:13.971791 25335 leader_election.cc:304] T 51277cd5f9374666bc60232309efc100 P e6d654389db14a49a1f1bf3dbe538a27 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ff0106754a84b39989b9e042b4af3d6, e6d654389db14a49a1f1bf3dbe538a27; no voters:
I20250109 08:59:13.972570 25450 raft_consensus.cc:2798] T 51277cd5f9374666bc60232309efc100 P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:59:13.972956 25450 raft_consensus.cc:491] T 51277cd5f9374666bc60232309efc100 P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:59:13.973277 25450 raft_consensus.cc:3054] T 51277cd5f9374666bc60232309efc100 P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:59:13.973629 25295 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51277cd5f9374666bc60232309efc100" candidate_uuid: "e6d654389db14a49a1f1bf3dbe538a27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d26e8b39691431590fd77e182046c66" is_pre_election: true
I20250109 08:59:13.974342 25295 raft_consensus.cc:2463] T 51277cd5f9374666bc60232309efc100 P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6d654389db14a49a1f1bf3dbe538a27 in term 0.
I20250109 08:59:13.979127 25450 raft_consensus.cc:513] T 51277cd5f9374666bc60232309efc100 P e6d654389db14a49a1f1bf3dbe538a27 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:13.981525 25222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51277cd5f9374666bc60232309efc100" candidate_uuid: "e6d654389db14a49a1f1bf3dbe538a27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ff0106754a84b39989b9e042b4af3d6"
I20250109 08:59:13.981262 25450 leader_election.cc:290] T 51277cd5f9374666bc60232309efc100 P e6d654389db14a49a1f1bf3dbe538a27 [CANDIDATE]: Term 1 election: Requested vote from peers 2ff0106754a84b39989b9e042b4af3d6 (127.19.226.193:34345), 9d26e8b39691431590fd77e182046c66 (127.19.226.194:33163)
I20250109 08:59:13.981982 25295 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51277cd5f9374666bc60232309efc100" candidate_uuid: "e6d654389db14a49a1f1bf3dbe538a27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d26e8b39691431590fd77e182046c66"
I20250109 08:59:13.982301 25222 raft_consensus.cc:3054] T 51277cd5f9374666bc60232309efc100 P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:59:13.984578 25295 raft_consensus.cc:3054] T 51277cd5f9374666bc60232309efc100 P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:59:13.988755 25222 raft_consensus.cc:2463] T 51277cd5f9374666bc60232309efc100 P 2ff0106754a84b39989b9e042b4af3d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6d654389db14a49a1f1bf3dbe538a27 in term 1.
I20250109 08:59:13.989341 25295 raft_consensus.cc:2463] T 51277cd5f9374666bc60232309efc100 P 9d26e8b39691431590fd77e182046c66 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6d654389db14a49a1f1bf3dbe538a27 in term 1.
I20250109 08:59:13.989833 25335 leader_election.cc:304] T 51277cd5f9374666bc60232309efc100 P e6d654389db14a49a1f1bf3dbe538a27 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ff0106754a84b39989b9e042b4af3d6, e6d654389db14a49a1f1bf3dbe538a27; no voters:
I20250109 08:59:13.990635 25450 raft_consensus.cc:2798] T 51277cd5f9374666bc60232309efc100 P e6d654389db14a49a1f1bf3dbe538a27 [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:59:13.991068 25450 raft_consensus.cc:695] T 51277cd5f9374666bc60232309efc100 P e6d654389db14a49a1f1bf3dbe538a27 [term 1 LEADER]: Becoming Leader. State: Replica: e6d654389db14a49a1f1bf3dbe538a27, State: Running, Role: LEADER
I20250109 08:59:13.991828 25450 consensus_queue.cc:237] T 51277cd5f9374666bc60232309efc100 P e6d654389db14a49a1f1bf3dbe538a27 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:13.992991 25531 raft_consensus.cc:491] T cb8ed58651e74cbabff2b027ba4a9a1e P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:59:13.993467 25531 raft_consensus.cc:513] T cb8ed58651e74cbabff2b027ba4a9a1e P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:13.995211 25531 leader_election.cc:290] T cb8ed58651e74cbabff2b027ba4a9a1e P e6d654389db14a49a1f1bf3dbe538a27 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2ff0106754a84b39989b9e042b4af3d6 (127.19.226.193:34345), 9d26e8b39691431590fd77e182046c66 (127.19.226.194:33163)
I20250109 08:59:13.995957 25222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8ed58651e74cbabff2b027ba4a9a1e" candidate_uuid: "e6d654389db14a49a1f1bf3dbe538a27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ff0106754a84b39989b9e042b4af3d6" is_pre_election: true
I20250109 08:59:13.996255 25295 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8ed58651e74cbabff2b027ba4a9a1e" candidate_uuid: "e6d654389db14a49a1f1bf3dbe538a27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d26e8b39691431590fd77e182046c66" is_pre_election: true
I20250109 08:59:13.996788 25222 raft_consensus.cc:2463] T cb8ed58651e74cbabff2b027ba4a9a1e P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6d654389db14a49a1f1bf3dbe538a27 in term 0.
I20250109 08:59:13.997022 25295 raft_consensus.cc:2463] T cb8ed58651e74cbabff2b027ba4a9a1e P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6d654389db14a49a1f1bf3dbe538a27 in term 0.
I20250109 08:59:13.997812 25335 leader_election.cc:304] T cb8ed58651e74cbabff2b027ba4a9a1e P e6d654389db14a49a1f1bf3dbe538a27 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ff0106754a84b39989b9e042b4af3d6, e6d654389db14a49a1f1bf3dbe538a27; no voters:
I20250109 08:59:13.998556 25531 raft_consensus.cc:2798] T cb8ed58651e74cbabff2b027ba4a9a1e P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:59:13.998894 25531 raft_consensus.cc:491] T cb8ed58651e74cbabff2b027ba4a9a1e P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:59:13.999212 25531 raft_consensus.cc:3054] T cb8ed58651e74cbabff2b027ba4a9a1e P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:59:14.000294 25119 catalog_manager.cc:5526] T 51277cd5f9374666bc60232309efc100 P e6d654389db14a49a1f1bf3dbe538a27 reported cstate change: term changed from 0 to 1, leader changed from <none> to e6d654389db14a49a1f1bf3dbe538a27 (127.19.226.195). New cstate: current_term: 1 leader_uuid: "e6d654389db14a49a1f1bf3dbe538a27" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } health_report { overall_health: HEALTHY } } }
I20250109 08:59:14.005419 25531 raft_consensus.cc:513] T cb8ed58651e74cbabff2b027ba4a9a1e P e6d654389db14a49a1f1bf3dbe538a27 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:14.007520 25531 leader_election.cc:290] T cb8ed58651e74cbabff2b027ba4a9a1e P e6d654389db14a49a1f1bf3dbe538a27 [CANDIDATE]: Term 1 election: Requested vote from peers 2ff0106754a84b39989b9e042b4af3d6 (127.19.226.193:34345), 9d26e8b39691431590fd77e182046c66 (127.19.226.194:33163)
I20250109 08:59:14.008245 25222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8ed58651e74cbabff2b027ba4a9a1e" candidate_uuid: "e6d654389db14a49a1f1bf3dbe538a27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ff0106754a84b39989b9e042b4af3d6"
I20250109 08:59:14.008725 25295 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8ed58651e74cbabff2b027ba4a9a1e" candidate_uuid: "e6d654389db14a49a1f1bf3dbe538a27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d26e8b39691431590fd77e182046c66"
I20250109 08:59:14.008973 25222 raft_consensus.cc:3054] T cb8ed58651e74cbabff2b027ba4a9a1e P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:59:14.009377 25295 raft_consensus.cc:3054] T cb8ed58651e74cbabff2b027ba4a9a1e P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:59:14.015959 25222 raft_consensus.cc:2463] T cb8ed58651e74cbabff2b027ba4a9a1e P 2ff0106754a84b39989b9e042b4af3d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6d654389db14a49a1f1bf3dbe538a27 in term 1.
I20250109 08:59:14.015959 25295 raft_consensus.cc:2463] T cb8ed58651e74cbabff2b027ba4a9a1e P 9d26e8b39691431590fd77e182046c66 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6d654389db14a49a1f1bf3dbe538a27 in term 1.
I20250109 08:59:14.017246 25336 leader_election.cc:304] T cb8ed58651e74cbabff2b027ba4a9a1e P e6d654389db14a49a1f1bf3dbe538a27 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d26e8b39691431590fd77e182046c66, e6d654389db14a49a1f1bf3dbe538a27; no voters:
I20250109 08:59:14.017876 25531 raft_consensus.cc:2798] T cb8ed58651e74cbabff2b027ba4a9a1e P e6d654389db14a49a1f1bf3dbe538a27 [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:59:14.018280 25531 raft_consensus.cc:695] T cb8ed58651e74cbabff2b027ba4a9a1e P e6d654389db14a49a1f1bf3dbe538a27 [term 1 LEADER]: Becoming Leader. State: Replica: e6d654389db14a49a1f1bf3dbe538a27, State: Running, Role: LEADER
I20250109 08:59:14.018860 25531 consensus_queue.cc:237] T cb8ed58651e74cbabff2b027ba4a9a1e P e6d654389db14a49a1f1bf3dbe538a27 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:14.026481 25119 catalog_manager.cc:5526] T cb8ed58651e74cbabff2b027ba4a9a1e P e6d654389db14a49a1f1bf3dbe538a27 reported cstate change: term changed from 0 to 1, leader changed from <none> to e6d654389db14a49a1f1bf3dbe538a27 (127.19.226.195). New cstate: current_term: 1 leader_uuid: "e6d654389db14a49a1f1bf3dbe538a27" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } health_report { overall_health: HEALTHY } } }
I20250109 08:59:14.091425 25531 raft_consensus.cc:491] T 346220ee65da44b8862c9f28d7aa6aa3 P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:59:14.091881 25531 raft_consensus.cc:513] T 346220ee65da44b8862c9f28d7aa6aa3 P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:14.093477 25531 leader_election.cc:290] T 346220ee65da44b8862c9f28d7aa6aa3 P e6d654389db14a49a1f1bf3dbe538a27 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2ff0106754a84b39989b9e042b4af3d6 (127.19.226.193:34345), 9d26e8b39691431590fd77e182046c66 (127.19.226.194:33163)
I20250109 08:59:14.094283 25222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "346220ee65da44b8862c9f28d7aa6aa3" candidate_uuid: "e6d654389db14a49a1f1bf3dbe538a27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ff0106754a84b39989b9e042b4af3d6" is_pre_election: true
I20250109 08:59:14.094396 25295 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "346220ee65da44b8862c9f28d7aa6aa3" candidate_uuid: "e6d654389db14a49a1f1bf3dbe538a27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d26e8b39691431590fd77e182046c66" is_pre_election: true
I20250109 08:59:14.094959 25222 raft_consensus.cc:2463] T 346220ee65da44b8862c9f28d7aa6aa3 P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6d654389db14a49a1f1bf3dbe538a27 in term 0.
I20250109 08:59:14.095027 25295 raft_consensus.cc:2463] T 346220ee65da44b8862c9f28d7aa6aa3 P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6d654389db14a49a1f1bf3dbe538a27 in term 0.
I20250109 08:59:14.096035 25335 leader_election.cc:304] T 346220ee65da44b8862c9f28d7aa6aa3 P e6d654389db14a49a1f1bf3dbe538a27 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ff0106754a84b39989b9e042b4af3d6, e6d654389db14a49a1f1bf3dbe538a27; no voters:
I20250109 08:59:14.096693 25531 raft_consensus.cc:2798] T 346220ee65da44b8862c9f28d7aa6aa3 P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250109 08:59:14.097024 25531 raft_consensus.cc:491] T 346220ee65da44b8862c9f28d7aa6aa3 P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:59:14.097278 25531 raft_consensus.cc:3054] T 346220ee65da44b8862c9f28d7aa6aa3 P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:59:14.101809 25531 raft_consensus.cc:513] T 346220ee65da44b8862c9f28d7aa6aa3 P e6d654389db14a49a1f1bf3dbe538a27 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:14.103329 25531 leader_election.cc:290] T 346220ee65da44b8862c9f28d7aa6aa3 P e6d654389db14a49a1f1bf3dbe538a27 [CANDIDATE]: Term 1 election: Requested vote from peers 2ff0106754a84b39989b9e042b4af3d6 (127.19.226.193:34345), 9d26e8b39691431590fd77e182046c66 (127.19.226.194:33163)
I20250109 08:59:14.104034 25222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "346220ee65da44b8862c9f28d7aa6aa3" candidate_uuid: "e6d654389db14a49a1f1bf3dbe538a27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ff0106754a84b39989b9e042b4af3d6"
I20250109 08:59:14.104223 25295 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "346220ee65da44b8862c9f28d7aa6aa3" candidate_uuid: "e6d654389db14a49a1f1bf3dbe538a27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d26e8b39691431590fd77e182046c66"
I20250109 08:59:14.104656 25222 raft_consensus.cc:3054] T 346220ee65da44b8862c9f28d7aa6aa3 P 2ff0106754a84b39989b9e042b4af3d6 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:59:14.104753 25295 raft_consensus.cc:3054] T 346220ee65da44b8862c9f28d7aa6aa3 P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:59:14.108925 25295 raft_consensus.cc:2463] T 346220ee65da44b8862c9f28d7aa6aa3 P 9d26e8b39691431590fd77e182046c66 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6d654389db14a49a1f1bf3dbe538a27 in term 1.
I20250109 08:59:14.109308 25222 raft_consensus.cc:2463] T 346220ee65da44b8862c9f28d7aa6aa3 P 2ff0106754a84b39989b9e042b4af3d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6d654389db14a49a1f1bf3dbe538a27 in term 1.
I20250109 08:59:14.109803 25336 leader_election.cc:304] T 346220ee65da44b8862c9f28d7aa6aa3 P e6d654389db14a49a1f1bf3dbe538a27 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d26e8b39691431590fd77e182046c66, e6d654389db14a49a1f1bf3dbe538a27; no voters:
I20250109 08:59:14.110641 25531 raft_consensus.cc:2798] T 346220ee65da44b8862c9f28d7aa6aa3 P e6d654389db14a49a1f1bf3dbe538a27 [term 1 FOLLOWER]: Leader election won for term 1
I20250109 08:59:14.111032 25531 raft_consensus.cc:695] T 346220ee65da44b8862c9f28d7aa6aa3 P e6d654389db14a49a1f1bf3dbe538a27 [term 1 LEADER]: Becoming Leader. State: Replica: e6d654389db14a49a1f1bf3dbe538a27, State: Running, Role: LEADER
I20250109 08:59:14.111734 25531 consensus_queue.cc:237] T 346220ee65da44b8862c9f28d7aa6aa3 P e6d654389db14a49a1f1bf3dbe538a27 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:14.119263 25119 catalog_manager.cc:5526] T 346220ee65da44b8862c9f28d7aa6aa3 P e6d654389db14a49a1f1bf3dbe538a27 reported cstate change: term changed from 0 to 1, leader changed from <none> to e6d654389db14a49a1f1bf3dbe538a27 (127.19.226.195). New cstate: current_term: 1 leader_uuid: "e6d654389db14a49a1f1bf3dbe538a27" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } health_report { overall_health: HEALTHY } } }
I20250109 08:59:14.127115 25452 raft_consensus.cc:491] T 3b37da4b59cf41b8889527eb2b0793c9 P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:59:14.127590 25452 raft_consensus.cc:513] T 3b37da4b59cf41b8889527eb2b0793c9 P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:14.129135 25452 leader_election.cc:290] T 3b37da4b59cf41b8889527eb2b0793c9 P 9d26e8b39691431590fd77e182046c66 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2ff0106754a84b39989b9e042b4af3d6 (127.19.226.193:34345), e6d654389db14a49a1f1bf3dbe538a27 (127.19.226.195:39911)
I20250109 08:59:14.129767 25222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b37da4b59cf41b8889527eb2b0793c9" candidate_uuid: "9d26e8b39691431590fd77e182046c66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ff0106754a84b39989b9e042b4af3d6" is_pre_election: true
I20250109 08:59:14.130082 25369 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b37da4b59cf41b8889527eb2b0793c9" candidate_uuid: "9d26e8b39691431590fd77e182046c66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6d654389db14a49a1f1bf3dbe538a27" is_pre_election: true
I20250109 08:59:14.130442 25222 raft_consensus.cc:2388] T 3b37da4b59cf41b8889527eb2b0793c9 P 2ff0106754a84b39989b9e042b4af3d6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9d26e8b39691431590fd77e182046c66 in current term 1: Already voted for candidate e6d654389db14a49a1f1bf3dbe538a27 in this term.
I20250109 08:59:14.131397 25263 leader_election.cc:304] T 3b37da4b59cf41b8889527eb2b0793c9 P 9d26e8b39691431590fd77e182046c66 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9d26e8b39691431590fd77e182046c66; no voters: 2ff0106754a84b39989b9e042b4af3d6, e6d654389db14a49a1f1bf3dbe538a27
I20250109 08:59:14.132025 25517 raft_consensus.cc:3054] T 3b37da4b59cf41b8889527eb2b0793c9 P 9d26e8b39691431590fd77e182046c66 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:59:14.137296 25517 raft_consensus.cc:2743] T 3b37da4b59cf41b8889527eb2b0793c9 P 9d26e8b39691431590fd77e182046c66 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250109 08:59:14.168877 25369 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "3b37da4b59cf41b8889527eb2b0793c9"
dest_uuid: "e6d654389db14a49a1f1bf3dbe538a27"
mode: GRACEFUL
new_leader_uuid: "2ff0106754a84b39989b9e042b4af3d6"
from {username='slave'} at 127.0.0.1:50372
I20250109 08:59:14.169497 25369 raft_consensus.cc:604] T 3b37da4b59cf41b8889527eb2b0793c9 P e6d654389db14a49a1f1bf3dbe538a27 [term 1 LEADER]: Received request to transfer leadership to TS 2ff0106754a84b39989b9e042b4af3d6
I20250109 08:59:14.173951 25295 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "8022fc0563b947669983f72b9b7450b3"
dest_uuid: "9d26e8b39691431590fd77e182046c66"
mode: GRACEFUL
new_leader_uuid: "2ff0106754a84b39989b9e042b4af3d6"
from {username='slave'} at 127.0.0.1:45190
I20250109 08:59:14.174392 25295 raft_consensus.cc:604] T 8022fc0563b947669983f72b9b7450b3 P 9d26e8b39691431590fd77e182046c66 [term 1 LEADER]: Received request to transfer leadership to TS 2ff0106754a84b39989b9e042b4af3d6
I20250109 08:59:14.178629 25295 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "9bf180fefb074fd192f6f80ae1460ba5"
dest_uuid: "9d26e8b39691431590fd77e182046c66"
mode: GRACEFUL
new_leader_uuid: "2ff0106754a84b39989b9e042b4af3d6"
from {username='slave'} at 127.0.0.1:45190
I20250109 08:59:14.179068 25295 raft_consensus.cc:604] T 9bf180fefb074fd192f6f80ae1460ba5 P 9d26e8b39691431590fd77e182046c66 [term 1 LEADER]: Received request to transfer leadership to TS 2ff0106754a84b39989b9e042b4af3d6
I20250109 08:59:14.183022 25369 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "cb8ed58651e74cbabff2b027ba4a9a1e"
dest_uuid: "e6d654389db14a49a1f1bf3dbe538a27"
mode: GRACEFUL
new_leader_uuid: "2ff0106754a84b39989b9e042b4af3d6"
from {username='slave'} at 127.0.0.1:50372
I20250109 08:59:14.183509 25369 raft_consensus.cc:604] T cb8ed58651e74cbabff2b027ba4a9a1e P e6d654389db14a49a1f1bf3dbe538a27 [term 1 LEADER]: Received request to transfer leadership to TS 2ff0106754a84b39989b9e042b4af3d6
I20250109 08:59:14.187722 25369 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "346220ee65da44b8862c9f28d7aa6aa3"
dest_uuid: "e6d654389db14a49a1f1bf3dbe538a27"
mode: GRACEFUL
new_leader_uuid: "9d26e8b39691431590fd77e182046c66"
from {username='slave'} at 127.0.0.1:50372
I20250109 08:59:14.188162 25369 raft_consensus.cc:604] T 346220ee65da44b8862c9f28d7aa6aa3 P e6d654389db14a49a1f1bf3dbe538a27 [term 1 LEADER]: Received request to transfer leadership to TS 9d26e8b39691431590fd77e182046c66
I20250109 08:59:14.192319 25222 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "9a3240fe2e094ba2bc002579bdb29e84"
dest_uuid: "2ff0106754a84b39989b9e042b4af3d6"
mode: GRACEFUL
new_leader_uuid: "9d26e8b39691431590fd77e182046c66"
from {username='slave'} at 127.0.0.1:55368
I20250109 08:59:14.192844 25222 raft_consensus.cc:604] T 9a3240fe2e094ba2bc002579bdb29e84 P 2ff0106754a84b39989b9e042b4af3d6 [term 1 LEADER]: Received request to transfer leadership to TS 9d26e8b39691431590fd77e182046c66
I20250109 08:59:14.197000 25517 consensus_queue.cc:1035] T 8022fc0563b947669983f72b9b7450b3 P 9d26e8b39691431590fd77e182046c66 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:59:14.197494 25222 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "3f4e7967061b4173bd00a9539a371fe3"
dest_uuid: "2ff0106754a84b39989b9e042b4af3d6"
mode: GRACEFUL
new_leader_uuid: "9d26e8b39691431590fd77e182046c66"
from {username='slave'} at 127.0.0.1:55368
I20250109 08:59:14.198128 25222 raft_consensus.cc:604] T 3f4e7967061b4173bd00a9539a371fe3 P 2ff0106754a84b39989b9e042b4af3d6 [term 1 LEADER]: Received request to transfer leadership to TS 9d26e8b39691431590fd77e182046c66
I20250109 08:59:14.202692 25369 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "51277cd5f9374666bc60232309efc100"
dest_uuid: "e6d654389db14a49a1f1bf3dbe538a27"
mode: GRACEFUL
new_leader_uuid: "9d26e8b39691431590fd77e182046c66"
from {username='slave'} at 127.0.0.1:50372
I20250109 08:59:14.203253 25369 raft_consensus.cc:604] T 51277cd5f9374666bc60232309efc100 P e6d654389db14a49a1f1bf3dbe538a27 [term 1 LEADER]: Received request to transfer leadership to TS 9d26e8b39691431590fd77e182046c66
I20250109 08:59:14.207706 25222 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "ed0e3d98ad79424caf0e24ecf3b5f295"
dest_uuid: "2ff0106754a84b39989b9e042b4af3d6"
mode: GRACEFUL
new_leader_uuid: "e6d654389db14a49a1f1bf3dbe538a27"
from {username='slave'} at 127.0.0.1:55368
I20250109 08:59:14.208201 25222 raft_consensus.cc:604] T ed0e3d98ad79424caf0e24ecf3b5f295 P 2ff0106754a84b39989b9e042b4af3d6 [term 1 LEADER]: Received request to transfer leadership to TS e6d654389db14a49a1f1bf3dbe538a27
I20250109 08:59:14.211932 25517 consensus_queue.cc:1035] T 8022fc0563b947669983f72b9b7450b3 P 9d26e8b39691431590fd77e182046c66 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:59:14.219763 25559 raft_consensus.cc:988] T 8022fc0563b947669983f72b9b7450b3 P 9d26e8b39691431590fd77e182046c66: : Instructing follower 2ff0106754a84b39989b9e042b4af3d6 to start an election
I20250109 08:59:14.220145 25517 raft_consensus.cc:1076] T 8022fc0563b947669983f72b9b7450b3 P 9d26e8b39691431590fd77e182046c66 [term 1 LEADER]: Signalling peer 2ff0106754a84b39989b9e042b4af3d6 to start an election
I20250109 08:59:14.221696 25222 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "8022fc0563b947669983f72b9b7450b3"
dest_uuid: "2ff0106754a84b39989b9e042b4af3d6"
from {username='slave'} at 127.0.0.1:55364
I20250109 08:59:14.222209 25222 raft_consensus.cc:491] T 8022fc0563b947669983f72b9b7450b3 P 2ff0106754a84b39989b9e042b4af3d6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:59:14.222538 25222 raft_consensus.cc:3054] T 8022fc0563b947669983f72b9b7450b3 P 2ff0106754a84b39989b9e042b4af3d6 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:59:14.228572 25222 raft_consensus.cc:513] T 8022fc0563b947669983f72b9b7450b3 P 2ff0106754a84b39989b9e042b4af3d6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:14.229859 25222 leader_election.cc:290] T 8022fc0563b947669983f72b9b7450b3 P 2ff0106754a84b39989b9e042b4af3d6 [CANDIDATE]: Term 2 election: Requested vote from peers 9d26e8b39691431590fd77e182046c66 (127.19.226.194:33163), e6d654389db14a49a1f1bf3dbe538a27 (127.19.226.195:39911)
I20250109 08:59:14.230656 25295 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8022fc0563b947669983f72b9b7450b3" candidate_uuid: "2ff0106754a84b39989b9e042b4af3d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9d26e8b39691431590fd77e182046c66"
I20250109 08:59:14.230829 25369 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8022fc0563b947669983f72b9b7450b3" candidate_uuid: "2ff0106754a84b39989b9e042b4af3d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e6d654389db14a49a1f1bf3dbe538a27"
I20250109 08:59:14.231393 25295 raft_consensus.cc:3049] T 8022fc0563b947669983f72b9b7450b3 P 9d26e8b39691431590fd77e182046c66 [term 1 LEADER]: Stepping down as leader of term 1
I20250109 08:59:14.231460 25369 raft_consensus.cc:3054] T 8022fc0563b947669983f72b9b7450b3 P e6d654389db14a49a1f1bf3dbe538a27 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:59:14.231736 25295 raft_consensus.cc:738] T 8022fc0563b947669983f72b9b7450b3 P 9d26e8b39691431590fd77e182046c66 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9d26e8b39691431590fd77e182046c66, State: Running, Role: LEADER
I20250109 08:59:14.232962 25295 consensus_queue.cc:260] T 8022fc0563b947669983f72b9b7450b3 P 9d26e8b39691431590fd77e182046c66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:14.234056 25295 raft_consensus.cc:3054] T 8022fc0563b947669983f72b9b7450b3 P 9d26e8b39691431590fd77e182046c66 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:59:14.237042 25369 raft_consensus.cc:2463] T 8022fc0563b947669983f72b9b7450b3 P e6d654389db14a49a1f1bf3dbe538a27 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ff0106754a84b39989b9e042b4af3d6 in term 2.
I20250109 08:59:14.238126 25188 leader_election.cc:304] T 8022fc0563b947669983f72b9b7450b3 P 2ff0106754a84b39989b9e042b4af3d6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ff0106754a84b39989b9e042b4af3d6, e6d654389db14a49a1f1bf3dbe538a27; no voters:
I20250109 08:59:14.238911 25532 raft_consensus.cc:2798] T 8022fc0563b947669983f72b9b7450b3 P 2ff0106754a84b39989b9e042b4af3d6 [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:59:14.239393 25532 raft_consensus.cc:695] T 8022fc0563b947669983f72b9b7450b3 P 2ff0106754a84b39989b9e042b4af3d6 [term 2 LEADER]: Becoming Leader. State: Replica: 2ff0106754a84b39989b9e042b4af3d6, State: Running, Role: LEADER
I20250109 08:59:14.239809 25295 raft_consensus.cc:2463] T 8022fc0563b947669983f72b9b7450b3 P 9d26e8b39691431590fd77e182046c66 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ff0106754a84b39989b9e042b4af3d6 in term 2.
I20250109 08:59:14.240192 25532 consensus_queue.cc:237] T 8022fc0563b947669983f72b9b7450b3 P 2ff0106754a84b39989b9e042b4af3d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:14.246719 25119 catalog_manager.cc:5526] T 8022fc0563b947669983f72b9b7450b3 P 2ff0106754a84b39989b9e042b4af3d6 reported cstate change: term changed from 1 to 2, leader changed from 9d26e8b39691431590fd77e182046c66 (127.19.226.194) to 2ff0106754a84b39989b9e042b4af3d6 (127.19.226.193). New cstate: current_term: 2 leader_uuid: "2ff0106754a84b39989b9e042b4af3d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } health_report { overall_health: UNKNOWN } } }
I20250109 08:59:14.279466 25369 raft_consensus.cc:3054] T 9a3240fe2e094ba2bc002579bdb29e84 P e6d654389db14a49a1f1bf3dbe538a27 [term 0 FOLLOWER]: Advancing to term 1
I20250109 08:59:14.280493 25531 consensus_queue.cc:1035] T 3b37da4b59cf41b8889527eb2b0793c9 P e6d654389db14a49a1f1bf3dbe538a27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:59:14.287796 25450 raft_consensus.cc:988] T 3b37da4b59cf41b8889527eb2b0793c9 P e6d654389db14a49a1f1bf3dbe538a27: : Instructing follower 2ff0106754a84b39989b9e042b4af3d6 to start an election
I20250109 08:59:14.288539 25531 raft_consensus.cc:1076] T 3b37da4b59cf41b8889527eb2b0793c9 P e6d654389db14a49a1f1bf3dbe538a27 [term 1 LEADER]: Signalling peer 2ff0106754a84b39989b9e042b4af3d6 to start an election
I20250109 08:59:14.288353 25532 consensus_queue.cc:1035] T 9a3240fe2e094ba2bc002579bdb29e84 P 2ff0106754a84b39989b9e042b4af3d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250109 08:59:14.291543 25220 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "3b37da4b59cf41b8889527eb2b0793c9"
dest_uuid: "2ff0106754a84b39989b9e042b4af3d6"
from {username='slave'} at 127.0.0.1:55350
I20250109 08:59:14.293565 25220 raft_consensus.cc:491] T 3b37da4b59cf41b8889527eb2b0793c9 P 2ff0106754a84b39989b9e042b4af3d6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:59:14.291549 25564 raft_consensus.cc:491] T 9a3240fe2e094ba2bc002579bdb29e84 P e6d654389db14a49a1f1bf3dbe538a27 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:59:14.293998 25220 raft_consensus.cc:3054] T 3b37da4b59cf41b8889527eb2b0793c9 P 2ff0106754a84b39989b9e042b4af3d6 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:59:14.294302 25564 raft_consensus.cc:513] T 9a3240fe2e094ba2bc002579bdb29e84 P e6d654389db14a49a1f1bf3dbe538a27 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:14.293939 25450 consensus_queue.cc:1035] T 3b37da4b59cf41b8889527eb2b0793c9 P e6d654389db14a49a1f1bf3dbe538a27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20250109 08:59:14.297284 25564 leader_election.cc:290] T 9a3240fe2e094ba2bc002579bdb29e84 P e6d654389db14a49a1f1bf3dbe538a27 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2ff0106754a84b39989b9e042b4af3d6 (127.19.226.193:34345), 9d26e8b39691431590fd77e182046c66 (127.19.226.194:33163)
I20250109 08:59:14.297497 25222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a3240fe2e094ba2bc002579bdb29e84" candidate_uuid: "e6d654389db14a49a1f1bf3dbe538a27" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ff0106754a84b39989b9e042b4af3d6" is_pre_election: true
I20250109 08:59:14.298952 25294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a3240fe2e094ba2bc002579bdb29e84" candidate_uuid: "e6d654389db14a49a1f1bf3dbe538a27" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d26e8b39691431590fd77e182046c66" is_pre_election: true
I20250109 08:59:14.299647 25294 raft_consensus.cc:2463] T 9a3240fe2e094ba2bc002579bdb29e84 P 9d26e8b39691431590fd77e182046c66 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6d654389db14a49a1f1bf3dbe538a27 in term 1.
I20250109 08:59:14.300581 25336 leader_election.cc:304] T 9a3240fe2e094ba2bc002579bdb29e84 P e6d654389db14a49a1f1bf3dbe538a27 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9d26e8b39691431590fd77e182046c66, e6d654389db14a49a1f1bf3dbe538a27; no voters: 2ff0106754a84b39989b9e042b4af3d6
I20250109 08:59:14.303972 25220 raft_consensus.cc:513] T 3b37da4b59cf41b8889527eb2b0793c9 P 2ff0106754a84b39989b9e042b4af3d6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:14.310693 25220 leader_election.cc:290] T 3b37da4b59cf41b8889527eb2b0793c9 P 2ff0106754a84b39989b9e042b4af3d6 [CANDIDATE]: Term 2 election: Requested vote from peers 9d26e8b39691431590fd77e182046c66 (127.19.226.194:33163), e6d654389db14a49a1f1bf3dbe538a27 (127.19.226.195:39911)
I20250109 08:59:14.312179 25531 raft_consensus.cc:2798] T 9a3240fe2e094ba2bc002579bdb29e84 P e6d654389db14a49a1f1bf3dbe538a27 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20250109 08:59:14.312028 25295 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b37da4b59cf41b8889527eb2b0793c9" candidate_uuid: "2ff0106754a84b39989b9e042b4af3d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9d26e8b39691431590fd77e182046c66"
I20250109 08:59:14.313623 25295 raft_consensus.cc:3054] T 3b37da4b59cf41b8889527eb2b0793c9 P 9d26e8b39691431590fd77e182046c66 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:59:14.313666 25371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b37da4b59cf41b8889527eb2b0793c9" candidate_uuid: "2ff0106754a84b39989b9e042b4af3d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e6d654389db14a49a1f1bf3dbe538a27"
I20250109 08:59:14.313959 25531 raft_consensus.cc:491] T 9a3240fe2e094ba2bc002579bdb29e84 P e6d654389db14a49a1f1bf3dbe538a27 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2ff0106754a84b39989b9e042b4af3d6)
I20250109 08:59:14.315991 25371 raft_consensus.cc:3049] T 3b37da4b59cf41b8889527eb2b0793c9 P e6d654389db14a49a1f1bf3dbe538a27 [term 1 LEADER]: Stepping down as leader of term 1
I20250109 08:59:14.316247 25531 raft_consensus.cc:3054] T 9a3240fe2e094ba2bc002579bdb29e84 P e6d654389db14a49a1f1bf3dbe538a27 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:59:14.316354 25371 raft_consensus.cc:738] T 3b37da4b59cf41b8889527eb2b0793c9 P e6d654389db14a49a1f1bf3dbe538a27 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e6d654389db14a49a1f1bf3dbe538a27, State: Running, Role: LEADER
I20250109 08:59:14.319046 25371 consensus_queue.cc:260] T 3b37da4b59cf41b8889527eb2b0793c9 P e6d654389db14a49a1f1bf3dbe538a27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:14.318197 25517 consensus_queue.cc:1035] T 9bf180fefb074fd192f6f80ae1460ba5 P 9d26e8b39691431590fd77e182046c66 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20250109 08:59:14.320317 25371 raft_consensus.cc:3054] T 3b37da4b59cf41b8889527eb2b0793c9 P e6d654389db14a49a1f1bf3dbe538a27 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:59:14.317889 25532 consensus_queue.cc:1035] T 9a3240fe2e094ba2bc002579bdb29e84 P 2ff0106754a84b39989b9e042b4af3d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20250109 08:59:14.321736 25295 raft_consensus.cc:2463] T 3b37da4b59cf41b8889527eb2b0793c9 P 9d26e8b39691431590fd77e182046c66 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ff0106754a84b39989b9e042b4af3d6 in term 2.
I20250109 08:59:14.323269 25531 raft_consensus.cc:513] T 9a3240fe2e094ba2bc002579bdb29e84 P e6d654389db14a49a1f1bf3dbe538a27 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:14.325069 25187 leader_election.cc:304] T 3b37da4b59cf41b8889527eb2b0793c9 P 2ff0106754a84b39989b9e042b4af3d6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ff0106754a84b39989b9e042b4af3d6, 9d26e8b39691431590fd77e182046c66; no voters:
I20250109 08:59:14.325420 25531 leader_election.cc:290] T 9a3240fe2e094ba2bc002579bdb29e84 P e6d654389db14a49a1f1bf3dbe538a27 [CANDIDATE]: Term 2 election: Requested vote from peers 2ff0106754a84b39989b9e042b4af3d6 (127.19.226.193:34345), 9d26e8b39691431590fd77e182046c66 (127.19.226.194:33163)
I20250109 08:59:14.326061 25369 raft_consensus.cc:1235] T 9a3240fe2e094ba2bc002579bdb29e84 P e6d654389db14a49a1f1bf3dbe538a27 [term 2 FOLLOWER]: Rejecting Update request from peer 2ff0106754a84b39989b9e042b4af3d6 for earlier term 1. Current term is 2. Ops: []
I20250109 08:59:14.325917 25424 consensus_queue.cc:1035] T 3f4e7967061b4173bd00a9539a371fe3 P 2ff0106754a84b39989b9e042b4af3d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:59:14.326560 25294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a3240fe2e094ba2bc002579bdb29e84" candidate_uuid: "e6d654389db14a49a1f1bf3dbe538a27" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9d26e8b39691431590fd77e182046c66"
I20250109 08:59:14.327165 25294 raft_consensus.cc:3054] T 9a3240fe2e094ba2bc002579bdb29e84 P 9d26e8b39691431590fd77e182046c66 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:59:14.326368 25532 raft_consensus.cc:2798] T 3b37da4b59cf41b8889527eb2b0793c9 P 2ff0106754a84b39989b9e042b4af3d6 [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:59:14.328706 25532 raft_consensus.cc:695] T 3b37da4b59cf41b8889527eb2b0793c9 P 2ff0106754a84b39989b9e042b4af3d6 [term 2 LEADER]: Becoming Leader. State: Replica: 2ff0106754a84b39989b9e042b4af3d6, State: Running, Role: LEADER
I20250109 08:59:14.329509 25532 consensus_queue.cc:237] T 3b37da4b59cf41b8889527eb2b0793c9 P 2ff0106754a84b39989b9e042b4af3d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:14.334278 25294 raft_consensus.cc:2463] T 9a3240fe2e094ba2bc002579bdb29e84 P 9d26e8b39691431590fd77e182046c66 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6d654389db14a49a1f1bf3dbe538a27 in term 2.
I20250109 08:59:14.341037 25295 raft_consensus.cc:1235] T 9a3240fe2e094ba2bc002579bdb29e84 P 9d26e8b39691431590fd77e182046c66 [term 2 FOLLOWER]: Rejecting Update request from peer 2ff0106754a84b39989b9e042b4af3d6 for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20250109 08:59:14.339643 25220 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a3240fe2e094ba2bc002579bdb29e84" candidate_uuid: "e6d654389db14a49a1f1bf3dbe538a27" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2ff0106754a84b39989b9e042b4af3d6"
I20250109 08:59:14.342023 25336 leader_election.cc:304] T 9a3240fe2e094ba2bc002579bdb29e84 P e6d654389db14a49a1f1bf3dbe538a27 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d26e8b39691431590fd77e182046c66, e6d654389db14a49a1f1bf3dbe538a27; no voters:
I20250109 08:59:14.343911 25550 consensus_queue.cc:1046] T 9a3240fe2e094ba2bc002579bdb29e84 P 2ff0106754a84b39989b9e042b4af3d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250109 08:59:14.344816 25532 raft_consensus.cc:3049] T 9a3240fe2e094ba2bc002579bdb29e84 P 2ff0106754a84b39989b9e042b4af3d6 [term 1 LEADER]: Stepping down as leader of term 1
I20250109 08:59:14.338824 25371 raft_consensus.cc:2463] T 3b37da4b59cf41b8889527eb2b0793c9 P e6d654389db14a49a1f1bf3dbe538a27 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ff0106754a84b39989b9e042b4af3d6 in term 2.
I20250109 08:59:14.345690 25564 raft_consensus.cc:2798] T 9a3240fe2e094ba2bc002579bdb29e84 P e6d654389db14a49a1f1bf3dbe538a27 [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:59:14.345244 25532 raft_consensus.cc:738] T 9a3240fe2e094ba2bc002579bdb29e84 P 2ff0106754a84b39989b9e042b4af3d6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2ff0106754a84b39989b9e042b4af3d6, State: Running, Role: LEADER
I20250109 08:59:14.346253 25564 raft_consensus.cc:695] T 9a3240fe2e094ba2bc002579bdb29e84 P e6d654389db14a49a1f1bf3dbe538a27 [term 2 LEADER]: Becoming Leader. State: Replica: e6d654389db14a49a1f1bf3dbe538a27, State: Running, Role: LEADER
I20250109 08:59:14.346822 25532 consensus_queue.cc:260] T 9a3240fe2e094ba2bc002579bdb29e84 P 2ff0106754a84b39989b9e042b4af3d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:14.347169 25564 consensus_queue.cc:237] T 9a3240fe2e094ba2bc002579bdb29e84 P e6d654389db14a49a1f1bf3dbe538a27 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:14.348172 25532 raft_consensus.cc:3054] T 9a3240fe2e094ba2bc002579bdb29e84 P 2ff0106754a84b39989b9e042b4af3d6 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:59:14.359109 25532 consensus_queue.cc:1035] T 3f4e7967061b4173bd00a9539a371fe3 P 2ff0106754a84b39989b9e042b4af3d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:59:14.359422 25120 catalog_manager.cc:5526] T 3b37da4b59cf41b8889527eb2b0793c9 P 2ff0106754a84b39989b9e042b4af3d6 reported cstate change: term changed from 1 to 2, leader changed from e6d654389db14a49a1f1bf3dbe538a27 (127.19.226.195) to 2ff0106754a84b39989b9e042b4af3d6 (127.19.226.193). New cstate: current_term: 2 leader_uuid: "2ff0106754a84b39989b9e042b4af3d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } health_report { overall_health: UNKNOWN } } }
I20250109 08:59:14.370148 25517 consensus_queue.cc:1035] T 9bf180fefb074fd192f6f80ae1460ba5 P 9d26e8b39691431590fd77e182046c66 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:59:14.370527 25424 raft_consensus.cc:988] T 3f4e7967061b4173bd00a9539a371fe3 P 2ff0106754a84b39989b9e042b4af3d6: : Instructing follower 9d26e8b39691431590fd77e182046c66 to start an election
I20250109 08:59:14.371047 25424 raft_consensus.cc:1076] T 3f4e7967061b4173bd00a9539a371fe3 P 2ff0106754a84b39989b9e042b4af3d6 [term 1 LEADER]: Signalling peer 9d26e8b39691431590fd77e182046c66 to start an election
I20250109 08:59:14.372574 25295 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "3f4e7967061b4173bd00a9539a371fe3"
dest_uuid: "9d26e8b39691431590fd77e182046c66"
from {username='slave'} at 127.0.0.1:45178
I20250109 08:59:14.373075 25295 raft_consensus.cc:491] T 3f4e7967061b4173bd00a9539a371fe3 P 9d26e8b39691431590fd77e182046c66 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:59:14.373409 25295 raft_consensus.cc:3054] T 3f4e7967061b4173bd00a9539a371fe3 P 9d26e8b39691431590fd77e182046c66 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:59:14.374788 25120 catalog_manager.cc:5526] T 9a3240fe2e094ba2bc002579bdb29e84 P e6d654389db14a49a1f1bf3dbe538a27 reported cstate change: term changed from 1 to 2, leader changed from 2ff0106754a84b39989b9e042b4af3d6 (127.19.226.193) to e6d654389db14a49a1f1bf3dbe538a27 (127.19.226.195). New cstate: current_term: 2 leader_uuid: "e6d654389db14a49a1f1bf3dbe538a27" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } health_report { overall_health: HEALTHY } } }
I20250109 08:59:14.379730 25295 raft_consensus.cc:513] T 3f4e7967061b4173bd00a9539a371fe3 P 9d26e8b39691431590fd77e182046c66 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:14.381697 25295 leader_election.cc:290] T 3f4e7967061b4173bd00a9539a371fe3 P 9d26e8b39691431590fd77e182046c66 [CANDIDATE]: Term 2 election: Requested vote from peers 2ff0106754a84b39989b9e042b4af3d6 (127.19.226.193:34345), e6d654389db14a49a1f1bf3dbe538a27 (127.19.226.195:39911)
I20250109 08:59:14.382426 25222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f4e7967061b4173bd00a9539a371fe3" candidate_uuid: "9d26e8b39691431590fd77e182046c66" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2ff0106754a84b39989b9e042b4af3d6"
I20250109 08:59:14.383024 25222 raft_consensus.cc:3049] T 3f4e7967061b4173bd00a9539a371fe3 P 2ff0106754a84b39989b9e042b4af3d6 [term 1 LEADER]: Stepping down as leader of term 1
I20250109 08:59:14.383311 25222 raft_consensus.cc:738] T 3f4e7967061b4173bd00a9539a371fe3 P 2ff0106754a84b39989b9e042b4af3d6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2ff0106754a84b39989b9e042b4af3d6, State: Running, Role: LEADER
I20250109 08:59:14.383973 25222 consensus_queue.cc:260] T 3f4e7967061b4173bd00a9539a371fe3 P 2ff0106754a84b39989b9e042b4af3d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:14.384925 25517 raft_consensus.cc:988] T 9bf180fefb074fd192f6f80ae1460ba5 P 9d26e8b39691431590fd77e182046c66: : Instructing follower 2ff0106754a84b39989b9e042b4af3d6 to start an election
I20250109 08:59:14.385181 25222 raft_consensus.cc:3054] T 3f4e7967061b4173bd00a9539a371fe3 P 2ff0106754a84b39989b9e042b4af3d6 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:59:14.385394 25517 raft_consensus.cc:1076] T 9bf180fefb074fd192f6f80ae1460ba5 P 9d26e8b39691431590fd77e182046c66 [term 1 LEADER]: Signalling peer 2ff0106754a84b39989b9e042b4af3d6 to start an election
I20250109 08:59:14.387019 25220 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "9bf180fefb074fd192f6f80ae1460ba5"
dest_uuid: "2ff0106754a84b39989b9e042b4af3d6"
from {username='slave'} at 127.0.0.1:55364
I20250109 08:59:14.387581 25220 raft_consensus.cc:491] T 9bf180fefb074fd192f6f80ae1460ba5 P 2ff0106754a84b39989b9e042b4af3d6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:59:14.387971 25220 raft_consensus.cc:3054] T 9bf180fefb074fd192f6f80ae1460ba5 P 2ff0106754a84b39989b9e042b4af3d6 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:59:14.387248 25368 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f4e7967061b4173bd00a9539a371fe3" candidate_uuid: "9d26e8b39691431590fd77e182046c66" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e6d654389db14a49a1f1bf3dbe538a27"
I20250109 08:59:14.390332 25368 raft_consensus.cc:3054] T 3f4e7967061b4173bd00a9539a371fe3 P e6d654389db14a49a1f1bf3dbe538a27 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:59:14.394371 25220 raft_consensus.cc:513] T 9bf180fefb074fd192f6f80ae1460ba5 P 2ff0106754a84b39989b9e042b4af3d6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:14.395079 25368 raft_consensus.cc:2463] T 3f4e7967061b4173bd00a9539a371fe3 P e6d654389db14a49a1f1bf3dbe538a27 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d26e8b39691431590fd77e182046c66 in term 2.
I20250109 08:59:14.396239 25263 leader_election.cc:304] T 3f4e7967061b4173bd00a9539a371fe3 P 9d26e8b39691431590fd77e182046c66 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d26e8b39691431590fd77e182046c66, e6d654389db14a49a1f1bf3dbe538a27; no voters:
I20250109 08:59:14.397068 25295 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bf180fefb074fd192f6f80ae1460ba5" candidate_uuid: "2ff0106754a84b39989b9e042b4af3d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9d26e8b39691431590fd77e182046c66"
I20250109 08:59:14.397282 25517 raft_consensus.cc:2798] T 3f4e7967061b4173bd00a9539a371fe3 P 9d26e8b39691431590fd77e182046c66 [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:59:14.397692 25295 raft_consensus.cc:3049] T 9bf180fefb074fd192f6f80ae1460ba5 P 9d26e8b39691431590fd77e182046c66 [term 1 LEADER]: Stepping down as leader of term 1
I20250109 08:59:14.395190 25222 raft_consensus.cc:2463] T 3f4e7967061b4173bd00a9539a371fe3 P 2ff0106754a84b39989b9e042b4af3d6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d26e8b39691431590fd77e182046c66 in term 2.
I20250109 08:59:14.398373 25295 raft_consensus.cc:738] T 9bf180fefb074fd192f6f80ae1460ba5 P 9d26e8b39691431590fd77e182046c66 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9d26e8b39691431590fd77e182046c66, State: Running, Role: LEADER
I20250109 08:59:14.397940 25220 leader_election.cc:290] T 9bf180fefb074fd192f6f80ae1460ba5 P 2ff0106754a84b39989b9e042b4af3d6 [CANDIDATE]: Term 2 election: Requested vote from peers 9d26e8b39691431590fd77e182046c66 (127.19.226.194:33163), e6d654389db14a49a1f1bf3dbe538a27 (127.19.226.195:39911)
I20250109 08:59:14.397830 25517 raft_consensus.cc:695] T 3f4e7967061b4173bd00a9539a371fe3 P 9d26e8b39691431590fd77e182046c66 [term 2 LEADER]: Becoming Leader. State: Replica: 9d26e8b39691431590fd77e182046c66, State: Running, Role: LEADER
I20250109 08:59:14.399144 25295 consensus_queue.cc:260] T 9bf180fefb074fd192f6f80ae1460ba5 P 9d26e8b39691431590fd77e182046c66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:14.399825 25517 consensus_queue.cc:237] T 3f4e7967061b4173bd00a9539a371fe3 P 9d26e8b39691431590fd77e182046c66 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:14.397871 25370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bf180fefb074fd192f6f80ae1460ba5" candidate_uuid: "2ff0106754a84b39989b9e042b4af3d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e6d654389db14a49a1f1bf3dbe538a27"
I20250109 08:59:14.400990 25370 raft_consensus.cc:3054] T 9bf180fefb074fd192f6f80ae1460ba5 P e6d654389db14a49a1f1bf3dbe538a27 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:59:14.400197 25295 raft_consensus.cc:3054] T 9bf180fefb074fd192f6f80ae1460ba5 P 9d26e8b39691431590fd77e182046c66 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:59:14.408638 25119 catalog_manager.cc:5526] T 3f4e7967061b4173bd00a9539a371fe3 P 9d26e8b39691431590fd77e182046c66 reported cstate change: term changed from 1 to 2, leader changed from 2ff0106754a84b39989b9e042b4af3d6 (127.19.226.193) to 9d26e8b39691431590fd77e182046c66 (127.19.226.194). New cstate: current_term: 2 leader_uuid: "9d26e8b39691431590fd77e182046c66" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } health_report { overall_health: UNKNOWN } } }
I20250109 08:59:14.409511 25370 raft_consensus.cc:2463] T 9bf180fefb074fd192f6f80ae1460ba5 P e6d654389db14a49a1f1bf3dbe538a27 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ff0106754a84b39989b9e042b4af3d6 in term 2.
I20250109 08:59:14.410485 25295 raft_consensus.cc:2463] T 9bf180fefb074fd192f6f80ae1460ba5 P 9d26e8b39691431590fd77e182046c66 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ff0106754a84b39989b9e042b4af3d6 in term 2.
I20250109 08:59:14.410704 25188 leader_election.cc:304] T 9bf180fefb074fd192f6f80ae1460ba5 P 2ff0106754a84b39989b9e042b4af3d6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ff0106754a84b39989b9e042b4af3d6, e6d654389db14a49a1f1bf3dbe538a27; no voters:
I20250109 08:59:14.411517 25532 raft_consensus.cc:2798] T 9bf180fefb074fd192f6f80ae1460ba5 P 2ff0106754a84b39989b9e042b4af3d6 [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:59:14.411983 25532 raft_consensus.cc:695] T 9bf180fefb074fd192f6f80ae1460ba5 P 2ff0106754a84b39989b9e042b4af3d6 [term 2 LEADER]: Becoming Leader. State: Replica: 2ff0106754a84b39989b9e042b4af3d6, State: Running, Role: LEADER
I20250109 08:59:14.412775 25532 consensus_queue.cc:237] T 9bf180fefb074fd192f6f80ae1460ba5 P 2ff0106754a84b39989b9e042b4af3d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:14.419888 25117 catalog_manager.cc:5526] T 9bf180fefb074fd192f6f80ae1460ba5 P 2ff0106754a84b39989b9e042b4af3d6 reported cstate change: term changed from 1 to 2, leader changed from 9d26e8b39691431590fd77e182046c66 (127.19.226.194) to 2ff0106754a84b39989b9e042b4af3d6 (127.19.226.193). New cstate: current_term: 2 leader_uuid: "2ff0106754a84b39989b9e042b4af3d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } health_report { overall_health: UNKNOWN } } }
I20250109 08:59:14.445175 25531 consensus_queue.cc:1035] T 51277cd5f9374666bc60232309efc100 P e6d654389db14a49a1f1bf3dbe538a27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:59:14.450740 25564 raft_consensus.cc:988] T 51277cd5f9374666bc60232309efc100 P e6d654389db14a49a1f1bf3dbe538a27: : Instructing follower 9d26e8b39691431590fd77e182046c66 to start an election
I20250109 08:59:14.451201 25450 raft_consensus.cc:1076] T 51277cd5f9374666bc60232309efc100 P e6d654389db14a49a1f1bf3dbe538a27 [term 1 LEADER]: Signalling peer 9d26e8b39691431590fd77e182046c66 to start an election
I20250109 08:59:14.452741 25294 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "51277cd5f9374666bc60232309efc100"
dest_uuid: "9d26e8b39691431590fd77e182046c66"
from {username='slave'} at 127.0.0.1:45172
I20250109 08:59:14.453322 25294 raft_consensus.cc:491] T 51277cd5f9374666bc60232309efc100 P 9d26e8b39691431590fd77e182046c66 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:59:14.453671 25294 raft_consensus.cc:3054] T 51277cd5f9374666bc60232309efc100 P 9d26e8b39691431590fd77e182046c66 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:59:14.458276 25450 consensus_queue.cc:1035] T 51277cd5f9374666bc60232309efc100 P e6d654389db14a49a1f1bf3dbe538a27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:59:14.460937 25294 raft_consensus.cc:513] T 51277cd5f9374666bc60232309efc100 P 9d26e8b39691431590fd77e182046c66 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:14.463016 25532 consensus_queue.cc:1035] T ed0e3d98ad79424caf0e24ecf3b5f295 P 2ff0106754a84b39989b9e042b4af3d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250109 08:59:14.463826 25295 raft_consensus.cc:1235] T 51277cd5f9374666bc60232309efc100 P 9d26e8b39691431590fd77e182046c66 [term 2 FOLLOWER]: Rejecting Update request from peer e6d654389db14a49a1f1bf3dbe538a27 for earlier term 1. Current term is 2. Ops: []
I20250109 08:59:14.464937 25220 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51277cd5f9374666bc60232309efc100" candidate_uuid: "9d26e8b39691431590fd77e182046c66" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2ff0106754a84b39989b9e042b4af3d6"
I20250109 08:59:14.463464 25294 leader_election.cc:290] T 51277cd5f9374666bc60232309efc100 P 9d26e8b39691431590fd77e182046c66 [CANDIDATE]: Term 2 election: Requested vote from peers 2ff0106754a84b39989b9e042b4af3d6 (127.19.226.193:34345), e6d654389db14a49a1f1bf3dbe538a27 (127.19.226.195:39911)
I20250109 08:59:14.465991 25450 consensus_queue.cc:1046] T 51277cd5f9374666bc60232309efc100 P e6d654389db14a49a1f1bf3dbe538a27 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:59:14.466866 25450 raft_consensus.cc:3049] T 51277cd5f9374666bc60232309efc100 P e6d654389db14a49a1f1bf3dbe538a27 [term 1 LEADER]: Stepping down as leader of term 1
I20250109 08:59:14.467376 25450 raft_consensus.cc:738] T 51277cd5f9374666bc60232309efc100 P e6d654389db14a49a1f1bf3dbe538a27 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e6d654389db14a49a1f1bf3dbe538a27, State: Running, Role: LEADER
I20250109 08:59:14.468632 25370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51277cd5f9374666bc60232309efc100" candidate_uuid: "9d26e8b39691431590fd77e182046c66" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e6d654389db14a49a1f1bf3dbe538a27"
I20250109 08:59:14.468228 25450 consensus_queue.cc:260] T 51277cd5f9374666bc60232309efc100 P e6d654389db14a49a1f1bf3dbe538a27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:14.469653 25450 raft_consensus.cc:3054] T 51277cd5f9374666bc60232309efc100 P e6d654389db14a49a1f1bf3dbe538a27 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:59:14.483649 25532 raft_consensus.cc:988] T ed0e3d98ad79424caf0e24ecf3b5f295 P 2ff0106754a84b39989b9e042b4af3d6: : Instructing follower e6d654389db14a49a1f1bf3dbe538a27 to start an election
I20250109 08:59:14.484184 25532 raft_consensus.cc:1076] T ed0e3d98ad79424caf0e24ecf3b5f295 P 2ff0106754a84b39989b9e042b4af3d6 [term 1 LEADER]: Signalling peer e6d654389db14a49a1f1bf3dbe538a27 to start an election
I20250109 08:59:14.486788 25370 raft_consensus.cc:2463] T 51277cd5f9374666bc60232309efc100 P e6d654389db14a49a1f1bf3dbe538a27 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d26e8b39691431590fd77e182046c66 in term 2.
I20250109 08:59:14.488504 25371 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "ed0e3d98ad79424caf0e24ecf3b5f295"
dest_uuid: "e6d654389db14a49a1f1bf3dbe538a27"
from {username='slave'} at 127.0.0.1:50366
I20250109 08:59:14.489125 25371 raft_consensus.cc:491] T ed0e3d98ad79424caf0e24ecf3b5f295 P e6d654389db14a49a1f1bf3dbe538a27 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:59:14.489503 25371 raft_consensus.cc:3054] T ed0e3d98ad79424caf0e24ecf3b5f295 P e6d654389db14a49a1f1bf3dbe538a27 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:59:14.492877 25263 leader_election.cc:304] T 51277cd5f9374666bc60232309efc100 P 9d26e8b39691431590fd77e182046c66 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9d26e8b39691431590fd77e182046c66, e6d654389db14a49a1f1bf3dbe538a27; no voters: 2ff0106754a84b39989b9e042b4af3d6
I20250109 08:59:14.495586 25371 raft_consensus.cc:513] T ed0e3d98ad79424caf0e24ecf3b5f295 P e6d654389db14a49a1f1bf3dbe538a27 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:14.498394 25371 leader_election.cc:290] T ed0e3d98ad79424caf0e24ecf3b5f295 P e6d654389db14a49a1f1bf3dbe538a27 [CANDIDATE]: Term 2 election: Requested vote from peers 2ff0106754a84b39989b9e042b4af3d6 (127.19.226.193:34345), 9d26e8b39691431590fd77e182046c66 (127.19.226.194:33163)
I20250109 08:59:14.499408 25295 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed0e3d98ad79424caf0e24ecf3b5f295" candidate_uuid: "e6d654389db14a49a1f1bf3dbe538a27" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9d26e8b39691431590fd77e182046c66"
I20250109 08:59:14.499972 25295 raft_consensus.cc:3054] T ed0e3d98ad79424caf0e24ecf3b5f295 P 9d26e8b39691431590fd77e182046c66 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:59:14.501121 25220 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed0e3d98ad79424caf0e24ecf3b5f295" candidate_uuid: "e6d654389db14a49a1f1bf3dbe538a27" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2ff0106754a84b39989b9e042b4af3d6"
I20250109 08:59:14.501778 25220 raft_consensus.cc:3049] T ed0e3d98ad79424caf0e24ecf3b5f295 P 2ff0106754a84b39989b9e042b4af3d6 [term 1 LEADER]: Stepping down as leader of term 1
I20250109 08:59:14.502094 25220 raft_consensus.cc:738] T ed0e3d98ad79424caf0e24ecf3b5f295 P 2ff0106754a84b39989b9e042b4af3d6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2ff0106754a84b39989b9e042b4af3d6, State: Running, Role: LEADER
I20250109 08:59:14.502884 25559 raft_consensus.cc:2798] T 51277cd5f9374666bc60232309efc100 P 9d26e8b39691431590fd77e182046c66 [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:59:14.503088 25450 consensus_queue.cc:1035] T cb8ed58651e74cbabff2b027ba4a9a1e P e6d654389db14a49a1f1bf3dbe538a27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:59:14.503708 25559 raft_consensus.cc:695] T 51277cd5f9374666bc60232309efc100 P 9d26e8b39691431590fd77e182046c66 [term 2 LEADER]: Becoming Leader. State: Replica: 9d26e8b39691431590fd77e182046c66, State: Running, Role: LEADER
I20250109 08:59:14.504528 25559 consensus_queue.cc:237] T 51277cd5f9374666bc60232309efc100 P 9d26e8b39691431590fd77e182046c66 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:14.507546 25295 raft_consensus.cc:2463] T ed0e3d98ad79424caf0e24ecf3b5f295 P 9d26e8b39691431590fd77e182046c66 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6d654389db14a49a1f1bf3dbe538a27 in term 2.
I20250109 08:59:14.508214 25294 raft_consensus.cc:1235] T ed0e3d98ad79424caf0e24ecf3b5f295 P 9d26e8b39691431590fd77e182046c66 [term 2 FOLLOWER]: Rejecting Update request from peer 2ff0106754a84b39989b9e042b4af3d6 for earlier term 1. Current term is 2. Ops: []
I20250109 08:59:14.502895 25220 consensus_queue.cc:260] T ed0e3d98ad79424caf0e24ecf3b5f295 P 2ff0106754a84b39989b9e042b4af3d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:14.509894 25336 leader_election.cc:304] T ed0e3d98ad79424caf0e24ecf3b5f295 P e6d654389db14a49a1f1bf3dbe538a27 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d26e8b39691431590fd77e182046c66, e6d654389db14a49a1f1bf3dbe538a27; no voters:
I20250109 08:59:14.510963 25220 raft_consensus.cc:3054] T ed0e3d98ad79424caf0e24ecf3b5f295 P 2ff0106754a84b39989b9e042b4af3d6 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:59:14.515468 25450 raft_consensus.cc:2798] T ed0e3d98ad79424caf0e24ecf3b5f295 P e6d654389db14a49a1f1bf3dbe538a27 [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:59:14.516203 25450 raft_consensus.cc:695] T ed0e3d98ad79424caf0e24ecf3b5f295 P e6d654389db14a49a1f1bf3dbe538a27 [term 2 LEADER]: Becoming Leader. State: Replica: e6d654389db14a49a1f1bf3dbe538a27, State: Running, Role: LEADER
I20250109 08:59:14.518188 25220 raft_consensus.cc:2463] T ed0e3d98ad79424caf0e24ecf3b5f295 P 2ff0106754a84b39989b9e042b4af3d6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6d654389db14a49a1f1bf3dbe538a27 in term 2.
I20250109 08:59:14.519907 25564 raft_consensus.cc:988] T cb8ed58651e74cbabff2b027ba4a9a1e P e6d654389db14a49a1f1bf3dbe538a27: : Instructing follower 2ff0106754a84b39989b9e042b4af3d6 to start an election
I20250109 08:59:14.519464 25120 catalog_manager.cc:5526] T 51277cd5f9374666bc60232309efc100 P 9d26e8b39691431590fd77e182046c66 reported cstate change: term changed from 1 to 2, leader changed from e6d654389db14a49a1f1bf3dbe538a27 (127.19.226.195) to 9d26e8b39691431590fd77e182046c66 (127.19.226.194). New cstate: current_term: 2 leader_uuid: "9d26e8b39691431590fd77e182046c66" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } health_report { overall_health: UNKNOWN } } }
I20250109 08:59:14.520499 25564 raft_consensus.cc:1076] T cb8ed58651e74cbabff2b027ba4a9a1e P e6d654389db14a49a1f1bf3dbe538a27 [term 1 LEADER]: Signalling peer 2ff0106754a84b39989b9e042b4af3d6 to start an election
I20250109 08:59:14.524135 25222 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "cb8ed58651e74cbabff2b027ba4a9a1e"
dest_uuid: "2ff0106754a84b39989b9e042b4af3d6"
from {username='slave'} at 127.0.0.1:55350
I20250109 08:59:14.524680 25222 raft_consensus.cc:491] T cb8ed58651e74cbabff2b027ba4a9a1e P 2ff0106754a84b39989b9e042b4af3d6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:59:14.525005 25222 raft_consensus.cc:3054] T cb8ed58651e74cbabff2b027ba4a9a1e P 2ff0106754a84b39989b9e042b4af3d6 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:59:14.526636 25531 consensus_queue.cc:1035] T cb8ed58651e74cbabff2b027ba4a9a1e P e6d654389db14a49a1f1bf3dbe538a27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:59:14.531774 25222 raft_consensus.cc:513] T cb8ed58651e74cbabff2b027ba4a9a1e P 2ff0106754a84b39989b9e042b4af3d6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:14.534415 25294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8ed58651e74cbabff2b027ba4a9a1e" candidate_uuid: "2ff0106754a84b39989b9e042b4af3d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9d26e8b39691431590fd77e182046c66"
I20250109 08:59:14.517081 25450 consensus_queue.cc:237] T ed0e3d98ad79424caf0e24ecf3b5f295 P e6d654389db14a49a1f1bf3dbe538a27 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:14.537297 25370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb8ed58651e74cbabff2b027ba4a9a1e" candidate_uuid: "2ff0106754a84b39989b9e042b4af3d6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e6d654389db14a49a1f1bf3dbe538a27"
I20250109 08:59:14.537882 25370 raft_consensus.cc:3049] T cb8ed58651e74cbabff2b027ba4a9a1e P e6d654389db14a49a1f1bf3dbe538a27 [term 1 LEADER]: Stepping down as leader of term 1
I20250109 08:59:14.538237 25370 raft_consensus.cc:738] T cb8ed58651e74cbabff2b027ba4a9a1e P e6d654389db14a49a1f1bf3dbe538a27 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e6d654389db14a49a1f1bf3dbe538a27, State: Running, Role: LEADER
I20250109 08:59:14.538969 25370 consensus_queue.cc:260] T cb8ed58651e74cbabff2b027ba4a9a1e P e6d654389db14a49a1f1bf3dbe538a27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:14.539875 25370 raft_consensus.cc:3054] T cb8ed58651e74cbabff2b027ba4a9a1e P e6d654389db14a49a1f1bf3dbe538a27 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:59:14.536612 25222 leader_election.cc:290] T cb8ed58651e74cbabff2b027ba4a9a1e P 2ff0106754a84b39989b9e042b4af3d6 [CANDIDATE]: Term 2 election: Requested vote from peers 9d26e8b39691431590fd77e182046c66 (127.19.226.194:33163), e6d654389db14a49a1f1bf3dbe538a27 (127.19.226.195:39911)
I20250109 08:59:14.550912 25370 raft_consensus.cc:2463] T cb8ed58651e74cbabff2b027ba4a9a1e P e6d654389db14a49a1f1bf3dbe538a27 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ff0106754a84b39989b9e042b4af3d6 in term 2.
I20250109 08:59:14.550644 25119 catalog_manager.cc:5526] T ed0e3d98ad79424caf0e24ecf3b5f295 P e6d654389db14a49a1f1bf3dbe538a27 reported cstate change: term changed from 1 to 2, leader changed from 2ff0106754a84b39989b9e042b4af3d6 (127.19.226.193) to e6d654389db14a49a1f1bf3dbe538a27 (127.19.226.195). New cstate: current_term: 2 leader_uuid: "e6d654389db14a49a1f1bf3dbe538a27" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } health_report { overall_health: HEALTHY } } }
I20250109 08:59:14.551954 25188 leader_election.cc:304] T cb8ed58651e74cbabff2b027ba4a9a1e P 2ff0106754a84b39989b9e042b4af3d6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2ff0106754a84b39989b9e042b4af3d6, e6d654389db14a49a1f1bf3dbe538a27; no voters: 9d26e8b39691431590fd77e182046c66
I20250109 08:59:14.552771 25532 raft_consensus.cc:2798] T cb8ed58651e74cbabff2b027ba4a9a1e P 2ff0106754a84b39989b9e042b4af3d6 [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:59:14.553251 25532 raft_consensus.cc:695] T cb8ed58651e74cbabff2b027ba4a9a1e P 2ff0106754a84b39989b9e042b4af3d6 [term 2 LEADER]: Becoming Leader. State: Replica: 2ff0106754a84b39989b9e042b4af3d6, State: Running, Role: LEADER
I20250109 08:59:14.553882 25532 consensus_queue.cc:237] T cb8ed58651e74cbabff2b027ba4a9a1e P 2ff0106754a84b39989b9e042b4af3d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:14.561700 25117 catalog_manager.cc:5526] T cb8ed58651e74cbabff2b027ba4a9a1e P 2ff0106754a84b39989b9e042b4af3d6 reported cstate change: term changed from 1 to 2, leader changed from e6d654389db14a49a1f1bf3dbe538a27 (127.19.226.195) to 2ff0106754a84b39989b9e042b4af3d6 (127.19.226.193). New cstate: current_term: 2 leader_uuid: "2ff0106754a84b39989b9e042b4af3d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } health_report { overall_health: UNKNOWN } } }
I20250109 08:59:14.622740 25450 consensus_queue.cc:1035] T 346220ee65da44b8862c9f28d7aa6aa3 P e6d654389db14a49a1f1bf3dbe538a27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:59:14.632292 25531 consensus_queue.cc:1035] T 346220ee65da44b8862c9f28d7aa6aa3 P e6d654389db14a49a1f1bf3dbe538a27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:59:14.639883 25531 raft_consensus.cc:988] T 346220ee65da44b8862c9f28d7aa6aa3 P e6d654389db14a49a1f1bf3dbe538a27: : Instructing follower 9d26e8b39691431590fd77e182046c66 to start an election
I20250109 08:59:14.640290 25450 raft_consensus.cc:1076] T 346220ee65da44b8862c9f28d7aa6aa3 P e6d654389db14a49a1f1bf3dbe538a27 [term 1 LEADER]: Signalling peer 9d26e8b39691431590fd77e182046c66 to start an election
I20250109 08:59:14.641564 25295 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "346220ee65da44b8862c9f28d7aa6aa3"
dest_uuid: "9d26e8b39691431590fd77e182046c66"
from {username='slave'} at 127.0.0.1:45172
I20250109 08:59:14.641976 25295 raft_consensus.cc:491] T 346220ee65da44b8862c9f28d7aa6aa3 P 9d26e8b39691431590fd77e182046c66 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:59:14.642211 25295 raft_consensus.cc:3054] T 346220ee65da44b8862c9f28d7aa6aa3 P 9d26e8b39691431590fd77e182046c66 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:59:14.647325 25295 raft_consensus.cc:513] T 346220ee65da44b8862c9f28d7aa6aa3 P 9d26e8b39691431590fd77e182046c66 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:14.649421 25295 leader_election.cc:290] T 346220ee65da44b8862c9f28d7aa6aa3 P 9d26e8b39691431590fd77e182046c66 [CANDIDATE]: Term 2 election: Requested vote from peers 2ff0106754a84b39989b9e042b4af3d6 (127.19.226.193:34345), e6d654389db14a49a1f1bf3dbe538a27 (127.19.226.195:39911)
I20250109 08:59:14.649713 25222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "346220ee65da44b8862c9f28d7aa6aa3" candidate_uuid: "9d26e8b39691431590fd77e182046c66" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2ff0106754a84b39989b9e042b4af3d6"
I20250109 08:59:14.650336 25222 raft_consensus.cc:3054] T 346220ee65da44b8862c9f28d7aa6aa3 P 2ff0106754a84b39989b9e042b4af3d6 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:59:14.650446 25370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "346220ee65da44b8862c9f28d7aa6aa3" candidate_uuid: "9d26e8b39691431590fd77e182046c66" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e6d654389db14a49a1f1bf3dbe538a27"
I20250109 08:59:14.651079 25370 raft_consensus.cc:3049] T 346220ee65da44b8862c9f28d7aa6aa3 P e6d654389db14a49a1f1bf3dbe538a27 [term 1 LEADER]: Stepping down as leader of term 1
I20250109 08:59:14.651404 25370 raft_consensus.cc:738] T 346220ee65da44b8862c9f28d7aa6aa3 P e6d654389db14a49a1f1bf3dbe538a27 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e6d654389db14a49a1f1bf3dbe538a27, State: Running, Role: LEADER
I20250109 08:59:14.652156 25370 consensus_queue.cc:260] T 346220ee65da44b8862c9f28d7aa6aa3 P e6d654389db14a49a1f1bf3dbe538a27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:14.653546 25370 raft_consensus.cc:3054] T 346220ee65da44b8862c9f28d7aa6aa3 P e6d654389db14a49a1f1bf3dbe538a27 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:59:14.656966 25222 raft_consensus.cc:2463] T 346220ee65da44b8862c9f28d7aa6aa3 P 2ff0106754a84b39989b9e042b4af3d6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d26e8b39691431590fd77e182046c66 in term 2.
I20250109 08:59:14.658136 25261 leader_election.cc:304] T 346220ee65da44b8862c9f28d7aa6aa3 P 9d26e8b39691431590fd77e182046c66 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ff0106754a84b39989b9e042b4af3d6, 9d26e8b39691431590fd77e182046c66; no voters:
I20250109 08:59:14.658807 25559 raft_consensus.cc:2798] T 346220ee65da44b8862c9f28d7aa6aa3 P 9d26e8b39691431590fd77e182046c66 [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:59:14.659198 25559 raft_consensus.cc:695] T 346220ee65da44b8862c9f28d7aa6aa3 P 9d26e8b39691431590fd77e182046c66 [term 2 LEADER]: Becoming Leader. State: Replica: 9d26e8b39691431590fd77e182046c66, State: Running, Role: LEADER
I20250109 08:59:14.659685 25370 raft_consensus.cc:2463] T 346220ee65da44b8862c9f28d7aa6aa3 P e6d654389db14a49a1f1bf3dbe538a27 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d26e8b39691431590fd77e182046c66 in term 2.
I20250109 08:59:14.659979 25559 consensus_queue.cc:237] T 346220ee65da44b8862c9f28d7aa6aa3 P 9d26e8b39691431590fd77e182046c66 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:14.667094 25120 catalog_manager.cc:5526] T 346220ee65da44b8862c9f28d7aa6aa3 P 9d26e8b39691431590fd77e182046c66 reported cstate change: term changed from 1 to 2, leader changed from e6d654389db14a49a1f1bf3dbe538a27 (127.19.226.195) to 9d26e8b39691431590fd77e182046c66 (127.19.226.194). New cstate: current_term: 2 leader_uuid: "9d26e8b39691431590fd77e182046c66" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } health_report { overall_health: UNKNOWN } } }
I20250109 08:59:14.711463 25370 raft_consensus.cc:1270] T 8022fc0563b947669983f72b9b7450b3 P e6d654389db14a49a1f1bf3dbe538a27 [term 2 FOLLOWER]: Refusing update from remote peer 2ff0106754a84b39989b9e042b4af3d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:59:14.712678 25532 consensus_queue.cc:1035] T 8022fc0563b947669983f72b9b7450b3 P 2ff0106754a84b39989b9e042b4af3d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:59:14.719657 25295 raft_consensus.cc:1270] T 8022fc0563b947669983f72b9b7450b3 P 9d26e8b39691431590fd77e182046c66 [term 2 FOLLOWER]: Refusing update from remote peer 2ff0106754a84b39989b9e042b4af3d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:59:14.721230 25424 consensus_queue.cc:1035] T 8022fc0563b947669983f72b9b7450b3 P 2ff0106754a84b39989b9e042b4af3d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:59:14.743512 25295 raft_consensus.cc:1270] T 9a3240fe2e094ba2bc002579bdb29e84 P 9d26e8b39691431590fd77e182046c66 [term 2 FOLLOWER]: Refusing update from remote peer e6d654389db14a49a1f1bf3dbe538a27: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:59:14.743512 25222 raft_consensus.cc:1270] T 9a3240fe2e094ba2bc002579bdb29e84 P 2ff0106754a84b39989b9e042b4af3d6 [term 2 FOLLOWER]: Refusing update from remote peer e6d654389db14a49a1f1bf3dbe538a27: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:59:14.744686 25531 consensus_queue.cc:1035] T 9a3240fe2e094ba2bc002579bdb29e84 P e6d654389db14a49a1f1bf3dbe538a27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:59:14.745380 25450 consensus_queue.cc:1035] T 9a3240fe2e094ba2bc002579bdb29e84 P e6d654389db14a49a1f1bf3dbe538a27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:59:14.823163 25370 raft_consensus.cc:1270] T 3b37da4b59cf41b8889527eb2b0793c9 P e6d654389db14a49a1f1bf3dbe538a27 [term 2 FOLLOWER]: Refusing update from remote peer 2ff0106754a84b39989b9e042b4af3d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:59:14.824349 25550 consensus_queue.cc:1035] T 3b37da4b59cf41b8889527eb2b0793c9 P 2ff0106754a84b39989b9e042b4af3d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:59:14.826097 25295 raft_consensus.cc:1270] T 9bf180fefb074fd192f6f80ae1460ba5 P 9d26e8b39691431590fd77e182046c66 [term 2 FOLLOWER]: Refusing update from remote peer 2ff0106754a84b39989b9e042b4af3d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:59:14.827309 25550 consensus_queue.cc:1035] T 9bf180fefb074fd192f6f80ae1460ba5 P 2ff0106754a84b39989b9e042b4af3d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:59:14.832916 25294 raft_consensus.cc:1270] T 3b37da4b59cf41b8889527eb2b0793c9 P 9d26e8b39691431590fd77e182046c66 [term 2 FOLLOWER]: Refusing update from remote peer 2ff0106754a84b39989b9e042b4af3d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:59:14.834758 25578 consensus_queue.cc:1035] T 3b37da4b59cf41b8889527eb2b0793c9 P 2ff0106754a84b39989b9e042b4af3d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:59:14.837801 25371 raft_consensus.cc:1270] T 9bf180fefb074fd192f6f80ae1460ba5 P e6d654389db14a49a1f1bf3dbe538a27 [term 2 FOLLOWER]: Refusing update from remote peer 2ff0106754a84b39989b9e042b4af3d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:59:14.839413 25424 consensus_queue.cc:1035] T 9bf180fefb074fd192f6f80ae1460ba5 P 2ff0106754a84b39989b9e042b4af3d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:59:14.896755 25370 raft_consensus.cc:1270] T 3f4e7967061b4173bd00a9539a371fe3 P e6d654389db14a49a1f1bf3dbe538a27 [term 2 FOLLOWER]: Refusing update from remote peer 9d26e8b39691431590fd77e182046c66: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:59:14.897912 25559 consensus_queue.cc:1035] T 3f4e7967061b4173bd00a9539a371fe3 P 9d26e8b39691431590fd77e182046c66 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:59:14.905915 25222 raft_consensus.cc:1270] T 3f4e7967061b4173bd00a9539a371fe3 P 2ff0106754a84b39989b9e042b4af3d6 [term 2 FOLLOWER]: Refusing update from remote peer 9d26e8b39691431590fd77e182046c66: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:59:14.907842 25559 consensus_queue.cc:1035] T 3f4e7967061b4173bd00a9539a371fe3 P 9d26e8b39691431590fd77e182046c66 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:59:14.952049 25294 raft_consensus.cc:1270] T ed0e3d98ad79424caf0e24ecf3b5f295 P 9d26e8b39691431590fd77e182046c66 [term 2 FOLLOWER]: Refusing update from remote peer e6d654389db14a49a1f1bf3dbe538a27: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:59:14.952039 25222 raft_consensus.cc:3054] T 51277cd5f9374666bc60232309efc100 P 2ff0106754a84b39989b9e042b4af3d6 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:59:14.953436 25450 consensus_queue.cc:1035] T ed0e3d98ad79424caf0e24ecf3b5f295 P e6d654389db14a49a1f1bf3dbe538a27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20250109 08:59:14.958621 25222 raft_consensus.cc:1270] T 51277cd5f9374666bc60232309efc100 P 2ff0106754a84b39989b9e042b4af3d6 [term 2 FOLLOWER]: Refusing update from remote peer 9d26e8b39691431590fd77e182046c66: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:59:14.960342 25581 consensus_queue.cc:1035] T 51277cd5f9374666bc60232309efc100 P 9d26e8b39691431590fd77e182046c66 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:59:14.962987 25222 raft_consensus.cc:1270] T ed0e3d98ad79424caf0e24ecf3b5f295 P 2ff0106754a84b39989b9e042b4af3d6 [term 2 FOLLOWER]: Refusing update from remote peer e6d654389db14a49a1f1bf3dbe538a27: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:59:14.964659 25531 consensus_queue.cc:1035] T ed0e3d98ad79424caf0e24ecf3b5f295 P e6d654389db14a49a1f1bf3dbe538a27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:59:14.973138 25371 raft_consensus.cc:1270] T 51277cd5f9374666bc60232309efc100 P e6d654389db14a49a1f1bf3dbe538a27 [term 2 FOLLOWER]: Refusing update from remote peer 9d26e8b39691431590fd77e182046c66: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:59:14.974332 25581 consensus_queue.cc:1035] T 51277cd5f9374666bc60232309efc100 P 9d26e8b39691431590fd77e182046c66 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:59:15.132735 25294 raft_consensus.cc:3054] T cb8ed58651e74cbabff2b027ba4a9a1e P 9d26e8b39691431590fd77e182046c66 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:59:15.133021 25371 raft_consensus.cc:1270] T 346220ee65da44b8862c9f28d7aa6aa3 P e6d654389db14a49a1f1bf3dbe538a27 [term 2 FOLLOWER]: Refusing update from remote peer 9d26e8b39691431590fd77e182046c66: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:59:15.134312 25581 consensus_queue.cc:1035] T 346220ee65da44b8862c9f28d7aa6aa3 P 9d26e8b39691431590fd77e182046c66 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:59:15.140094 25294 raft_consensus.cc:1270] T cb8ed58651e74cbabff2b027ba4a9a1e P 9d26e8b39691431590fd77e182046c66 [term 2 FOLLOWER]: Refusing update from remote peer 2ff0106754a84b39989b9e042b4af3d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:59:15.141800 25424 consensus_queue.cc:1035] T cb8ed58651e74cbabff2b027ba4a9a1e P 2ff0106754a84b39989b9e042b4af3d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:59:15.142657 25220 raft_consensus.cc:1270] T 346220ee65da44b8862c9f28d7aa6aa3 P 2ff0106754a84b39989b9e042b4af3d6 [term 2 FOLLOWER]: Refusing update from remote peer 9d26e8b39691431590fd77e182046c66: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:59:15.145694 25559 consensus_queue.cc:1035] T 346220ee65da44b8862c9f28d7aa6aa3 P 9d26e8b39691431590fd77e182046c66 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:59:15.151599 25371 raft_consensus.cc:1270] T cb8ed58651e74cbabff2b027ba4a9a1e P e6d654389db14a49a1f1bf3dbe538a27 [term 2 FOLLOWER]: Refusing update from remote peer 2ff0106754a84b39989b9e042b4af3d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:59:15.155663 25550 consensus_queue.cc:1035] T cb8ed58651e74cbabff2b027ba4a9a1e P 2ff0106754a84b39989b9e042b4af3d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:59:17.210960 25120 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48134:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20250109 08:59:17.228754 25120 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:48134:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20250109 08:59:17.268131 20363 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250109 08:59:17.357483 25629 tablet.cc:1721] T 3f4e7967061b4173bd00a9539a371fe3 P e6d654389db14a49a1f1bf3dbe538a27: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
I20250109 08:59:17.355751 25634 tablet.cc:1721] T 3f4e7967061b4173bd00a9539a371fe3 P 9d26e8b39691431590fd77e182046c66: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
I20250109 08:59:17.366355 25636 tablet.cc:1721] T 9a3240fe2e094ba2bc002579bdb29e84 P e6d654389db14a49a1f1bf3dbe538a27: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
I20250109 08:59:17.369242 25638 tablet.cc:1721] T 346220ee65da44b8862c9f28d7aa6aa3 P 2ff0106754a84b39989b9e042b4af3d6: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
I20250109 08:59:17.389955 25626 tablet.cc:1721] T 346220ee65da44b8862c9f28d7aa6aa3 P 9d26e8b39691431590fd77e182046c66: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
I20250109 08:59:17.398372 25643 tablet.cc:1721] T 9a3240fe2e094ba2bc002579bdb29e84 P 9d26e8b39691431590fd77e182046c66: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
W20250109 08:59:17.400748 25263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.226.195:39911, user_credentials={real_user=slave}} blocked reactor thread for 60717.1us
W20250109 08:59:17.403672 25188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.19.226.195:39911, user_credentials={real_user=slave}} blocked reactor thread for 46647.1us
I20250109 08:59:17.445950 25649 tablet.cc:1721] T 9bf180fefb074fd192f6f80ae1460ba5 P 2ff0106754a84b39989b9e042b4af3d6: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
I20250109 08:59:17.448856 25644 tablet.cc:1721] T 51277cd5f9374666bc60232309efc100 P 9d26e8b39691431590fd77e182046c66: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
I20250109 08:59:17.447883 25646 tablet.cc:1721] T 9a3240fe2e094ba2bc002579bdb29e84 P 2ff0106754a84b39989b9e042b4af3d6: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
I20250109 08:59:17.458143 25650 tablet.cc:1721] T 51277cd5f9374666bc60232309efc100 P e6d654389db14a49a1f1bf3dbe538a27: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
W20250109 08:59:17.427217 25621 log.cc:926] Time spent T 3b37da4b59cf41b8889527eb2b0793c9 P 9d26e8b39691431590fd77e182046c66: Append to log took a long time: real 0.062s user 0.000s sys 0.001s
I20250109 08:59:17.475934 25295 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "075f0030bd964a1687ccee531080fd1c"
dest_uuid: "9d26e8b39691431590fd77e182046c66"
mode: GRACEFUL
new_leader_uuid: "e6d654389db14a49a1f1bf3dbe538a27"
from {username='slave'} at 127.0.0.1:45204
I20250109 08:59:17.476557 25295 raft_consensus.cc:604] T 075f0030bd964a1687ccee531080fd1c P 9d26e8b39691431590fd77e182046c66 [term 2 LEADER]: Received request to transfer leadership to TS e6d654389db14a49a1f1bf3dbe538a27
I20250109 08:59:17.494946 25644 tablet.cc:1721] T ed0e3d98ad79424caf0e24ecf3b5f295 P 9d26e8b39691431590fd77e182046c66: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
I20250109 08:59:17.524505 25636 tablet.cc:1721] T ed0e3d98ad79424caf0e24ecf3b5f295 P e6d654389db14a49a1f1bf3dbe538a27: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
I20250109 08:59:17.536217 25629 tablet.cc:1721] T 346220ee65da44b8862c9f28d7aa6aa3 P e6d654389db14a49a1f1bf3dbe538a27: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
I20250109 08:59:17.538676 25638 tablet.cc:1721] T 3f4e7967061b4173bd00a9539a371fe3 P 2ff0106754a84b39989b9e042b4af3d6: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
I20250109 08:59:17.554616 25634 tablet.cc:1721] T 8022fc0563b947669983f72b9b7450b3 P 9d26e8b39691431590fd77e182046c66: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
I20250109 08:59:17.560833 25643 tablet.cc:1721] T 9bf180fefb074fd192f6f80ae1460ba5 P 9d26e8b39691431590fd77e182046c66: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
I20250109 08:59:17.565912 25559 raft_consensus.cc:988] T 075f0030bd964a1687ccee531080fd1c P 9d26e8b39691431590fd77e182046c66: : Instructing follower e6d654389db14a49a1f1bf3dbe538a27 to start an election
I20250109 08:59:17.565584 25646 tablet.cc:1721] T 8022fc0563b947669983f72b9b7450b3 P 2ff0106754a84b39989b9e042b4af3d6: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
I20250109 08:59:17.566731 25586 raft_consensus.cc:1076] T 075f0030bd964a1687ccee531080fd1c P 9d26e8b39691431590fd77e182046c66 [term 2 LEADER]: Signalling peer e6d654389db14a49a1f1bf3dbe538a27 to start an election
I20250109 08:59:17.554711 25649 tablet.cc:1721] T 51277cd5f9374666bc60232309efc100 P 2ff0106754a84b39989b9e042b4af3d6: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
I20250109 08:59:17.553998 25626 tablet.cc:1721] T 3b37da4b59cf41b8889527eb2b0793c9 P 9d26e8b39691431590fd77e182046c66: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
I20250109 08:59:17.576424 25368 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "075f0030bd964a1687ccee531080fd1c"
dest_uuid: "e6d654389db14a49a1f1bf3dbe538a27"
from {username='slave'} at 127.0.0.1:50362
I20250109 08:59:17.577181 25368 raft_consensus.cc:491] T 075f0030bd964a1687ccee531080fd1c P e6d654389db14a49a1f1bf3dbe538a27 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:59:17.577646 25368 raft_consensus.cc:3054] T 075f0030bd964a1687ccee531080fd1c P e6d654389db14a49a1f1bf3dbe538a27 [term 2 FOLLOWER]: Advancing to term 3
I20250109 08:59:17.584239 25368 raft_consensus.cc:513] T 075f0030bd964a1687ccee531080fd1c P e6d654389db14a49a1f1bf3dbe538a27 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:17.608690 25294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "075f0030bd964a1687ccee531080fd1c" candidate_uuid: "e6d654389db14a49a1f1bf3dbe538a27" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "9d26e8b39691431590fd77e182046c66"
I20250109 08:59:17.609325 25294 raft_consensus.cc:3049] T 075f0030bd964a1687ccee531080fd1c P 9d26e8b39691431590fd77e182046c66 [term 2 LEADER]: Stepping down as leader of term 2
I20250109 08:59:17.609647 25294 raft_consensus.cc:738] T 075f0030bd964a1687ccee531080fd1c P 9d26e8b39691431590fd77e182046c66 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9d26e8b39691431590fd77e182046c66, State: Running, Role: LEADER
I20250109 08:59:17.610391 25294 consensus_queue.cc:260] T 075f0030bd964a1687ccee531080fd1c P 9d26e8b39691431590fd77e182046c66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:17.617803 25656 tablet.cc:1721] T 3b37da4b59cf41b8889527eb2b0793c9 P 2ff0106754a84b39989b9e042b4af3d6: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
I20250109 08:59:17.611505 25294 raft_consensus.cc:3054] T 075f0030bd964a1687ccee531080fd1c P 9d26e8b39691431590fd77e182046c66 [term 2 FOLLOWER]: Advancing to term 3
I20250109 08:59:17.616463 25368 leader_election.cc:290] T 075f0030bd964a1687ccee531080fd1c P e6d654389db14a49a1f1bf3dbe538a27 [CANDIDATE]: Term 3 election: Requested vote from peers 2ff0106754a84b39989b9e042b4af3d6 (127.19.226.193:34345), 9d26e8b39691431590fd77e182046c66 (127.19.226.194:33163)
I20250109 08:59:17.628577 25221 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "075f0030bd964a1687ccee531080fd1c" candidate_uuid: "e6d654389db14a49a1f1bf3dbe538a27" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "2ff0106754a84b39989b9e042b4af3d6"
I20250109 08:59:17.632527 25655 tablet.cc:1721] T 3b37da4b59cf41b8889527eb2b0793c9 P e6d654389db14a49a1f1bf3dbe538a27: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
I20250109 08:59:17.635118 25650 tablet.cc:1721] T 9bf180fefb074fd192f6f80ae1460ba5 P e6d654389db14a49a1f1bf3dbe538a27: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
I20250109 08:59:17.635247 25636 tablet.cc:1721] T 8022fc0563b947669983f72b9b7450b3 P e6d654389db14a49a1f1bf3dbe538a27: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
I20250109 08:59:17.644523 25221 raft_consensus.cc:3054] T 075f0030bd964a1687ccee531080fd1c P 2ff0106754a84b39989b9e042b4af3d6 [term 2 FOLLOWER]: Advancing to term 3
I20250109 08:59:17.648507 25220 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "da8d4429a2364a3781155ecc9a3b18b3"
dest_uuid: "2ff0106754a84b39989b9e042b4af3d6"
mode: GRACEFUL
new_leader_uuid: "e6d654389db14a49a1f1bf3dbe538a27"
from {username='slave'} at 127.0.0.1:55380
I20250109 08:59:17.649207 25220 raft_consensus.cc:604] T da8d4429a2364a3781155ecc9a3b18b3 P 2ff0106754a84b39989b9e042b4af3d6 [term 1 LEADER]: Received request to transfer leadership to TS e6d654389db14a49a1f1bf3dbe538a27
I20250109 08:59:17.650672 20363 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 2
I20250109 08:59:17.650876 25294 raft_consensus.cc:2463] T 075f0030bd964a1687ccee531080fd1c P 9d26e8b39691431590fd77e182046c66 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6d654389db14a49a1f1bf3dbe538a27 in term 3.
I20250109 08:59:17.651322 20363 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250109 08:59:17.652380 25336 leader_election.cc:304] T 075f0030bd964a1687ccee531080fd1c P e6d654389db14a49a1f1bf3dbe538a27 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d26e8b39691431590fd77e182046c66, e6d654389db14a49a1f1bf3dbe538a27; no voters:
I20250109 08:59:17.659075 25622 raft_consensus.cc:2798] T 075f0030bd964a1687ccee531080fd1c P e6d654389db14a49a1f1bf3dbe538a27 [term 3 FOLLOWER]: Leader election won for term 3
I20250109 08:59:17.659615 25622 raft_consensus.cc:695] T 075f0030bd964a1687ccee531080fd1c P e6d654389db14a49a1f1bf3dbe538a27 [term 3 LEADER]: Becoming Leader. State: Replica: e6d654389db14a49a1f1bf3dbe538a27, State: Running, Role: LEADER
I20250109 08:59:17.660449 25622 consensus_queue.cc:237] T 075f0030bd964a1687ccee531080fd1c P e6d654389db14a49a1f1bf3dbe538a27 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:17.665608 25638 tablet.cc:1721] T cb8ed58651e74cbabff2b027ba4a9a1e P 2ff0106754a84b39989b9e042b4af3d6: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
I20250109 08:59:17.667901 25643 tablet.cc:1721] T cb8ed58651e74cbabff2b027ba4a9a1e P 9d26e8b39691431590fd77e182046c66: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
I20250109 08:59:17.671846 25221 raft_consensus.cc:2463] T 075f0030bd964a1687ccee531080fd1c P 2ff0106754a84b39989b9e042b4af3d6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6d654389db14a49a1f1bf3dbe538a27 in term 3.
I20250109 08:59:17.686360 25649 tablet.cc:1721] T ed0e3d98ad79424caf0e24ecf3b5f295 P 2ff0106754a84b39989b9e042b4af3d6: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
I20250109 08:59:17.696445 25636 tablet.cc:1721] T cb8ed58651e74cbabff2b027ba4a9a1e P e6d654389db14a49a1f1bf3dbe538a27: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
I20250109 08:59:17.712129 25119 catalog_manager.cc:5526] T 075f0030bd964a1687ccee531080fd1c P e6d654389db14a49a1f1bf3dbe538a27 reported cstate change: term changed from 2 to 3, leader changed from 9d26e8b39691431590fd77e182046c66 (127.19.226.194) to e6d654389db14a49a1f1bf3dbe538a27 (127.19.226.195). New cstate: current_term: 3 leader_uuid: "e6d654389db14a49a1f1bf3dbe538a27" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } health_report { overall_health: HEALTHY } } }
I20250109 08:59:17.715251 25120 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 7eb8c6a9d79149bc93b8c4660b491282: soft_deleted_table [id=66f95f53887e43ad8d999c406068e0a1] alter complete (version 1)
I20250109 08:59:17.796901 25654 raft_consensus.cc:988] T da8d4429a2364a3781155ecc9a3b18b3 P 2ff0106754a84b39989b9e042b4af3d6: : Instructing follower e6d654389db14a49a1f1bf3dbe538a27 to start an election
I20250109 08:59:17.797415 25652 raft_consensus.cc:1076] T da8d4429a2364a3781155ecc9a3b18b3 P 2ff0106754a84b39989b9e042b4af3d6 [term 1 LEADER]: Signalling peer e6d654389db14a49a1f1bf3dbe538a27 to start an election
I20250109 08:59:17.798754 25371 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "da8d4429a2364a3781155ecc9a3b18b3"
dest_uuid: "e6d654389db14a49a1f1bf3dbe538a27"
from {username='slave'} at 127.0.0.1:50366
I20250109 08:59:17.799180 25371 raft_consensus.cc:491] T da8d4429a2364a3781155ecc9a3b18b3 P e6d654389db14a49a1f1bf3dbe538a27 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250109 08:59:17.799397 25371 raft_consensus.cc:3054] T da8d4429a2364a3781155ecc9a3b18b3 P e6d654389db14a49a1f1bf3dbe538a27 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:59:17.803530 25371 raft_consensus.cc:513] T da8d4429a2364a3781155ecc9a3b18b3 P e6d654389db14a49a1f1bf3dbe538a27 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:17.804944 25371 leader_election.cc:290] T da8d4429a2364a3781155ecc9a3b18b3 P e6d654389db14a49a1f1bf3dbe538a27 [CANDIDATE]: Term 2 election: Requested vote from peers 2ff0106754a84b39989b9e042b4af3d6 (127.19.226.193:34345), 9d26e8b39691431590fd77e182046c66 (127.19.226.194:33163)
I20250109 08:59:17.805792 25221 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da8d4429a2364a3781155ecc9a3b18b3" candidate_uuid: "e6d654389db14a49a1f1bf3dbe538a27" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2ff0106754a84b39989b9e042b4af3d6"
I20250109 08:59:17.805984 25294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da8d4429a2364a3781155ecc9a3b18b3" candidate_uuid: "e6d654389db14a49a1f1bf3dbe538a27" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9d26e8b39691431590fd77e182046c66"
I20250109 08:59:17.806396 25221 raft_consensus.cc:3049] T da8d4429a2364a3781155ecc9a3b18b3 P 2ff0106754a84b39989b9e042b4af3d6 [term 1 LEADER]: Stepping down as leader of term 1
I20250109 08:59:17.806528 25294 raft_consensus.cc:3054] T da8d4429a2364a3781155ecc9a3b18b3 P 9d26e8b39691431590fd77e182046c66 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:59:17.806679 25221 raft_consensus.cc:738] T da8d4429a2364a3781155ecc9a3b18b3 P 2ff0106754a84b39989b9e042b4af3d6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2ff0106754a84b39989b9e042b4af3d6, State: Running, Role: LEADER
I20250109 08:59:17.807360 25221 consensus_queue.cc:260] T da8d4429a2364a3781155ecc9a3b18b3 P 2ff0106754a84b39989b9e042b4af3d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:17.808293 25221 raft_consensus.cc:3054] T da8d4429a2364a3781155ecc9a3b18b3 P 2ff0106754a84b39989b9e042b4af3d6 [term 1 FOLLOWER]: Advancing to term 2
I20250109 08:59:17.811491 25294 raft_consensus.cc:2463] T da8d4429a2364a3781155ecc9a3b18b3 P 9d26e8b39691431590fd77e182046c66 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6d654389db14a49a1f1bf3dbe538a27 in term 2.
I20250109 08:59:17.812670 25336 leader_election.cc:304] T da8d4429a2364a3781155ecc9a3b18b3 P e6d654389db14a49a1f1bf3dbe538a27 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d26e8b39691431590fd77e182046c66, e6d654389db14a49a1f1bf3dbe538a27; no voters:
I20250109 08:59:17.813369 25221 raft_consensus.cc:2463] T da8d4429a2364a3781155ecc9a3b18b3 P 2ff0106754a84b39989b9e042b4af3d6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6d654389db14a49a1f1bf3dbe538a27 in term 2.
I20250109 08:59:17.813457 25622 raft_consensus.cc:2798] T da8d4429a2364a3781155ecc9a3b18b3 P e6d654389db14a49a1f1bf3dbe538a27 [term 2 FOLLOWER]: Leader election won for term 2
I20250109 08:59:17.814204 25622 raft_consensus.cc:695] T da8d4429a2364a3781155ecc9a3b18b3 P e6d654389db14a49a1f1bf3dbe538a27 [term 2 LEADER]: Becoming Leader. State: Replica: e6d654389db14a49a1f1bf3dbe538a27, State: Running, Role: LEADER
I20250109 08:59:17.815007 25622 consensus_queue.cc:237] T da8d4429a2364a3781155ecc9a3b18b3 P e6d654389db14a49a1f1bf3dbe538a27 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:17.821797 25117 catalog_manager.cc:5526] T da8d4429a2364a3781155ecc9a3b18b3 P e6d654389db14a49a1f1bf3dbe538a27 reported cstate change: term changed from 1 to 2, leader changed from 2ff0106754a84b39989b9e042b4af3d6 (127.19.226.193) to e6d654389db14a49a1f1bf3dbe538a27 (127.19.226.195). New cstate: current_term: 2 leader_uuid: "e6d654389db14a49a1f1bf3dbe538a27" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } health_report { overall_health: HEALTHY } } }
I20250109 08:59:18.188594 25221 raft_consensus.cc:1270] T 075f0030bd964a1687ccee531080fd1c P 2ff0106754a84b39989b9e042b4af3d6 [term 3 FOLLOWER]: Refusing update from remote peer e6d654389db14a49a1f1bf3dbe538a27: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20250109 08:59:18.189706 25622 consensus_queue.cc:1035] T 075f0030bd964a1687ccee531080fd1c P e6d654389db14a49a1f1bf3dbe538a27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:59:18.195449 25294 raft_consensus.cc:1270] T da8d4429a2364a3781155ecc9a3b18b3 P 9d26e8b39691431590fd77e182046c66 [term 2 FOLLOWER]: Refusing update from remote peer e6d654389db14a49a1f1bf3dbe538a27: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:59:18.200863 25584 consensus_queue.cc:1035] T da8d4429a2364a3781155ecc9a3b18b3 P e6d654389db14a49a1f1bf3dbe538a27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250109 08:59:18.204777 25294 raft_consensus.cc:1270] T 075f0030bd964a1687ccee531080fd1c P 9d26e8b39691431590fd77e182046c66 [term 3 FOLLOWER]: Refusing update from remote peer e6d654389db14a49a1f1bf3dbe538a27: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20250109 08:59:18.208607 25584 consensus_queue.cc:1035] T 075f0030bd964a1687ccee531080fd1c P e6d654389db14a49a1f1bf3dbe538a27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250109 08:59:18.213066 25221 raft_consensus.cc:1270] T da8d4429a2364a3781155ecc9a3b18b3 P 2ff0106754a84b39989b9e042b4af3d6 [term 2 FOLLOWER]: Refusing update from remote peer e6d654389db14a49a1f1bf3dbe538a27: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250109 08:59:18.214906 25666 consensus_queue.cc:1035] T da8d4429a2364a3781155ecc9a3b18b3 P e6d654389db14a49a1f1bf3dbe538a27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20250109 08:59:18.652349 20363 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250109 08:59:18.658651 20363 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250109 08:59:18.659240 20363 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250109 08:59:19.660764 20363 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250109 08:59:19.678989 20363 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250109 08:59:19.679669 20363 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250109 08:59:20.680734 20363 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250109 08:59:20.685599 20363 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250109 08:59:20.686028 20363 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250109 08:59:21.789510 20363 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250109 08:59:22.046198 20363 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250109 08:59:22.046993 20363 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250109 08:59:22.572050 25682 raft_consensus.cc:491] T 3f4e7967061b4173bd00a9539a371fe3 P 2ff0106754a84b39989b9e042b4af3d6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9d26e8b39691431590fd77e182046c66)
I20250109 08:59:22.573210 25682 raft_consensus.cc:513] T 3f4e7967061b4173bd00a9539a371fe3 P 2ff0106754a84b39989b9e042b4af3d6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:22.576274 25682 leader_election.cc:290] T 3f4e7967061b4173bd00a9539a371fe3 P 2ff0106754a84b39989b9e042b4af3d6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9d26e8b39691431590fd77e182046c66 (127.19.226.194:33163), e6d654389db14a49a1f1bf3dbe538a27 (127.19.226.195:39911)
I20250109 08:59:22.679860 25677 raft_consensus.cc:491] T 9bf180fefb074fd192f6f80ae1460ba5 P 9d26e8b39691431590fd77e182046c66 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2ff0106754a84b39989b9e042b4af3d6)
I20250109 08:59:22.680625 25677 raft_consensus.cc:513] T 9bf180fefb074fd192f6f80ae1460ba5 P 9d26e8b39691431590fd77e182046c66 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:22.683913 25681 raft_consensus.cc:491] T da8d4429a2364a3781155ecc9a3b18b3 P 2ff0106754a84b39989b9e042b4af3d6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e6d654389db14a49a1f1bf3dbe538a27)
I20250109 08:59:22.684746 25681 raft_consensus.cc:513] T da8d4429a2364a3781155ecc9a3b18b3 P 2ff0106754a84b39989b9e042b4af3d6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:22.685578 25218 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bf180fefb074fd192f6f80ae1460ba5" candidate_uuid: "9d26e8b39691431590fd77e182046c66" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "2ff0106754a84b39989b9e042b4af3d6" is_pre_election: true
I20250109 08:59:22.687789 25681 leader_election.cc:290] T da8d4429a2364a3781155ecc9a3b18b3 P 2ff0106754a84b39989b9e042b4af3d6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9d26e8b39691431590fd77e182046c66 (127.19.226.194:33163), e6d654389db14a49a1f1bf3dbe538a27 (127.19.226.195:39911)
I20250109 08:59:22.688547 25676 raft_consensus.cc:491] T 7622fd22d50942618088f5e65422e713 P e6d654389db14a49a1f1bf3dbe538a27 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9d26e8b39691431590fd77e182046c66)
I20250109 08:59:22.689137 25676 raft_consensus.cc:513] T 7622fd22d50942618088f5e65422e713 P e6d654389db14a49a1f1bf3dbe538a27 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:22.702757 25676 leader_election.cc:290] T 7622fd22d50942618088f5e65422e713 P e6d654389db14a49a1f1bf3dbe538a27 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2ff0106754a84b39989b9e042b4af3d6 (127.19.226.193:34345), 9d26e8b39691431590fd77e182046c66 (127.19.226.194:33163)
I20250109 08:59:22.707859 25683 raft_consensus.cc:491] T 8022fc0563b947669983f72b9b7450b3 P 9d26e8b39691431590fd77e182046c66 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2ff0106754a84b39989b9e042b4af3d6)
I20250109 08:59:22.713505 25294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7622fd22d50942618088f5e65422e713" candidate_uuid: "e6d654389db14a49a1f1bf3dbe538a27" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9d26e8b39691431590fd77e182046c66" is_pre_election: true
I20250109 08:59:22.708530 25683 raft_consensus.cc:513] T 8022fc0563b947669983f72b9b7450b3 P 9d26e8b39691431590fd77e182046c66 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:22.713599 25677 leader_election.cc:290] T 9bf180fefb074fd192f6f80ae1460ba5 P 9d26e8b39691431590fd77e182046c66 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2ff0106754a84b39989b9e042b4af3d6 (127.19.226.193:34345), e6d654389db14a49a1f1bf3dbe538a27 (127.19.226.195:39911)
I20250109 08:59:22.717653 25683 leader_election.cc:290] T 8022fc0563b947669983f72b9b7450b3 P 9d26e8b39691431590fd77e182046c66 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2ff0106754a84b39989b9e042b4af3d6 (127.19.226.193:34345), e6d654389db14a49a1f1bf3dbe538a27 (127.19.226.195:39911)
I20250109 08:59:22.718468 25220 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8022fc0563b947669983f72b9b7450b3" candidate_uuid: "9d26e8b39691431590fd77e182046c66" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "2ff0106754a84b39989b9e042b4af3d6" is_pre_election: true
I20250109 08:59:22.719779 25220 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7622fd22d50942618088f5e65422e713" candidate_uuid: "e6d654389db14a49a1f1bf3dbe538a27" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2ff0106754a84b39989b9e042b4af3d6" is_pre_election: true
I20250109 08:59:22.720713 25687 raft_consensus.cc:491] T 346220ee65da44b8862c9f28d7aa6aa3 P e6d654389db14a49a1f1bf3dbe538a27 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9d26e8b39691431590fd77e182046c66)
I20250109 08:59:22.721225 25335 leader_election.cc:304] T 7622fd22d50942618088f5e65422e713 P e6d654389db14a49a1f1bf3dbe538a27 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e6d654389db14a49a1f1bf3dbe538a27; no voters: 2ff0106754a84b39989b9e042b4af3d6, 9d26e8b39691431590fd77e182046c66
I20250109 08:59:22.721415 25687 raft_consensus.cc:513] T 346220ee65da44b8862c9f28d7aa6aa3 P e6d654389db14a49a1f1bf3dbe538a27 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:22.723799 25686 raft_consensus.cc:2743] T 7622fd22d50942618088f5e65422e713 P e6d654389db14a49a1f1bf3dbe538a27 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20250109 08:59:22.724012 25687 leader_election.cc:290] T 346220ee65da44b8862c9f28d7aa6aa3 P e6d654389db14a49a1f1bf3dbe538a27 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2ff0106754a84b39989b9e042b4af3d6 (127.19.226.193:34345), 9d26e8b39691431590fd77e182046c66 (127.19.226.194:33163)
I20250109 08:59:22.724656 25294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "346220ee65da44b8862c9f28d7aa6aa3" candidate_uuid: "e6d654389db14a49a1f1bf3dbe538a27" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "9d26e8b39691431590fd77e182046c66" is_pre_election: true
I20250109 08:59:22.725482 25368 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bf180fefb074fd192f6f80ae1460ba5" candidate_uuid: "9d26e8b39691431590fd77e182046c66" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "e6d654389db14a49a1f1bf3dbe538a27" is_pre_election: true
I20250109 08:59:22.726121 25367 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8022fc0563b947669983f72b9b7450b3" candidate_uuid: "9d26e8b39691431590fd77e182046c66" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "e6d654389db14a49a1f1bf3dbe538a27" is_pre_election: true
I20250109 08:59:22.726704 25367 raft_consensus.cc:2463] T 8022fc0563b947669983f72b9b7450b3 P e6d654389db14a49a1f1bf3dbe538a27 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d26e8b39691431590fd77e182046c66 in term 2.
I20250109 08:59:22.727440 25220 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "346220ee65da44b8862c9f28d7aa6aa3" candidate_uuid: "e6d654389db14a49a1f1bf3dbe538a27" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "2ff0106754a84b39989b9e042b4af3d6" is_pre_election: true
I20250109 08:59:22.728910 25335 leader_election.cc:304] T 346220ee65da44b8862c9f28d7aa6aa3 P e6d654389db14a49a1f1bf3dbe538a27 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e6d654389db14a49a1f1bf3dbe538a27; no voters: 2ff0106754a84b39989b9e042b4af3d6, 9d26e8b39691431590fd77e182046c66
I20250109 08:59:22.730122 25367 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f4e7967061b4173bd00a9539a371fe3" candidate_uuid: "2ff0106754a84b39989b9e042b4af3d6" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "e6d654389db14a49a1f1bf3dbe538a27" is_pre_election: true
I20250109 08:59:22.731360 25687 raft_consensus.cc:2743] T 346220ee65da44b8862c9f28d7aa6aa3 P e6d654389db14a49a1f1bf3dbe538a27 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20250109 08:59:22.732723 25294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f4e7967061b4173bd00a9539a371fe3" candidate_uuid: "2ff0106754a84b39989b9e042b4af3d6" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "9d26e8b39691431590fd77e182046c66" is_pre_election: true
I20250109 08:59:22.734359 25368 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da8d4429a2364a3781155ecc9a3b18b3" candidate_uuid: "2ff0106754a84b39989b9e042b4af3d6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e6d654389db14a49a1f1bf3dbe538a27" is_pre_election: true
I20250109 08:59:22.734807 25292 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da8d4429a2364a3781155ecc9a3b18b3" candidate_uuid: "2ff0106754a84b39989b9e042b4af3d6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9d26e8b39691431590fd77e182046c66" is_pre_election: true
I20250109 08:59:22.737115 25263 leader_election.cc:304] T 9bf180fefb074fd192f6f80ae1460ba5 P 9d26e8b39691431590fd77e182046c66 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9d26e8b39691431590fd77e182046c66; no voters: 2ff0106754a84b39989b9e042b4af3d6, e6d654389db14a49a1f1bf3dbe538a27
I20250109 08:59:22.738204 25263 leader_election.cc:304] T 8022fc0563b947669983f72b9b7450b3 P 9d26e8b39691431590fd77e182046c66 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9d26e8b39691431590fd77e182046c66, e6d654389db14a49a1f1bf3dbe538a27; no voters: 2ff0106754a84b39989b9e042b4af3d6
I20250109 08:59:22.738989 25691 raft_consensus.cc:2743] T 9bf180fefb074fd192f6f80ae1460ba5 P 9d26e8b39691431590fd77e182046c66 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20250109 08:59:22.739228 25677 raft_consensus.cc:2798] T 8022fc0563b947669983f72b9b7450b3 P 9d26e8b39691431590fd77e182046c66 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20250109 08:59:22.739651 25677 raft_consensus.cc:491] T 8022fc0563b947669983f72b9b7450b3 P 9d26e8b39691431590fd77e182046c66 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 2ff0106754a84b39989b9e042b4af3d6)
I20250109 08:59:22.739979 25677 raft_consensus.cc:3054] T 8022fc0563b947669983f72b9b7450b3 P 9d26e8b39691431590fd77e182046c66 [term 2 FOLLOWER]: Advancing to term 3
I20250109 08:59:22.740242 25187 leader_election.cc:304] T 3f4e7967061b4173bd00a9539a371fe3 P 2ff0106754a84b39989b9e042b4af3d6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ff0106754a84b39989b9e042b4af3d6; no voters: 9d26e8b39691431590fd77e182046c66, e6d654389db14a49a1f1bf3dbe538a27
I20250109 08:59:22.741205 25682 raft_consensus.cc:2743] T 3f4e7967061b4173bd00a9539a371fe3 P 2ff0106754a84b39989b9e042b4af3d6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20250109 08:59:22.748725 25187 leader_election.cc:304] T da8d4429a2364a3781155ecc9a3b18b3 P 2ff0106754a84b39989b9e042b4af3d6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ff0106754a84b39989b9e042b4af3d6; no voters: 9d26e8b39691431590fd77e182046c66, e6d654389db14a49a1f1bf3dbe538a27
I20250109 08:59:22.749480 25682 raft_consensus.cc:2743] T da8d4429a2364a3781155ecc9a3b18b3 P 2ff0106754a84b39989b9e042b4af3d6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20250109 08:59:22.750216 25677 raft_consensus.cc:513] T 8022fc0563b947669983f72b9b7450b3 P 9d26e8b39691431590fd77e182046c66 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:22.752074 25292 raft_consensus.cc:1235] T 8022fc0563b947669983f72b9b7450b3 P 9d26e8b39691431590fd77e182046c66 [term 3 FOLLOWER]: Rejecting Update request from peer 2ff0106754a84b39989b9e042b4af3d6 for earlier term 2. Current term is 3. Ops: []
I20250109 08:59:22.752141 25677 leader_election.cc:290] T 8022fc0563b947669983f72b9b7450b3 P 9d26e8b39691431590fd77e182046c66 [CANDIDATE]: Term 3 election: Requested vote from peers 2ff0106754a84b39989b9e042b4af3d6 (127.19.226.193:34345), e6d654389db14a49a1f1bf3dbe538a27 (127.19.226.195:39911)
I20250109 08:59:22.752761 25220 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8022fc0563b947669983f72b9b7450b3" candidate_uuid: "9d26e8b39691431590fd77e182046c66" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "2ff0106754a84b39989b9e042b4af3d6"
I20250109 08:59:22.753048 25368 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8022fc0563b947669983f72b9b7450b3" candidate_uuid: "9d26e8b39691431590fd77e182046c66" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "e6d654389db14a49a1f1bf3dbe538a27"
I20250109 08:59:22.753811 25682 consensus_queue.cc:1046] T 8022fc0563b947669983f72b9b7450b3 P 2ff0106754a84b39989b9e042b4af3d6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 }, Status: INVALID_TERM, Last received: 2.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250109 08:59:22.754321 25263 leader_election.cc:304] T 8022fc0563b947669983f72b9b7450b3 P 9d26e8b39691431590fd77e182046c66 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9d26e8b39691431590fd77e182046c66; no voters: 2ff0106754a84b39989b9e042b4af3d6, e6d654389db14a49a1f1bf3dbe538a27
I20250109 08:59:22.754714 25681 raft_consensus.cc:3049] T 8022fc0563b947669983f72b9b7450b3 P 2ff0106754a84b39989b9e042b4af3d6 [term 2 LEADER]: Stepping down as leader of term 2
I20250109 08:59:22.755049 25677 raft_consensus.cc:2743] T 8022fc0563b947669983f72b9b7450b3 P 9d26e8b39691431590fd77e182046c66 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20250109 08:59:22.755129 25681 raft_consensus.cc:738] T 8022fc0563b947669983f72b9b7450b3 P 2ff0106754a84b39989b9e042b4af3d6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 2ff0106754a84b39989b9e042b4af3d6, State: Running, Role: LEADER
I20250109 08:59:22.756104 25681 consensus_queue.cc:260] T 8022fc0563b947669983f72b9b7450b3 P 2ff0106754a84b39989b9e042b4af3d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:22.757105 25681 raft_consensus.cc:3054] T 8022fc0563b947669983f72b9b7450b3 P 2ff0106754a84b39989b9e042b4af3d6 [term 2 FOLLOWER]: Advancing to term 3
W20250109 08:59:22.938431 25151 debug-util.cc:398] Leaking SignalData structure 0x7b080003a8e0 after lost signal to thread 20366
W20250109 08:59:22.939450 25151 debug-util.cc:398] Leaking SignalData structure 0x7b080008e980 after lost signal to thread 25154
W20250109 08:59:22.940487 25151 debug-util.cc:398] Leaking SignalData structure 0x7b0800274d40 after lost signal to thread 25246
W20250109 08:59:22.941394 25151 debug-util.cc:398] Leaking SignalData structure 0x7b08000cbfe0 after lost signal to thread 25321
W20250109 08:59:22.942257 25151 debug-util.cc:398] Leaking SignalData structure 0x7b08001813e0 after lost signal to thread 25395
W20250109 08:59:22.942581 25151 debug-util.cc:398] Leaking SignalData structure 0x7b0800260a00 after lost signal to thread 25680
I20250109 08:59:23.048139 20363 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250109 08:59:23.055852 20363 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250109 08:59:23.056468 20363 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250109 08:59:24.057390 20363 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250109 08:59:24.065001 20363 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250109 08:59:24.065392 20363 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250109 08:59:24.230924 25687 raft_consensus.cc:491] T 8022fc0563b947669983f72b9b7450b3 P e6d654389db14a49a1f1bf3dbe538a27 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2ff0106754a84b39989b9e042b4af3d6)
I20250109 08:59:24.231532 25687 raft_consensus.cc:513] T 8022fc0563b947669983f72b9b7450b3 P e6d654389db14a49a1f1bf3dbe538a27 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:24.233778 25687 leader_election.cc:290] T 8022fc0563b947669983f72b9b7450b3 P e6d654389db14a49a1f1bf3dbe538a27 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2ff0106754a84b39989b9e042b4af3d6 (127.19.226.193:34345), 9d26e8b39691431590fd77e182046c66 (127.19.226.194:33163)
I20250109 08:59:24.234231 25218 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8022fc0563b947669983f72b9b7450b3" candidate_uuid: "e6d654389db14a49a1f1bf3dbe538a27" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "2ff0106754a84b39989b9e042b4af3d6" is_pre_election: true
I20250109 08:59:24.234532 25296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8022fc0563b947669983f72b9b7450b3" candidate_uuid: "e6d654389db14a49a1f1bf3dbe538a27" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "9d26e8b39691431590fd77e182046c66" is_pre_election: true
I20250109 08:59:24.235015 25218 raft_consensus.cc:2463] T 8022fc0563b947669983f72b9b7450b3 P 2ff0106754a84b39989b9e042b4af3d6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6d654389db14a49a1f1bf3dbe538a27 in term 3.
I20250109 08:59:24.236500 25335 leader_election.cc:304] T 8022fc0563b947669983f72b9b7450b3 P e6d654389db14a49a1f1bf3dbe538a27 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ff0106754a84b39989b9e042b4af3d6, e6d654389db14a49a1f1bf3dbe538a27; no voters:
I20250109 08:59:24.237396 25296 raft_consensus.cc:2388] T 8022fc0563b947669983f72b9b7450b3 P 9d26e8b39691431590fd77e182046c66 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e6d654389db14a49a1f1bf3dbe538a27 in current term 3: Already voted for candidate 9d26e8b39691431590fd77e182046c66 in this term.
I20250109 08:59:24.238432 25687 raft_consensus.cc:2798] T 8022fc0563b947669983f72b9b7450b3 P e6d654389db14a49a1f1bf3dbe538a27 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20250109 08:59:24.238857 25687 raft_consensus.cc:491] T 8022fc0563b947669983f72b9b7450b3 P e6d654389db14a49a1f1bf3dbe538a27 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 2ff0106754a84b39989b9e042b4af3d6)
I20250109 08:59:24.239451 25687 raft_consensus.cc:3054] T 8022fc0563b947669983f72b9b7450b3 P e6d654389db14a49a1f1bf3dbe538a27 [term 2 FOLLOWER]: Advancing to term 3
I20250109 08:59:24.247192 25687 raft_consensus.cc:513] T 8022fc0563b947669983f72b9b7450b3 P e6d654389db14a49a1f1bf3dbe538a27 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:24.249526 25687 leader_election.cc:290] T 8022fc0563b947669983f72b9b7450b3 P e6d654389db14a49a1f1bf3dbe538a27 [CANDIDATE]: Term 3 election: Requested vote from peers 2ff0106754a84b39989b9e042b4af3d6 (127.19.226.193:34345), 9d26e8b39691431590fd77e182046c66 (127.19.226.194:33163)
I20250109 08:59:24.249573 25218 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8022fc0563b947669983f72b9b7450b3" candidate_uuid: "e6d654389db14a49a1f1bf3dbe538a27" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "2ff0106754a84b39989b9e042b4af3d6"
I20250109 08:59:24.249778 25296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8022fc0563b947669983f72b9b7450b3" candidate_uuid: "e6d654389db14a49a1f1bf3dbe538a27" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "9d26e8b39691431590fd77e182046c66"
I20250109 08:59:24.250785 25296 raft_consensus.cc:2388] T 8022fc0563b947669983f72b9b7450b3 P 9d26e8b39691431590fd77e182046c66 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e6d654389db14a49a1f1bf3dbe538a27 in current term 3: Already voted for candidate 9d26e8b39691431590fd77e182046c66 in this term.
I20250109 08:59:24.255393 25218 raft_consensus.cc:2463] T 8022fc0563b947669983f72b9b7450b3 P 2ff0106754a84b39989b9e042b4af3d6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6d654389db14a49a1f1bf3dbe538a27 in term 3.
I20250109 08:59:24.256466 25335 leader_election.cc:304] T 8022fc0563b947669983f72b9b7450b3 P e6d654389db14a49a1f1bf3dbe538a27 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2ff0106754a84b39989b9e042b4af3d6, e6d654389db14a49a1f1bf3dbe538a27; no voters: 9d26e8b39691431590fd77e182046c66
I20250109 08:59:24.257131 25687 raft_consensus.cc:2798] T 8022fc0563b947669983f72b9b7450b3 P e6d654389db14a49a1f1bf3dbe538a27 [term 3 FOLLOWER]: Leader election won for term 3
I20250109 08:59:24.257622 25687 raft_consensus.cc:695] T 8022fc0563b947669983f72b9b7450b3 P e6d654389db14a49a1f1bf3dbe538a27 [term 3 LEADER]: Becoming Leader. State: Replica: e6d654389db14a49a1f1bf3dbe538a27, State: Running, Role: LEADER
I20250109 08:59:24.258669 25687 consensus_queue.cc:237] T 8022fc0563b947669983f72b9b7450b3 P e6d654389db14a49a1f1bf3dbe538a27 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:24.270416 25119 catalog_manager.cc:5526] T 8022fc0563b947669983f72b9b7450b3 P e6d654389db14a49a1f1bf3dbe538a27 reported cstate change: term changed from 2 to 3, leader changed from 2ff0106754a84b39989b9e042b4af3d6 (127.19.226.193) to e6d654389db14a49a1f1bf3dbe538a27 (127.19.226.195). New cstate: current_term: 3 leader_uuid: "e6d654389db14a49a1f1bf3dbe538a27" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } health_report { overall_health: HEALTHY } } }
I20250109 08:59:24.348562 25694 raft_consensus.cc:491] T 8022fc0563b947669983f72b9b7450b3 P 9d26e8b39691431590fd77e182046c66 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250109 08:59:24.349013 25694 raft_consensus.cc:513] T 8022fc0563b947669983f72b9b7450b3 P 9d26e8b39691431590fd77e182046c66 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:24.350795 25694 leader_election.cc:290] T 8022fc0563b947669983f72b9b7450b3 P 9d26e8b39691431590fd77e182046c66 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2ff0106754a84b39989b9e042b4af3d6 (127.19.226.193:34345), e6d654389db14a49a1f1bf3dbe538a27 (127.19.226.195:39911)
I20250109 08:59:24.351653 25218 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8022fc0563b947669983f72b9b7450b3" candidate_uuid: "9d26e8b39691431590fd77e182046c66" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "2ff0106754a84b39989b9e042b4af3d6" is_pre_election: true
I20250109 08:59:24.351938 25367 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8022fc0563b947669983f72b9b7450b3" candidate_uuid: "9d26e8b39691431590fd77e182046c66" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "e6d654389db14a49a1f1bf3dbe538a27" is_pre_election: true
I20250109 08:59:24.352429 25218 raft_consensus.cc:2463] T 8022fc0563b947669983f72b9b7450b3 P 2ff0106754a84b39989b9e042b4af3d6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d26e8b39691431590fd77e182046c66 in term 3.
I20250109 08:59:24.353595 25261 leader_election.cc:304] T 8022fc0563b947669983f72b9b7450b3 P 9d26e8b39691431590fd77e182046c66 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2ff0106754a84b39989b9e042b4af3d6, 9d26e8b39691431590fd77e182046c66; no voters: e6d654389db14a49a1f1bf3dbe538a27
I20250109 08:59:24.354380 25694 raft_consensus.cc:2798] T 8022fc0563b947669983f72b9b7450b3 P 9d26e8b39691431590fd77e182046c66 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20250109 08:59:24.354662 25694 raft_consensus.cc:491] T 8022fc0563b947669983f72b9b7450b3 P 9d26e8b39691431590fd77e182046c66 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250109 08:59:24.354902 25694 raft_consensus.cc:3054] T 8022fc0563b947669983f72b9b7450b3 P 9d26e8b39691431590fd77e182046c66 [term 3 FOLLOWER]: Advancing to term 4
I20250109 08:59:24.360232 25694 raft_consensus.cc:513] T 8022fc0563b947669983f72b9b7450b3 P 9d26e8b39691431590fd77e182046c66 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:24.361764 25694 leader_election.cc:290] T 8022fc0563b947669983f72b9b7450b3 P 9d26e8b39691431590fd77e182046c66 [CANDIDATE]: Term 4 election: Requested vote from peers 2ff0106754a84b39989b9e042b4af3d6 (127.19.226.193:34345), e6d654389db14a49a1f1bf3dbe538a27 (127.19.226.195:39911)
I20250109 08:59:24.362673 25218 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8022fc0563b947669983f72b9b7450b3" candidate_uuid: "9d26e8b39691431590fd77e182046c66" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "2ff0106754a84b39989b9e042b4af3d6"
I20250109 08:59:24.362808 25367 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8022fc0563b947669983f72b9b7450b3" candidate_uuid: "9d26e8b39691431590fd77e182046c66" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "e6d654389db14a49a1f1bf3dbe538a27"
I20250109 08:59:24.363229 25218 raft_consensus.cc:3054] T 8022fc0563b947669983f72b9b7450b3 P 2ff0106754a84b39989b9e042b4af3d6 [term 3 FOLLOWER]: Advancing to term 4
I20250109 08:59:24.368168 25218 raft_consensus.cc:2463] T 8022fc0563b947669983f72b9b7450b3 P 2ff0106754a84b39989b9e042b4af3d6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d26e8b39691431590fd77e182046c66 in term 4.
I20250109 08:59:24.369249 25261 leader_election.cc:304] T 8022fc0563b947669983f72b9b7450b3 P 9d26e8b39691431590fd77e182046c66 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2ff0106754a84b39989b9e042b4af3d6, 9d26e8b39691431590fd77e182046c66; no voters: e6d654389db14a49a1f1bf3dbe538a27
I20250109 08:59:24.369907 25694 raft_consensus.cc:2798] T 8022fc0563b947669983f72b9b7450b3 P 9d26e8b39691431590fd77e182046c66 [term 4 FOLLOWER]: Leader election won for term 4
I20250109 08:59:24.370416 25694 raft_consensus.cc:695] T 8022fc0563b947669983f72b9b7450b3 P 9d26e8b39691431590fd77e182046c66 [term 4 LEADER]: Becoming Leader. State: Replica: 9d26e8b39691431590fd77e182046c66, State: Running, Role: LEADER
I20250109 08:59:24.371395 25694 consensus_queue.cc:237] T 8022fc0563b947669983f72b9b7450b3 P 9d26e8b39691431590fd77e182046c66 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:24.381930 25119 catalog_manager.cc:5526] T 8022fc0563b947669983f72b9b7450b3 P 9d26e8b39691431590fd77e182046c66 reported cstate change: term changed from 3 to 4, leader changed from e6d654389db14a49a1f1bf3dbe538a27 (127.19.226.195) to 9d26e8b39691431590fd77e182046c66 (127.19.226.194). New cstate: current_term: 4 leader_uuid: "9d26e8b39691431590fd77e182046c66" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } health_report { overall_health: UNKNOWN } } }
I20250109 08:59:24.658581 25218 raft_consensus.cc:1235] T 8022fc0563b947669983f72b9b7450b3 P 2ff0106754a84b39989b9e042b4af3d6 [term 4 FOLLOWER]: Rejecting Update request from peer e6d654389db14a49a1f1bf3dbe538a27 for earlier term 3. Current term is 4. Ops: []
I20250109 08:59:24.659813 25687 consensus_queue.cc:1046] T 8022fc0563b947669983f72b9b7450b3 P e6d654389db14a49a1f1bf3dbe538a27 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250109 08:59:24.660476 25689 raft_consensus.cc:3049] T 8022fc0563b947669983f72b9b7450b3 P e6d654389db14a49a1f1bf3dbe538a27 [term 3 LEADER]: Stepping down as leader of term 3
I20250109 08:59:24.660838 25689 raft_consensus.cc:738] T 8022fc0563b947669983f72b9b7450b3 P e6d654389db14a49a1f1bf3dbe538a27 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e6d654389db14a49a1f1bf3dbe538a27, State: Running, Role: LEADER
I20250109 08:59:24.661614 25689 consensus_queue.cc:260] T 8022fc0563b947669983f72b9b7450b3 P e6d654389db14a49a1f1bf3dbe538a27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.4, Last appended by leader: 4, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 } } peers { permanent_uuid: "9d26e8b39691431590fd77e182046c66" member_type: VOTER last_known_addr { host: "127.19.226.194" port: 33163 } } peers { permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 } }
I20250109 08:59:24.662835 25689 raft_consensus.cc:3054] T 8022fc0563b947669983f72b9b7450b3 P e6d654389db14a49a1f1bf3dbe538a27 [term 3 FOLLOWER]: Advancing to term 4
I20250109 08:59:24.753861 25367 raft_consensus.cc:1270] T 8022fc0563b947669983f72b9b7450b3 P e6d654389db14a49a1f1bf3dbe538a27 [term 4 FOLLOWER]: Refusing update from remote peer 9d26e8b39691431590fd77e182046c66: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 4 index: 4. (term mismatch)
I20250109 08:59:24.754233 25367 pending_rounds.cc:77] T 8022fc0563b947669983f72b9b7450b3 P e6d654389db14a49a1f1bf3dbe538a27: Aborting all ops after (but not including) 3
I20250109 08:59:24.754451 25367 pending_rounds.cc:99] T 8022fc0563b947669983f72b9b7450b3 P e6d654389db14a49a1f1bf3dbe538a27: Aborting uncommitted NO_OP operation due to leader change: 3.4
I20250109 08:59:24.754725 25367 raft_consensus.cc:2883] T 8022fc0563b947669983f72b9b7450b3 P e6d654389db14a49a1f1bf3dbe538a27 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20250109 08:59:24.755978 25694 consensus_queue.cc:1035] T 8022fc0563b947669983f72b9b7450b3 P 9d26e8b39691431590fd77e182046c66 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6d654389db14a49a1f1bf3dbe538a27" member_type: VOTER last_known_addr { host: "127.19.226.195" port: 39911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250109 08:59:24.776974 25218 raft_consensus.cc:1270] T 8022fc0563b947669983f72b9b7450b3 P 2ff0106754a84b39989b9e042b4af3d6 [term 4 FOLLOWER]: Refusing update from remote peer 9d26e8b39691431590fd77e182046c66: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20250109 08:59:24.778486 25703 consensus_queue.cc:1035] T 8022fc0563b947669983f72b9b7450b3 P 9d26e8b39691431590fd77e182046c66 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ff0106754a84b39989b9e042b4af3d6" member_type: VOTER last_known_addr { host: "127.19.226.193" port: 34345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20250109 08:59:25.066133 20363 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250109 08:59:25.071159 20363 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250109 08:59:25.071555 20363 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250109 08:59:26.072283 20363 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250109 08:59:26.077448 20363 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250109 08:59:26.077860 20363 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250109 08:59:27.078603 20363 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250109 08:59:27.083503 20363 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250109 08:59:27.083914 20363 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
/home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/master/auto_leader_rebalancer-test.cc:596: Failure
Expected: (leader.second) != (3), actual: 3 vs 3
I20250109 08:59:28.135015 20363 tablet_server.cc:178] TabletServer@127.19.226.193:0 shutting down...
W20250109 08:59:28.145965 25335 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 7 similar messages]
W20250109 08:59:28.149909 25335 consensus_peers.cc:487] T 422f918bde184ce1ae2532c9dee609a7 P e6d654389db14a49a1f1bf3dbe538a27 -> Peer 2ff0106754a84b39989b9e042b4af3d6 (127.19.226.193:34345): Couldn't send request to peer 2ff0106754a84b39989b9e042b4af3d6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:59:28.158310 20363 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250109 08:59:28.159150 20363 tablet_replica.cc:331] T 51277cd5f9374666bc60232309efc100 P 2ff0106754a84b39989b9e042b4af3d6: stopping tablet replica
I20250109 08:59:28.160166 20363 raft_consensus.cc:2238] T 51277cd5f9374666bc60232309efc100 P 2ff0106754a84b39989b9e042b4af3d6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:59:28.160877 20363 raft_consensus.cc:2267] T 51277cd5f9374666bc60232309efc100 P 2ff0106754a84b39989b9e042b4af3d6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:28.163177 20363 tablet_replica.cc:331] T 9a3240fe2e094ba2bc002579bdb29e84 P 2ff0106754a84b39989b9e042b4af3d6: stopping tablet replica
I20250109 08:59:28.163844 20363 raft_consensus.cc:2238] T 9a3240fe2e094ba2bc002579bdb29e84 P 2ff0106754a84b39989b9e042b4af3d6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:59:28.164430 20363 raft_consensus.cc:2267] T 9a3240fe2e094ba2bc002579bdb29e84 P 2ff0106754a84b39989b9e042b4af3d6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:28.166939 20363 tablet_replica.cc:331] T 3b37da4b59cf41b8889527eb2b0793c9 P 2ff0106754a84b39989b9e042b4af3d6: stopping tablet replica
I20250109 08:59:28.168656 20363 raft_consensus.cc:2238] T 3b37da4b59cf41b8889527eb2b0793c9 P 2ff0106754a84b39989b9e042b4af3d6 [term 2 LEADER]: Raft consensus shutting down.
I20250109 08:59:28.169685 20363 raft_consensus.cc:2267] T 3b37da4b59cf41b8889527eb2b0793c9 P 2ff0106754a84b39989b9e042b4af3d6 [term 2 FOLLOWER]: Raft consensus is shut down!
W20250109 08:59:28.170898 25335 consensus_peers.cc:487] T da8d4429a2364a3781155ecc9a3b18b3 P e6d654389db14a49a1f1bf3dbe538a27 -> Peer 2ff0106754a84b39989b9e042b4af3d6 (127.19.226.193:34345): Couldn't send request to peer 2ff0106754a84b39989b9e042b4af3d6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:59:28.172283 20363 tablet_replica.cc:331] T cb8ed58651e74cbabff2b027ba4a9a1e P 2ff0106754a84b39989b9e042b4af3d6: stopping tablet replica
I20250109 08:59:28.173074 20363 raft_consensus.cc:2238] T cb8ed58651e74cbabff2b027ba4a9a1e P 2ff0106754a84b39989b9e042b4af3d6 [term 2 LEADER]: Raft consensus shutting down.
I20250109 08:59:28.173878 20363 raft_consensus.cc:2267] T cb8ed58651e74cbabff2b027ba4a9a1e P 2ff0106754a84b39989b9e042b4af3d6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:28.175891 20363 tablet_replica.cc:331] T 9bf180fefb074fd192f6f80ae1460ba5 P 2ff0106754a84b39989b9e042b4af3d6: stopping tablet replica
I20250109 08:59:28.176512 20363 raft_consensus.cc:2238] T 9bf180fefb074fd192f6f80ae1460ba5 P 2ff0106754a84b39989b9e042b4af3d6 [term 2 LEADER]: Raft consensus shutting down.
I20250109 08:59:28.177277 20363 raft_consensus.cc:2267] T 9bf180fefb074fd192f6f80ae1460ba5 P 2ff0106754a84b39989b9e042b4af3d6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:28.179222 20363 tablet_replica.cc:331] T 9f829cc37f3b447a800c8ee3c51bd346 P 2ff0106754a84b39989b9e042b4af3d6: stopping tablet replica
I20250109 08:59:28.179744 20363 raft_consensus.cc:2238] T 9f829cc37f3b447a800c8ee3c51bd346 P 2ff0106754a84b39989b9e042b4af3d6 [term 2 LEADER]: Raft consensus shutting down.
I20250109 08:59:28.180546 20363 raft_consensus.cc:2267] T 9f829cc37f3b447a800c8ee3c51bd346 P 2ff0106754a84b39989b9e042b4af3d6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:28.182600 20363 tablet_replica.cc:331] T 709470e8d22046dd87f2a20e8a8ce745 P 2ff0106754a84b39989b9e042b4af3d6: stopping tablet replica
I20250109 08:59:28.183084 20363 raft_consensus.cc:2238] T 709470e8d22046dd87f2a20e8a8ce745 P 2ff0106754a84b39989b9e042b4af3d6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:59:28.183498 20363 raft_consensus.cc:2267] T 709470e8d22046dd87f2a20e8a8ce745 P 2ff0106754a84b39989b9e042b4af3d6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:28.185504 20363 tablet_replica.cc:331] T ed0e3d98ad79424caf0e24ecf3b5f295 P 2ff0106754a84b39989b9e042b4af3d6: stopping tablet replica
I20250109 08:59:28.186030 20363 raft_consensus.cc:2238] T ed0e3d98ad79424caf0e24ecf3b5f295 P 2ff0106754a84b39989b9e042b4af3d6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:59:28.186501 20363 raft_consensus.cc:2267] T ed0e3d98ad79424caf0e24ecf3b5f295 P 2ff0106754a84b39989b9e042b4af3d6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:28.188491 20363 tablet_replica.cc:331] T c16639dc193644c8bbcf05ed9f02295e P 2ff0106754a84b39989b9e042b4af3d6: stopping tablet replica
I20250109 08:59:28.189013 20363 raft_consensus.cc:2238] T c16639dc193644c8bbcf05ed9f02295e P 2ff0106754a84b39989b9e042b4af3d6 [term 2 LEADER]: Raft consensus shutting down.
I20250109 08:59:28.189764 20363 raft_consensus.cc:2267] T c16639dc193644c8bbcf05ed9f02295e P 2ff0106754a84b39989b9e042b4af3d6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:28.191617 20363 tablet_replica.cc:331] T 7622fd22d50942618088f5e65422e713 P 2ff0106754a84b39989b9e042b4af3d6: stopping tablet replica
I20250109 08:59:28.192080 20363 raft_consensus.cc:2238] T 7622fd22d50942618088f5e65422e713 P 2ff0106754a84b39989b9e042b4af3d6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:59:28.192493 20363 raft_consensus.cc:2267] T 7622fd22d50942618088f5e65422e713 P 2ff0106754a84b39989b9e042b4af3d6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:28.194397 20363 tablet_replica.cc:331] T 3f4e7967061b4173bd00a9539a371fe3 P 2ff0106754a84b39989b9e042b4af3d6: stopping tablet replica
I20250109 08:59:28.194836 20363 raft_consensus.cc:2238] T 3f4e7967061b4173bd00a9539a371fe3 P 2ff0106754a84b39989b9e042b4af3d6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:59:28.195241 20363 raft_consensus.cc:2267] T 3f4e7967061b4173bd00a9539a371fe3 P 2ff0106754a84b39989b9e042b4af3d6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:28.197446 20363 tablet_replica.cc:331] T da8d4429a2364a3781155ecc9a3b18b3 P 2ff0106754a84b39989b9e042b4af3d6: stopping tablet replica
I20250109 08:59:28.197875 20363 raft_consensus.cc:2238] T da8d4429a2364a3781155ecc9a3b18b3 P 2ff0106754a84b39989b9e042b4af3d6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:59:28.198241 20363 raft_consensus.cc:2267] T da8d4429a2364a3781155ecc9a3b18b3 P 2ff0106754a84b39989b9e042b4af3d6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:28.200192 20363 tablet_replica.cc:331] T 346220ee65da44b8862c9f28d7aa6aa3 P 2ff0106754a84b39989b9e042b4af3d6: stopping tablet replica
I20250109 08:59:28.200700 20363 raft_consensus.cc:2238] T 346220ee65da44b8862c9f28d7aa6aa3 P 2ff0106754a84b39989b9e042b4af3d6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:59:28.201129 20363 raft_consensus.cc:2267] T 346220ee65da44b8862c9f28d7aa6aa3 P 2ff0106754a84b39989b9e042b4af3d6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:28.203119 20363 tablet_replica.cc:331] T 075f0030bd964a1687ccee531080fd1c P 2ff0106754a84b39989b9e042b4af3d6: stopping tablet replica
I20250109 08:59:28.203604 20363 raft_consensus.cc:2238] T 075f0030bd964a1687ccee531080fd1c P 2ff0106754a84b39989b9e042b4af3d6 [term 3 FOLLOWER]: Raft consensus shutting down.
I20250109 08:59:28.204010 20363 raft_consensus.cc:2267] T 075f0030bd964a1687ccee531080fd1c P 2ff0106754a84b39989b9e042b4af3d6 [term 3 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:28.205861 20363 tablet_replica.cc:331] T 8022fc0563b947669983f72b9b7450b3 P 2ff0106754a84b39989b9e042b4af3d6: stopping tablet replica
I20250109 08:59:28.206341 20363 raft_consensus.cc:2238] T 8022fc0563b947669983f72b9b7450b3 P 2ff0106754a84b39989b9e042b4af3d6 [term 4 FOLLOWER]: Raft consensus shutting down.
I20250109 08:59:28.206674 20363 raft_consensus.cc:2267] T 8022fc0563b947669983f72b9b7450b3 P 2ff0106754a84b39989b9e042b4af3d6 [term 4 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:28.208449 20363 tablet_replica.cc:331] T 422f918bde184ce1ae2532c9dee609a7 P 2ff0106754a84b39989b9e042b4af3d6: stopping tablet replica
I20250109 08:59:28.208974 20363 raft_consensus.cc:2238] T 422f918bde184ce1ae2532c9dee609a7 P 2ff0106754a84b39989b9e042b4af3d6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:59:28.209409 20363 raft_consensus.cc:2267] T 422f918bde184ce1ae2532c9dee609a7 P 2ff0106754a84b39989b9e042b4af3d6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:28.211241 20363 tablet_replica.cc:331] T 486c1bbdc3e247cdbbf13e1f7456e729 P 2ff0106754a84b39989b9e042b4af3d6: stopping tablet replica
I20250109 08:59:28.211683 20363 raft_consensus.cc:2238] T 486c1bbdc3e247cdbbf13e1f7456e729 P 2ff0106754a84b39989b9e042b4af3d6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:59:28.212071 20363 raft_consensus.cc:2267] T 486c1bbdc3e247cdbbf13e1f7456e729 P 2ff0106754a84b39989b9e042b4af3d6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:28.213898 20363 tablet_replica.cc:331] T 1d01a6ffe04e4c7bae266bbc80ee5d54 P 2ff0106754a84b39989b9e042b4af3d6: stopping tablet replica
I20250109 08:59:28.214323 20363 raft_consensus.cc:2238] T 1d01a6ffe04e4c7bae266bbc80ee5d54 P 2ff0106754a84b39989b9e042b4af3d6 [term 2 LEADER]: Raft consensus shutting down.
I20250109 08:59:28.215051 20363 raft_consensus.cc:2267] T 1d01a6ffe04e4c7bae266bbc80ee5d54 P 2ff0106754a84b39989b9e042b4af3d6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:28.259640 20363 tablet_server.cc:195] TabletServer@127.19.226.193:0 shutdown complete.
W20250109 08:59:28.262756 25261 consensus_peers.cc:487] T 486c1bbdc3e247cdbbf13e1f7456e729 P 9d26e8b39691431590fd77e182046c66 -> Peer 2ff0106754a84b39989b9e042b4af3d6 (127.19.226.193:34345): Couldn't send request to peer 2ff0106754a84b39989b9e042b4af3d6. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.193:34345: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20250109 08:59:28.296628 25261 consensus_peers.cc:487] T 709470e8d22046dd87f2a20e8a8ce745 P 9d26e8b39691431590fd77e182046c66 -> Peer 2ff0106754a84b39989b9e042b4af3d6 (127.19.226.193:34345): Couldn't send request to peer 2ff0106754a84b39989b9e042b4af3d6. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.193:34345: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20250109 08:59:28.301007 25261 consensus_peers.cc:487] T 346220ee65da44b8862c9f28d7aa6aa3 P 9d26e8b39691431590fd77e182046c66 -> Peer 2ff0106754a84b39989b9e042b4af3d6 (127.19.226.193:34345): Couldn't send request to peer 2ff0106754a84b39989b9e042b4af3d6. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.193:34345: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250109 08:59:28.302189 20363 tablet_server.cc:178] TabletServer@127.19.226.194:0 shutting down...
W20250109 08:59:28.308955 25335 consensus_peers.cc:487] T 075f0030bd964a1687ccee531080fd1c P e6d654389db14a49a1f1bf3dbe538a27 -> Peer 2ff0106754a84b39989b9e042b4af3d6 (127.19.226.193:34345): Couldn't send request to peer 2ff0106754a84b39989b9e042b4af3d6. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.193:34345: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20250109 08:59:28.327014 25261 consensus_peers.cc:487] T 8022fc0563b947669983f72b9b7450b3 P 9d26e8b39691431590fd77e182046c66 -> Peer 2ff0106754a84b39989b9e042b4af3d6 (127.19.226.193:34345): Couldn't send request to peer 2ff0106754a84b39989b9e042b4af3d6. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.193:34345: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250109 08:59:28.331382 20363 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250109 08:59:28.332051 20363 tablet_replica.cc:331] T 3b37da4b59cf41b8889527eb2b0793c9 P 9d26e8b39691431590fd77e182046c66: stopping tablet replica
I20250109 08:59:28.332803 20363 raft_consensus.cc:2238] T 3b37da4b59cf41b8889527eb2b0793c9 P 9d26e8b39691431590fd77e182046c66 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:59:28.333330 20363 raft_consensus.cc:2267] T 3b37da4b59cf41b8889527eb2b0793c9 P 9d26e8b39691431590fd77e182046c66 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:28.335373 20363 tablet_replica.cc:331] T cb8ed58651e74cbabff2b027ba4a9a1e P 9d26e8b39691431590fd77e182046c66: stopping tablet replica
I20250109 08:59:28.335919 20363 raft_consensus.cc:2238] T cb8ed58651e74cbabff2b027ba4a9a1e P 9d26e8b39691431590fd77e182046c66 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:59:28.336345 20363 raft_consensus.cc:2267] T cb8ed58651e74cbabff2b027ba4a9a1e P 9d26e8b39691431590fd77e182046c66 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:28.338259 20363 tablet_replica.cc:331] T 9bf180fefb074fd192f6f80ae1460ba5 P 9d26e8b39691431590fd77e182046c66: stopping tablet replica
I20250109 08:59:28.338874 20363 raft_consensus.cc:2238] T 9bf180fefb074fd192f6f80ae1460ba5 P 9d26e8b39691431590fd77e182046c66 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:59:28.339349 20363 raft_consensus.cc:2267] T 9bf180fefb074fd192f6f80ae1460ba5 P 9d26e8b39691431590fd77e182046c66 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:28.341287 20363 tablet_replica.cc:331] T 9a3240fe2e094ba2bc002579bdb29e84 P 9d26e8b39691431590fd77e182046c66: stopping tablet replica
I20250109 08:59:28.341789 20363 raft_consensus.cc:2238] T 9a3240fe2e094ba2bc002579bdb29e84 P 9d26e8b39691431590fd77e182046c66 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:59:28.342238 20363 raft_consensus.cc:2267] T 9a3240fe2e094ba2bc002579bdb29e84 P 9d26e8b39691431590fd77e182046c66 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:28.344097 20363 tablet_replica.cc:331] T 51277cd5f9374666bc60232309efc100 P 9d26e8b39691431590fd77e182046c66: stopping tablet replica
I20250109 08:59:28.344689 20363 raft_consensus.cc:2238] T 51277cd5f9374666bc60232309efc100 P 9d26e8b39691431590fd77e182046c66 [term 2 LEADER]: Raft consensus shutting down.
I20250109 08:59:28.345504 20363 raft_consensus.cc:2267] T 51277cd5f9374666bc60232309efc100 P 9d26e8b39691431590fd77e182046c66 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:28.347525 20363 tablet_replica.cc:331] T 486c1bbdc3e247cdbbf13e1f7456e729 P 9d26e8b39691431590fd77e182046c66: stopping tablet replica
I20250109 08:59:28.348052 20363 raft_consensus.cc:2238] T 486c1bbdc3e247cdbbf13e1f7456e729 P 9d26e8b39691431590fd77e182046c66 [term 2 LEADER]: Raft consensus shutting down.
I20250109 08:59:28.348834 20363 raft_consensus.cc:2267] T 486c1bbdc3e247cdbbf13e1f7456e729 P 9d26e8b39691431590fd77e182046c66 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:28.350795 20363 tablet_replica.cc:331] T 3f4e7967061b4173bd00a9539a371fe3 P 9d26e8b39691431590fd77e182046c66: stopping tablet replica
I20250109 08:59:28.351253 20363 raft_consensus.cc:2238] T 3f4e7967061b4173bd00a9539a371fe3 P 9d26e8b39691431590fd77e182046c66 [term 2 LEADER]: Raft consensus shutting down.
I20250109 08:59:28.351955 20363 raft_consensus.cc:2267] T 3f4e7967061b4173bd00a9539a371fe3 P 9d26e8b39691431590fd77e182046c66 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:28.353827 20363 tablet_replica.cc:331] T da8d4429a2364a3781155ecc9a3b18b3 P 9d26e8b39691431590fd77e182046c66: stopping tablet replica
I20250109 08:59:28.354379 20363 raft_consensus.cc:2238] T da8d4429a2364a3781155ecc9a3b18b3 P 9d26e8b39691431590fd77e182046c66 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:59:28.354748 20363 raft_consensus.cc:2267] T da8d4429a2364a3781155ecc9a3b18b3 P 9d26e8b39691431590fd77e182046c66 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:28.356696 20363 tablet_replica.cc:331] T ed0e3d98ad79424caf0e24ecf3b5f295 P 9d26e8b39691431590fd77e182046c66: stopping tablet replica
I20250109 08:59:28.357213 20363 raft_consensus.cc:2238] T ed0e3d98ad79424caf0e24ecf3b5f295 P 9d26e8b39691431590fd77e182046c66 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:59:28.357607 20363 raft_consensus.cc:2267] T ed0e3d98ad79424caf0e24ecf3b5f295 P 9d26e8b39691431590fd77e182046c66 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:28.359757 20363 tablet_replica.cc:331] T 709470e8d22046dd87f2a20e8a8ce745 P 9d26e8b39691431590fd77e182046c66: stopping tablet replica
I20250109 08:59:28.360494 20363 raft_consensus.cc:2238] T 709470e8d22046dd87f2a20e8a8ce745 P 9d26e8b39691431590fd77e182046c66 [term 2 LEADER]: Raft consensus shutting down.
I20250109 08:59:28.361367 20363 raft_consensus.cc:2267] T 709470e8d22046dd87f2a20e8a8ce745 P 9d26e8b39691431590fd77e182046c66 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:28.363644 20363 tablet_replica.cc:331] T c16639dc193644c8bbcf05ed9f02295e P 9d26e8b39691431590fd77e182046c66: stopping tablet replica
I20250109 08:59:28.364347 20363 raft_consensus.cc:2238] T c16639dc193644c8bbcf05ed9f02295e P 9d26e8b39691431590fd77e182046c66 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:59:28.364949 20363 raft_consensus.cc:2267] T c16639dc193644c8bbcf05ed9f02295e P 9d26e8b39691431590fd77e182046c66 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:28.367576 20363 tablet_replica.cc:331] T 9f829cc37f3b447a800c8ee3c51bd346 P 9d26e8b39691431590fd77e182046c66: stopping tablet replica
I20250109 08:59:28.368096 20363 raft_consensus.cc:2238] T 9f829cc37f3b447a800c8ee3c51bd346 P 9d26e8b39691431590fd77e182046c66 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:59:28.368525 20363 raft_consensus.cc:2267] T 9f829cc37f3b447a800c8ee3c51bd346 P 9d26e8b39691431590fd77e182046c66 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:28.370453 20363 tablet_replica.cc:331] T 346220ee65da44b8862c9f28d7aa6aa3 P 9d26e8b39691431590fd77e182046c66: stopping tablet replica
I20250109 08:59:28.370961 20363 raft_consensus.cc:2238] T 346220ee65da44b8862c9f28d7aa6aa3 P 9d26e8b39691431590fd77e182046c66 [term 2 LEADER]: Raft consensus shutting down.
I20250109 08:59:28.371726 20363 raft_consensus.cc:2267] T 346220ee65da44b8862c9f28d7aa6aa3 P 9d26e8b39691431590fd77e182046c66 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:28.374128 20363 tablet_replica.cc:331] T 075f0030bd964a1687ccee531080fd1c P 9d26e8b39691431590fd77e182046c66: stopping tablet replica
I20250109 08:59:28.374689 20363 raft_consensus.cc:2238] T 075f0030bd964a1687ccee531080fd1c P 9d26e8b39691431590fd77e182046c66 [term 3 FOLLOWER]: Raft consensus shutting down.
I20250109 08:59:28.375140 20363 raft_consensus.cc:2267] T 075f0030bd964a1687ccee531080fd1c P 9d26e8b39691431590fd77e182046c66 [term 3 FOLLOWER]: Raft consensus is shut down!
W20250109 08:59:28.376300 25336 consensus_peers.cc:487] T 422f918bde184ce1ae2532c9dee609a7 P e6d654389db14a49a1f1bf3dbe538a27 -> Peer 9d26e8b39691431590fd77e182046c66 (127.19.226.194:33163): Couldn't send request to peer 9d26e8b39691431590fd77e182046c66. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250109 08:59:28.377277 20363 tablet_replica.cc:331] T 8022fc0563b947669983f72b9b7450b3 P 9d26e8b39691431590fd77e182046c66: stopping tablet replica
I20250109 08:59:28.377905 20363 raft_consensus.cc:2238] T 8022fc0563b947669983f72b9b7450b3 P 9d26e8b39691431590fd77e182046c66 [term 4 LEADER]: Raft consensus shutting down.
I20250109 08:59:28.378633 20363 raft_consensus.cc:2267] T 8022fc0563b947669983f72b9b7450b3 P 9d26e8b39691431590fd77e182046c66 [term 4 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:28.380527 20363 tablet_replica.cc:331] T 422f918bde184ce1ae2532c9dee609a7 P 9d26e8b39691431590fd77e182046c66: stopping tablet replica
I20250109 08:59:28.381052 20363 raft_consensus.cc:2238] T 422f918bde184ce1ae2532c9dee609a7 P 9d26e8b39691431590fd77e182046c66 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:59:28.381455 20363 raft_consensus.cc:2267] T 422f918bde184ce1ae2532c9dee609a7 P 9d26e8b39691431590fd77e182046c66 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:28.383409 20363 tablet_replica.cc:331] T 7622fd22d50942618088f5e65422e713 P 9d26e8b39691431590fd77e182046c66: stopping tablet replica
I20250109 08:59:28.383905 20363 raft_consensus.cc:2238] T 7622fd22d50942618088f5e65422e713 P 9d26e8b39691431590fd77e182046c66 [term 2 LEADER]: Raft consensus shutting down.
I20250109 08:59:28.384722 20363 raft_consensus.cc:2267] T 7622fd22d50942618088f5e65422e713 P 9d26e8b39691431590fd77e182046c66 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:28.386710 20363 tablet_replica.cc:331] T 1d01a6ffe04e4c7bae266bbc80ee5d54 P 9d26e8b39691431590fd77e182046c66: stopping tablet replica
I20250109 08:59:28.387168 20363 raft_consensus.cc:2238] T 1d01a6ffe04e4c7bae266bbc80ee5d54 P 9d26e8b39691431590fd77e182046c66 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:59:28.387691 20363 raft_consensus.cc:2267] T 1d01a6ffe04e4c7bae266bbc80ee5d54 P 9d26e8b39691431590fd77e182046c66 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:28.426399 20363 tablet_server.cc:195] TabletServer@127.19.226.194:0 shutdown complete.
W20250109 08:59:28.431512 25335 consensus_peers.cc:487] T 9a3240fe2e094ba2bc002579bdb29e84 P e6d654389db14a49a1f1bf3dbe538a27 -> Peer 2ff0106754a84b39989b9e042b4af3d6 (127.19.226.193:34345): Couldn't send request to peer 2ff0106754a84b39989b9e042b4af3d6. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.193:34345: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250109 08:59:28.461921 20363 tablet_server.cc:178] TabletServer@127.19.226.195:0 shutting down...
I20250109 08:59:28.487267 20363 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250109 08:59:28.488078 20363 tablet_replica.cc:331] T 51277cd5f9374666bc60232309efc100 P e6d654389db14a49a1f1bf3dbe538a27: stopping tablet replica
I20250109 08:59:28.488899 20363 raft_consensus.cc:2238] T 51277cd5f9374666bc60232309efc100 P e6d654389db14a49a1f1bf3dbe538a27 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:59:28.489504 20363 raft_consensus.cc:2267] T 51277cd5f9374666bc60232309efc100 P e6d654389db14a49a1f1bf3dbe538a27 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:28.491808 20363 tablet_replica.cc:331] T 9a3240fe2e094ba2bc002579bdb29e84 P e6d654389db14a49a1f1bf3dbe538a27: stopping tablet replica
W20250109 08:59:28.492024 25336 consensus_peers.cc:487] T 075f0030bd964a1687ccee531080fd1c P e6d654389db14a49a1f1bf3dbe538a27 -> Peer 9d26e8b39691431590fd77e182046c66 (127.19.226.194:33163): Couldn't send request to peer 9d26e8b39691431590fd77e182046c66. Status: Network error: Client connection negotiation failed: client connection to 127.19.226.194:33163: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250109 08:59:28.492503 20363 raft_consensus.cc:2238] T 9a3240fe2e094ba2bc002579bdb29e84 P e6d654389db14a49a1f1bf3dbe538a27 [term 2 LEADER]: Raft consensus shutting down.
I20250109 08:59:28.493350 20363 raft_consensus.cc:2267] T 9a3240fe2e094ba2bc002579bdb29e84 P e6d654389db14a49a1f1bf3dbe538a27 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:28.495229 20363 tablet_replica.cc:331] T cb8ed58651e74cbabff2b027ba4a9a1e P e6d654389db14a49a1f1bf3dbe538a27: stopping tablet replica
I20250109 08:59:28.495774 20363 raft_consensus.cc:2238] T cb8ed58651e74cbabff2b027ba4a9a1e P e6d654389db14a49a1f1bf3dbe538a27 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:59:28.496193 20363 raft_consensus.cc:2267] T cb8ed58651e74cbabff2b027ba4a9a1e P e6d654389db14a49a1f1bf3dbe538a27 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:28.498036 20363 tablet_replica.cc:331] T 9bf180fefb074fd192f6f80ae1460ba5 P e6d654389db14a49a1f1bf3dbe538a27: stopping tablet replica
I20250109 08:59:28.498526 20363 raft_consensus.cc:2238] T 9bf180fefb074fd192f6f80ae1460ba5 P e6d654389db14a49a1f1bf3dbe538a27 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:59:28.498976 20363 raft_consensus.cc:2267] T 9bf180fefb074fd192f6f80ae1460ba5 P e6d654389db14a49a1f1bf3dbe538a27 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:28.500931 20363 tablet_replica.cc:331] T 3f4e7967061b4173bd00a9539a371fe3 P e6d654389db14a49a1f1bf3dbe538a27: stopping tablet replica
I20250109 08:59:28.501441 20363 raft_consensus.cc:2238] T 3f4e7967061b4173bd00a9539a371fe3 P e6d654389db14a49a1f1bf3dbe538a27 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:59:28.501878 20363 raft_consensus.cc:2267] T 3f4e7967061b4173bd00a9539a371fe3 P e6d654389db14a49a1f1bf3dbe538a27 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:28.503614 20363 tablet_replica.cc:331] T da8d4429a2364a3781155ecc9a3b18b3 P e6d654389db14a49a1f1bf3dbe538a27: stopping tablet replica
I20250109 08:59:28.504108 20363 raft_consensus.cc:2238] T da8d4429a2364a3781155ecc9a3b18b3 P e6d654389db14a49a1f1bf3dbe538a27 [term 2 LEADER]: Raft consensus shutting down.
I20250109 08:59:28.504889 20363 raft_consensus.cc:2267] T da8d4429a2364a3781155ecc9a3b18b3 P e6d654389db14a49a1f1bf3dbe538a27 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:28.506675 20363 tablet_replica.cc:331] T 1d01a6ffe04e4c7bae266bbc80ee5d54 P e6d654389db14a49a1f1bf3dbe538a27: stopping tablet replica
I20250109 08:59:28.507179 20363 raft_consensus.cc:2238] T 1d01a6ffe04e4c7bae266bbc80ee5d54 P e6d654389db14a49a1f1bf3dbe538a27 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:59:28.507575 20363 raft_consensus.cc:2267] T 1d01a6ffe04e4c7bae266bbc80ee5d54 P e6d654389db14a49a1f1bf3dbe538a27 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:28.509413 20363 tablet_replica.cc:331] T 346220ee65da44b8862c9f28d7aa6aa3 P e6d654389db14a49a1f1bf3dbe538a27: stopping tablet replica
I20250109 08:59:28.509934 20363 raft_consensus.cc:2238] T 346220ee65da44b8862c9f28d7aa6aa3 P e6d654389db14a49a1f1bf3dbe538a27 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:59:28.510413 20363 raft_consensus.cc:2267] T 346220ee65da44b8862c9f28d7aa6aa3 P e6d654389db14a49a1f1bf3dbe538a27 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:28.512095 20363 tablet_replica.cc:331] T 075f0030bd964a1687ccee531080fd1c P e6d654389db14a49a1f1bf3dbe538a27: stopping tablet replica
I20250109 08:59:28.512617 20363 raft_consensus.cc:2238] T 075f0030bd964a1687ccee531080fd1c P e6d654389db14a49a1f1bf3dbe538a27 [term 3 LEADER]: Raft consensus shutting down.
I20250109 08:59:28.513373 20363 raft_consensus.cc:2267] T 075f0030bd964a1687ccee531080fd1c P e6d654389db14a49a1f1bf3dbe538a27 [term 3 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:28.515378 20363 tablet_replica.cc:331] T 8022fc0563b947669983f72b9b7450b3 P e6d654389db14a49a1f1bf3dbe538a27: stopping tablet replica
I20250109 08:59:28.515901 20363 raft_consensus.cc:2238] T 8022fc0563b947669983f72b9b7450b3 P e6d654389db14a49a1f1bf3dbe538a27 [term 4 FOLLOWER]: Raft consensus shutting down.
I20250109 08:59:28.516343 20363 raft_consensus.cc:2267] T 8022fc0563b947669983f72b9b7450b3 P e6d654389db14a49a1f1bf3dbe538a27 [term 4 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:28.518236 20363 tablet_replica.cc:331] T 422f918bde184ce1ae2532c9dee609a7 P e6d654389db14a49a1f1bf3dbe538a27: stopping tablet replica
I20250109 08:59:28.518719 20363 raft_consensus.cc:2238] T 422f918bde184ce1ae2532c9dee609a7 P e6d654389db14a49a1f1bf3dbe538a27 [term 2 LEADER]: Raft consensus shutting down.
I20250109 08:59:28.519606 20363 raft_consensus.cc:2267] T 422f918bde184ce1ae2532c9dee609a7 P e6d654389db14a49a1f1bf3dbe538a27 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:28.522413 20363 tablet_replica.cc:331] T ed0e3d98ad79424caf0e24ecf3b5f295 P e6d654389db14a49a1f1bf3dbe538a27: stopping tablet replica
I20250109 08:59:28.523170 20363 raft_consensus.cc:2238] T ed0e3d98ad79424caf0e24ecf3b5f295 P e6d654389db14a49a1f1bf3dbe538a27 [term 2 LEADER]: Raft consensus shutting down.
I20250109 08:59:28.524139 20363 raft_consensus.cc:2267] T ed0e3d98ad79424caf0e24ecf3b5f295 P e6d654389db14a49a1f1bf3dbe538a27 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:28.527012 20363 tablet_replica.cc:331] T c16639dc193644c8bbcf05ed9f02295e P e6d654389db14a49a1f1bf3dbe538a27: stopping tablet replica
I20250109 08:59:28.527761 20363 raft_consensus.cc:2238] T c16639dc193644c8bbcf05ed9f02295e P e6d654389db14a49a1f1bf3dbe538a27 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:59:28.528389 20363 raft_consensus.cc:2267] T c16639dc193644c8bbcf05ed9f02295e P e6d654389db14a49a1f1bf3dbe538a27 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:28.531057 20363 tablet_replica.cc:331] T 709470e8d22046dd87f2a20e8a8ce745 P e6d654389db14a49a1f1bf3dbe538a27: stopping tablet replica
I20250109 08:59:28.531773 20363 raft_consensus.cc:2238] T 709470e8d22046dd87f2a20e8a8ce745 P e6d654389db14a49a1f1bf3dbe538a27 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:59:28.532298 20363 raft_consensus.cc:2267] T 709470e8d22046dd87f2a20e8a8ce745 P e6d654389db14a49a1f1bf3dbe538a27 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:28.534888 20363 tablet_replica.cc:331] T 3b37da4b59cf41b8889527eb2b0793c9 P e6d654389db14a49a1f1bf3dbe538a27: stopping tablet replica
I20250109 08:59:28.535583 20363 raft_consensus.cc:2238] T 3b37da4b59cf41b8889527eb2b0793c9 P e6d654389db14a49a1f1bf3dbe538a27 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:59:28.536185 20363 raft_consensus.cc:2267] T 3b37da4b59cf41b8889527eb2b0793c9 P e6d654389db14a49a1f1bf3dbe538a27 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:28.538928 20363 tablet_replica.cc:331] T 7622fd22d50942618088f5e65422e713 P e6d654389db14a49a1f1bf3dbe538a27: stopping tablet replica
I20250109 08:59:28.539613 20363 raft_consensus.cc:2238] T 7622fd22d50942618088f5e65422e713 P e6d654389db14a49a1f1bf3dbe538a27 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:59:28.540156 20363 raft_consensus.cc:2267] T 7622fd22d50942618088f5e65422e713 P e6d654389db14a49a1f1bf3dbe538a27 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:28.542832 20363 tablet_replica.cc:331] T 486c1bbdc3e247cdbbf13e1f7456e729 P e6d654389db14a49a1f1bf3dbe538a27: stopping tablet replica
I20250109 08:59:28.543457 20363 raft_consensus.cc:2238] T 486c1bbdc3e247cdbbf13e1f7456e729 P e6d654389db14a49a1f1bf3dbe538a27 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:59:28.543884 20363 raft_consensus.cc:2267] T 486c1bbdc3e247cdbbf13e1f7456e729 P e6d654389db14a49a1f1bf3dbe538a27 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:28.546037 20363 tablet_replica.cc:331] T 9f829cc37f3b447a800c8ee3c51bd346 P e6d654389db14a49a1f1bf3dbe538a27: stopping tablet replica
I20250109 08:59:28.546469 20363 raft_consensus.cc:2238] T 9f829cc37f3b447a800c8ee3c51bd346 P e6d654389db14a49a1f1bf3dbe538a27 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250109 08:59:28.546860 20363 raft_consensus.cc:2267] T 9f829cc37f3b447a800c8ee3c51bd346 P e6d654389db14a49a1f1bf3dbe538a27 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:28.586205 20363 tablet_server.cc:195] TabletServer@127.19.226.195:0 shutdown complete.
I20250109 08:59:28.618094 20363 master.cc:537] Master@127.19.226.254:43781 shutting down...
I20250109 08:59:28.639613 20363 raft_consensus.cc:2238] T 00000000000000000000000000000000 P 7eb8c6a9d79149bc93b8c4660b491282 [term 1 LEADER]: Raft consensus shutting down.
I20250109 08:59:28.640170 20363 raft_consensus.cc:2267] T 00000000000000000000000000000000 P 7eb8c6a9d79149bc93b8c4660b491282 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250109 08:59:28.640511 20363 tablet_replica.cc:331] T 00000000000000000000000000000000 P 7eb8c6a9d79149bc93b8c4660b491282: stopping tablet replica
I20250109 08:59:28.676815 20363 master.cc:559] Master@127.19.226.254:43781 shutdown complete.
I20250109 08:59:28.708743 20363 test_util.cc:181] -----------------------------------------------
I20250109 08:59:28.708949 20363 test_util.cc:182] Had failures, leaving test files at /tmp/dist-test-taskPnoZ2B/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1736412987538469-20363-0
[ FAILED ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1, where GetParam() = true (20387 ms)
[----------] 2 tests from FilterSoftDeletedTableTest (40963 ms total)
[----------] Global test environment tear-down
[==========] 7 tests from 2 test suites ran. (181012 ms total)
[ PASSED ] 5 tests.
[ SKIPPED ] 1 test, listed below:
[ SKIPPED ] LeaderRebalancerTest.TestMaintenanceMode
[ FAILED ] 1 test, listed below:
[ FAILED ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1, where GetParam() = true
1 FAILED TEST
I20250109 08:59:28.715233 20363 logging.cc:424] LogThrottler /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/tablet/tablet_mm_ops.cc:220: suppressed but not reported on 2163 messages since previous log ~31 seconds ago
I20250109 08:59:28.715462 20363 logging.cc:424] LogThrottler /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/tablet/tablet_mm_ops.cc:304: suppressed but not reported on 2162 messages since previous log ~31 seconds ago
I20250109 08:59:28.715629 20363 logging.cc:424] LogThrottler /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/tablet/tablet_mm_ops.cc:139: suppressed but not reported on 2159 messages since previous log ~31 seconds ago
I20250109 08:59:28.715804 20363 logging.cc:424] LogThrottler /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/tserver/ts_tablet_manager.cc:594: suppressed but not reported on 44 messages since previous log ~58 seconds ago
I20250109 08:59:28.715962 20363 logging.cc:424] LogThrottler /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/tserver/ts_tablet_manager.cc:542: suppressed but not reported on 44 messages since previous log ~58 seconds ago
I20250109 08:59:28.716197 20363 logging.cc:424] LogThrottler /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/proxy.cc:239: suppressed but not reported on 9 messages since previous log ~0 seconds ago
I20250109 08:59:28.716528 20363 logging.cc:424] LogThrottler /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/tablet/tablet.cc:2367: suppressed but not reported on 92 messages since previous log ~39 seconds ago